Data security detection method and device, electronic equipment and storage medium

Abstract

This invention provides a data security detection method, apparatus, electronic device, and storage medium. The method includes: extracting a preset number of sample data from user-uploaded data to be stored; classifying multiple data items in the sample data based on data type, and determining corresponding binning methods based on different classifications; determining a target binning method whose stability meets preset conditions; binning the data to be stored using the target binning method; performing security detection on the data to be stored in each bin, and storing the data that has passed the security detection when the detection values ​​of each bin are not less than a preset threshold. By performing security detection on the data before uploading, and storing the data only after the security detection is passed, the strength of data security control is enhanced, ensuring the effectiveness of data security control. Furthermore, by binning the data and then performing security detection on the binned data, the efficiency of data security detection is improved.

Description

Technical Field

[0001] This invention relates to the field of data security technology, and in particular to a data security detection method, a data security detection device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the advent of the big data era and the development of cloud computing and deep learning, more efficient data processing technologies have brought higher productivity, but have also triggered data security crises, especially in operation and maintenance systems and data auditing systems. Data breaches can result in significant losses. Therefore, data security has become an indispensable part of big data, and ensuring data security is receiving increasing attention.

[0003] Existing solutions only detect data access through technologies such as firewalls and authentication. Their core is to detect access behavior, but they do not involve the security detection of the data itself, thus weakening the strength of data security control and failing to guarantee the effectiveness of data security control. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a data security detection method that overcomes or at least partially solves the above problems.

[0005] This invention also provides a data security detection device, an electronic device, and a storage medium to ensure the implementation of the above methods.

[0006] To address the above problems, this invention discloses a data security detection method, the method comprising:

[0007] Receive user-uploaded data to be stored, and extract a preset number of sample data from the data to be stored; the sample data includes multiple data items;

[0008] The multiple data items are classified according to their data types to obtain different types of data items;

[0009] Determine the binning method corresponding to each of the different types of data items;

[0010] From the binning methods corresponding to the different types of data items, determine the target binning method whose stability meets the preset conditions;

[0011] The data to be stored is binned using the target binning method described above.

[0012] Perform security checks on the data to be stored in each bin and obtain the check values ​​for each bin;

[0013] When the detection values ​​of each bin are greater than or equal to a preset threshold, the data to be stored is determined to have passed the security test, and the data to be stored that has passed the security test is stored.

[0014] Optionally, the data type includes numeric and non-numeric types; the classification of the multiple data items based on data type to obtain different types of data items includes:

[0015] From the plurality of data items, determine data items that do not contain non-numeric values, and determine data items that contain non-numeric values;

[0016] The data items that do not contain non-numeric values ​​are defined as the numeric data items, and the data items that contain non-numeric values ​​are defined as the non-numeric data items.

[0017] Optionally, determining the binning method corresponding to the different types of data items includes:

[0018] For the numerical data item, extract multiple first initial data values ​​from the numerical data item;

[0019] Sort the initial data values.

[0020] The sorted initial data values ​​are binned at equal intervals to obtain the first target data value for each bin.

[0021] Verify whether the first target data values ​​in the same bin have the same meaning;

[0022] If so, then the equidistant binning is determined as the binning method corresponding to the numerical data item.

[0023] Optionally, after verifying whether the first target data values ​​in the same bin have the same meaning, the method further includes:

[0024] If not, then the first target data values ​​of each sub-bin are divided into equal-frequency sub-bins to obtain the third target data values ​​of each sub-bin;

[0025] Verify whether the third target data values ​​in the same bin have the same meaning;

[0026] If so, then the equidistant binning and the equal-frequency binning are determined as the binning methods corresponding to the numerical data items.

[0027] Optionally, sorting the first initial data values ​​includes:

[0028] Obtain the encoding rules corresponding to the numerical data items;

[0029] According to the encoding rules, each first initial data value is divided to obtain multiple sets of sub-data corresponding to each first initial data value;

[0030] For each of the first initial data values, a first score is obtained for each group; the first score is used to characterize the correlation between each group of sub-data and the data item to which it belongs;

[0031] For each of the first initial data values, the multiple groups of sub-data are sorted based on the first score.

[0032] Optionally, determining the binning method corresponding to the different types of data items includes:

[0033] For the non-numerical data item, extract multiple second initial data values ​​from the non-numerical data item;

[0034] The plurality of second initial data values ​​are classified to obtain different types of second initial data values;

[0035] Sort the second initial data values ​​of the different types respectively;

[0036] The sorted initial data values ​​are binned at equal intervals to obtain the target data values ​​for each bin.

[0037] Verify whether the second target data values ​​in the same bin have the same meaning;

[0038] If so, then the equidistant binning is determined as the binning method corresponding to the non-numerical data item.

[0039] Optionally, after verifying whether the second target data values ​​in the same bin have the same meaning, the method further includes:

[0040] If not, then the second target data values ​​of each sub-bin are divided into equal-frequency sub-bins to obtain the fourth target data values ​​of each sub-bin;

[0041] Verify whether the fourth target data values ​​in the same bin have the same meaning;

[0042] If so, then the equidistant binning and the equal-frequency binning are determined as the binning methods corresponding to the non-numerical data items.

[0043] Optionally, sorting the second initial data values ​​of the different types respectively includes:

[0044] Determine the number of types involved in the different types of second initial data values;

[0045] For a second initial data value of the same type, calculate the first hash value corresponding to the data item description of the data item, calculate the second hash value corresponding to the number of the type, and calculate the third hash value corresponding to the second initial data value of the same type;

[0046] For the same type of second initial data value, the first hash value is determined as the first identifier, the second hash value is determined as the second identifier, and the third hash value is determined as the third identifier;

[0047] For the same type of second initial data value, determine the second score corresponding to the first identifier, the second identifier, and the third identifier respectively;

[0048] The second initial data values ​​of the same type are sorted based on the second score.

[0049] Optionally, the bin sorting method includes a numerical first bin sorting method and a non-numerical second bin sorting method; determining the target bin sorting method whose stability meets preset conditions from the multiple bin sorting methods includes:

[0050] Calculate the first stability corresponding to the first binning method, and calculate the second stability corresponding to the second binning method;

[0051] Determine whether the first stability is within the target prediction capability range, and determine whether the second stability is within the target prediction capability range;

[0052] If the first stability is within the target prediction capability range, while the second stability is not within the target prediction capability range, then the first binning method is determined as the target binning method;

[0053] If the second stability is within the target prediction capability range, while the first stability is not within the target prediction capability range, then the second binning method is determined as the target binning method.

[0054] Optionally, after determining whether the first stability is within the target prediction capability range and whether the second stability is within the target prediction capability range, the method further includes:

[0055] If both the first stability and the second stability are within the target prediction capability range, then compare the first stability and the second stability.

[0056] If the first stability is greater than the second stability, then the first binning method is determined as the target binning method;

[0057] If the second stability is greater than the first stability, then the second binning method is determined as the target binning method.

[0058] Optionally, the step of performing security checks on the data to be stored in each bin to obtain the detection value for each bin includes:

[0059] Extract the feature values ​​corresponding to the data to be stored in each bin;

[0060] Based on the characteristic values, determine the number of first data that meets the data requirements and the number of second data that does not meet the data requirements in each bin;

[0061] The detection value of each sub-box is calculated using the first data quantity and the second data quantity.

[0062] This invention also discloses a data security detection device, the device comprising:

[0063] The data to be stored receiving module is used to receive data to be stored uploaded by users and extract a preset number of sample data from the data to be stored; the sample data includes multiple data items;

[0064] The data item classification module is used to classify the multiple data items based on data type to obtain different types of data items;

[0065] The binning method determination module is used to determine the binning method corresponding to the different types of data items respectively;

[0066] The target binning method determination module is used to determine the target binning method whose stability meets preset conditions from the binning methods corresponding to the different types of data items;

[0067] The data binning module is used to bin the data to be stored using the target binning method.

[0068] The data security detection module is used to perform security detection on the data to be stored in each bin and obtain the detection value of each bin.

[0069] The data storage module is used to determine that the data to be stored has passed the security test when the detection values ​​of each bin are greater than or equal to a preset threshold, so as to store the data to be stored that has passed the security test.

[0070] Optionally, the data type includes numeric and non-numeric types; the data item classification module includes:

[0071] The non-numeric determination submodule is used to determine, from the plurality of data items, data items that do not contain non-numeric values, and data items that do contain non-numeric values;

[0072] The data item classification submodule is used to identify data items that do not contain non-numeric values ​​as numeric data items, and to identify data items that contain non-numeric values ​​as non-numeric data items.

[0073] Optionally, the bin-sorting method determination module includes:

[0074] The first initial data value extraction submodule is used to extract multiple first initial data values ​​from the numerical data items.

[0075] The first initial data value sorting submodule is used to sort the various first initial data values;

[0076] The first initial data value equal-distance binning module is used to divide the sorted first initial data values ​​into equal-distance bins to obtain the first target data value of each bin.

[0077] The first target data value verification submodule is used to verify whether the first target data values ​​in the same bin have the same meaning.

[0078] The first binning method determination submodule is used to determine the equidistant binning as the binning method corresponding to the numerical data item if the condition is met.

[0079] Optionally, after verifying whether the first target data values ​​in the same bin have the same meaning, the binning method determination module further includes:

[0080] The first equal-frequency binning module is used to perform equal-frequency binning on the first target data value of each bin to obtain the third target data value of each bin if no.

[0081] The third target data value verification submodule is used to verify whether the third target data values ​​in the same bin have the same meaning.

[0082] The second binning method determination submodule is used to determine, if so, the equidistant binning and the equal-frequency binning as the binning method corresponding to the numerical data item.

[0083] Optionally, the first initial data value sorting submodule includes:

[0084] The encoding rule acquisition unit is used to acquire the encoding rule corresponding to the numerical data item;

[0085] A partitioning unit is used to partition each first initial data value according to the encoding rules to obtain multiple sets of sub-data corresponding to each first initial data value;

[0086] The first score acquisition unit acquires a first score for each group based on the first initial data value; the first score is used to characterize the correlation between each group of sub-data and the data item to which it belongs;

[0087] The first initial data value sorting unit is used to sort the multiple groups of sub-data based on the first score for each of the first initial data values.

[0088] Optionally, the bin-sorting method determination module includes:

[0089] The second initial data value extraction submodule is used to extract multiple second initial data values ​​from the non-numerical data items.

[0090] The second initial data value classification submodule is used to classify the plurality of second initial data values ​​to obtain different types of second initial data values;

[0091] The second initial data value sorting submodule is used to sort the second initial data values ​​of different types respectively;

[0092] The second equidistant binning module is used to divide the sorted multiple second initial data values ​​into equidistant bins to obtain the second target data value of each bin.

[0093] The second target data value verification submodule is used to verify whether the second target data values ​​in the same bin have the same meaning.

[0094] The third binning method determination submodule is used to determine the equidistant binning as the binning method corresponding to the non-numerical data item if the condition is met.

[0095] Optionally, after verifying whether the second target data values ​​in the same bin have the same meaning, the binning method determination module further includes:

[0096] The second equal-frequency binning module is used to perform equal-frequency binning on the second target data values ​​of each bin to obtain the fourth target data values ​​of each bin if no.

[0097] The fourth target data value verification submodule is used to verify whether the fourth target data values ​​in the same bin have the same meaning.

[0098] The fourth binning method determination submodule is used to determine the equidistant binning and the equal-frequency binning as the binning method corresponding to the non-numerical data item if the condition is met.

[0099] Optionally, the second initial data value sorting submodule includes:

[0100] A type quantity determination unit is used to determine the number of types involved in the second initial data values ​​of the different types;

[0101] The hash value calculation unit is used to calculate, for the same type of second initial data value, a first hash value corresponding to the data item description of the data item, a second hash value corresponding to the number of the type, and a third hash value corresponding to the same type of second initial data value;

[0102] The identifier determination unit is configured to, for a second initial data value of the same type, determine the first hash value as a first identifier, determine the second hash value as a second identifier, and determine the third hash value as a third identifier;

[0103] The second score determination unit is used to determine the second score corresponding to the first identifier, the second identifier, and the third identifier for the same type of second initial data value;

[0104] The second initial data value sorting unit is used to sort the second initial data values ​​of the same type based on the second score.

[0105] Optionally, the bin sorting method includes a numerical first bin sorting method and a non-numerical second bin sorting method; the target bin sorting method determination module includes:

[0106] The stability calculation submodule is used to calculate the first stability corresponding to the first binning method and the second stability corresponding to the second binning method.

[0107] The stability determination submodule is used to determine whether the first stability is within the target prediction capability range and whether the second stability is within the target prediction capability range.

[0108] The first target binning method determination submodule is used to determine the first binning method as the target binning method if the first stability is within the target prediction capability range, but the second stability is not within the target prediction capability range.

[0109] The second target binning method determination submodule is used to determine the second binning method as the target binning method if the second stability is within the target prediction capability range, but the first stability is not within the target prediction capability range.

[0110] Optionally, after determining whether the first stability is within the target prediction capability range and whether the second stability is within the target prediction capability range, the target binning method determination module further includes:

[0111] A stability comparison submodule is used to compare the first stability and the second stability if both the first stability and the second stability are within the target prediction capability range.

[0112] The third target binning method determination submodule is used to determine the first binning method as the target binning method if the first stability is greater than the second stability.

[0113] The fourth target binning method determination submodule is used to determine the second binning method as the target binning method if the second stability is greater than the first stability.

[0114] Optionally, the data security detection module to be stored includes:

[0115] The feature extraction submodule is used to extract the feature values ​​corresponding to the data to be stored in each bin;

[0116] The data quantity determination submodule is used to determine, based on the feature value, the first data quantity that meets the data requirements and the second data quantity that does not meet the data requirements in each bin;

[0117] The detection value calculation submodule is used to calculate the detection value of each sub-box using the first data quantity and the second data quantity.

[0118] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0119] The memory is used to store computer programs;

[0120] When the processor executes the program stored in the memory, it implements the data security detection method as described in the embodiments of the present invention.

[0121] This invention also discloses one or more computer-readable media storing instructions that, when executed by one or more processors, cause the processors to perform the data security detection method as described in this invention.

[0122] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0123] In this embodiment of the invention, user-uploaded data to be stored is received, and a preset number of sample data are extracted from the data to be stored. The sample data includes multiple data items. The multiple data items are classified according to data type to obtain different types of data items. The binning method corresponding to each type of data item is determined. From the binning methods corresponding to different types of data items, a target binning method whose stability meets preset conditions is determined. The target binning method is used to bin the data to be stored. Security detection is performed on the data to be stored in each bin to obtain the detection value of each bin. When the detection value of each bin is greater than or equal to a preset threshold, it is determined that the data to be stored has passed the security detection, and the data to be stored that has passed the security detection is stored. This embodiment of the invention strengthens the data security control by performing security detection on the data before uploading it, ensuring the effectiveness of data security control. Furthermore, by binning the data and then performing security detection on the binned data, the efficiency of data security detection is improved. Attached Figure Description

[0124] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0125] Figure 1 This is a flowchart of the steps of a data security detection method provided in an embodiment of the present invention;

[0126] Figure 2 This is a structural block diagram of a data security detection device provided in an embodiment of the present invention. Detailed Implementation

[0127] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0128] With the advent of the big data era and the development of cloud computing and deep learning, more efficient data processing technologies have brought higher productivity, but have also triggered data security crises, especially in operation and maintenance systems and data auditing systems. Data breaches can result in significant losses. Therefore, data security has become an indispensable part of big data, and ensuring data security is receiving increasing attention.

[0129] Existing solutions only detect data access through technologies such as firewalls and authentication. Their core function is to detect access behavior, not the security of the data itself. For example, a data aggregation device might monitor the data aggregation process, such as whether the data uploader is authorized and whether the upload is secure, but it doesn't check the security of the uploaded data itself.

[0130] To address the aforementioned issues, this invention provides a data security detection method. This method strengthens data security control by performing security checks on data during the upload process before storing it, ensuring the effectiveness of data security control. Furthermore, by binning the data and then performing security checks on the binned data, the efficiency of data security detection is improved.

[0131] Reference Figure 1 The diagram illustrates a flowchart of a data security detection method provided by an embodiment of the present invention. The method may specifically include the following steps:

[0132] Step 101: Receive the data to be stored uploaded by the user, and extract a preset number of sample data from the data to be stored; the sample data includes multiple data items.

[0133] In this embodiment of the invention, it can be applied to a data aggregation device that can receive data uploaded by users for storage. Prior to this, the data aggregation device can first perform security controls on the user's upload behavior, then on the data content, and subsequently on access to that data content.

[0134] In practice, when data to be stored is detected and needs to be uploaded, the data aggregation device can first verify whether the user uploading the data has the authorization permission. If the user is confirmed to have the authorization permission, the data aggregation device can approve the user's upload behavior.

[0135] After receiving the user-uploaded data to be stored, the data aggregation device can extract a preset number of sample data from the data to be stored. The preset number can be n, where n is a positive integer. Specifically, the data aggregation device can randomly select n data points as sample data from the data to be stored. Here, n can be a single value or a ratio.

[0136] As an example, assuming n is 1000, the data aggregation device can randomly select 1000 data points from the data to be stored.

[0137] As another example, assuming n is 10%, if there are 10,000 data items to be stored, then the data aggregation device can randomly select 1,000 data items from the data to be stored.

[0138] It should be noted that the extraction method can be other than random, such as extracting the first n data points, as long as the sample data can represent the data to be stored. This embodiment of the invention does not limit this.

[0139] In this embodiment of the invention, the sample data may consist of multiple data items. For example, user information data may include, but is not limited to, data items such as user identifier, gender, age, education level, hobbies, and mobile phone number.

[0140] Step 102: Classify the multiple data items based on data type to obtain different types of data items.

[0141] In this embodiment of the invention, each data item has a corresponding data type, and the data aggregation device can classify multiple data items based on the data type, thereby obtaining data items of different types.

[0142] In an optional embodiment of the present invention, the data type includes numeric and non-numeric types; step 102 may include the following sub-steps:

[0143] Sub-step S11: From the plurality of data items, determine data items that do not contain non-numeric values, and determine data items that contain non-numeric values;

[0144] Sub-step S12: Determine the data items that do not contain non-numeric values ​​as the numeric data items, and determine the data items that contain non-numeric values ​​as the non-numeric data items.

[0145] In this embodiment of the invention, the data type may include numeric and non-numeric types. The data aggregation device can process each data item, such as user identifier, gender, age, education level, hobbies, mobile phone number, etc., to determine whether it is a numeric type, thereby dividing multiple data items into two categories: one is numeric data items, and the other is non-numeric data items.

[0146] In practical implementation, the data aggregation device can identify data items that do not contain non-numeric values ​​from multiple data items. Then, it can identify data items that do not contain non-numeric values ​​as numeric data items, such as mobile phone numbers and ages. Whether they are integers or floating-point numbers, as long as they do not contain non-numeric values, they can be classified as numeric data items.

[0147] In practical implementation, the data aggregation device can identify data items containing non-numeric values ​​from multiple data items, and then classify the data items containing non-numeric values ​​as non-numerical data items, such as user identifier, gender, education level, hobbies, etc. Whether they are character type or Boolean type, as long as they may contain non-numeric values, they can be classified as non-numerical data items.

[0148] Step 103: Determine the binning method corresponding to the different types of data items respectively.

[0149] After classifying data items into different types, the data aggregation device can determine the binning method corresponding to each type of data item. Specifically, after classifying data items into numerical and non-numerical types, the data aggregation device can determine the binning method corresponding to the numerical data items, and it can also determine the binning method corresponding to the non-numerical data items.

[0150] In an optional embodiment of the present invention, step 103 may include the following sub-steps:

[0151] Sub-step S21: For the numerical data item, extract multiple first initial data values ​​from the numerical data item;

[0152] Sub-step S22: Sort each of the first initial data values.

[0153] For numerical data items, the data aggregation device can extract multiple initial data values ​​from the numerical data items and then sort each initial data value. Each initial data value can be a specific data value for each data item. For example, mobile phone number data item 1 is 13011112222, mobile phone number data item 2 is 13633334444, age data item 1 is 20, age data item 2 is 40, and age data item 3 is 30. Since mobile phone number data items 1-2 and age data items 1-3 are all numerical data items, multiple initial data values ​​can be extracted from the numerical data items, including 13011112222, 13633334444, 20, 40, and 30.

[0154] In an optional embodiment of the present invention, sub-step S22 may include the following sub-steps:

[0155] Sub-step S221: Obtain the encoding rule corresponding to the numerical data item;

[0156] Sub-step S222: According to the encoding rules, each first initial data value is divided to obtain multiple sets of sub-data corresponding to each first initial data value;

[0157] Sub-step S223: For each of the first initial data values, obtain the first score for each group; the first score is used to characterize the correlation between each group of sub-data and the data item to which it belongs;

[0158] Sub-step S224: For each of the first initial data values, sort the multiple groups of sub-data based on the first score.

[0159] In this embodiment of the invention, sorting each first initial data value is not a simple sorting from largest to smallest or smallest to largest, but rather a sorting using the semantic sorting method provided in this embodiment. Specifically, the data aggregation device can obtain the encoding rules corresponding to the numerical data items. Each first initial data value can contain multiple sub-data. The data aggregation device can divide the multiple sub-data in each first initial data value according to the encoding rules, thereby obtaining multiple sets of sub-data corresponding to each first initial data value.

[0160] Taking mobile phone numbers as an example, the encoding rule for mobile phone numbers can be: MAC+H0 H1 H2 H3+ABCD.

[0161] MAC (Media Access Control) is a type of mobile access code. For example, operator 1 includes 134, 135, 136, 137, 138, 139, 150, 151, 152, 158, 159, 157, 182, 187, 188, etc.; operator 2 includes 130, 131, 132, 155, 156, 182, 185, 186, etc.; and operator 3 includes 133, 153, 180, 189, etc.

[0162] H0, H1, H2, and H3 are HLR (Home Location Register) identification codes, uniformly assigned by the operator. Among them, H0, H1, and H2 are uniformly assigned to each province by the operator, while H3 is assigned to the local network by each province. Therefore, H0, H1, H2, and H3 determine the location of the mobile phone number.

[0163] ABCD are mobile subscriber numbers, which are assigned by each HLR.

[0164] Based on the above encoding rules, we can conclude that: the first 3 digits of a mobile phone number are the network identification number, which can be used to indicate which operator the mobile phone number belongs to; the middle 4 digits are the area code, which can be used to indicate the region where the mobile phone number is located; and the last 4 digits are the user number, which can be used to indicate the user's identity.

[0165] In this way, mobile phone numbers can be divided according to the coding rules. For example, the first 3 digits can be grouped together, the middle 4 digits can be grouped together, and the last 4 digits can be grouped together. For the convenience of describing the following steps, the group containing the first 3 digits can be called group 1, the group containing the middle 4 digits can be called group 2, and the group containing the last 4 digits can be called group 3.

[0166] In this embodiment of the invention, after obtaining multiple sets of sub-data corresponding to each first initial data value, the data aggregation device can obtain the first score of each set for each first initial data value.

[0167] The first score is used to characterize the correlation between each group of sub-data and its corresponding data item. Specifically, the first score of a group can be used to characterize the importance of that group of sub-data to its corresponding data item. For example, the stronger the correlation between a group of sub-data and its corresponding data item, the greater the importance of that group of sub-data, and thus the higher its first score; conversely, the weaker the correlation between a group of sub-data and its corresponding data item, the less important that group of sub-data, and thus the lower its first score.

[0168] The first score for each group is determined based on the coding rules, which are set by the user when setting the coding rules. The data aggregation device can directly obtain the first score determined by the user.

[0169] Continuing with the mobile phone number example above, the user sets the first score for Group 1 to 0.9, Group 2 to 0.7, and Group 3 to 0.5. From these first scores, it can be seen that the importance of Group 1 > Group 2 > Group 3. Group 1 is the network identification number, which identifies the service provider, i.e., which operator the mobile phone number belongs to. Therefore, the correlation between the Group 1 sub-data and the mobile phone number data item is strong.

[0170] It should be noted that importance is simply a measure of the importance of a given group and has no relation to other groups. Therefore, the sum of the importance of multiple groups is not necessarily equal to 1. For example, the first score of group 1 + the first score of group 2 + the first score of group 3 ≠ 1.

[0171] In this embodiment of the invention, after obtaining the first score for each group, the data aggregation device can sort the multiple groups of sub-data based on the first score for each first initial data value. Specifically, based on the first score of each group, the data aggregation device can determine the importance of each group, and thus sort them from high to low according to their importance.

[0172] Continuing with the mobile phone number example above, the importance of group 1 > group 2 > group 3. Therefore, we can first sort the sub-data of group 1. If the sub-data in group 1 are all different values, such as 134, 156, 180, etc., then we only need to sort the sub-data of group 1. If the sub-data in group 1 have the same value, such as 133, 155, 188, etc., then we can sort them based on the sub-data of group 2. If the sub-data of group 1 and group 2 are different, for example, group 1 is 133 and group 2 is 1111, then we only need to sort the sub-data of group 1 and group 2. If the sub-data of group 1 and group 2 are the same, for example, group 1 is 188 and group 2 is 1880, then we can sort them based on the sub-data of group 3 to obtain the final sequence.

[0173] It should be noted that the groups are sorted from highest to lowest importance, while the sub-data within a group can be sorted in descending order or ascending order. For example, the sub-data of group 1 can be sorted in descending order. This embodiment of the invention does not limit this.

[0174] The above sorting scheme enables sorting to represent the attributes of data, that is, to represent the meaning of data, making sorting more valuable.

[0175] Sub-step S23: Divide the sorted initial data values ​​into equal-distance bins to obtain the first target data value for each bin;

[0176] Sub-step S24 verifies whether the first target data values ​​in the same bin have the same meaning;

[0177] Sub-step S25: If so, then the equidistant binning is determined as the binning method corresponding to the numerical data item.

[0178] In this embodiment of the invention, after sorting each first initial data value, the data aggregation device can divide the sorted first initial data values ​​into equal-distance bins, thereby obtaining the first target data value for each bin. Equal-distance binning refers to finding the maximum and minimum values ​​in the sorted data, and dividing the target quantile points at equal distances between the maximum and minimum values. The multiple first initial data values ​​can be divided into multiple bins using these target quantile points.

[0179] After equal-distance binning, the data aggregation device can verify whether the first target data value in the same bin has the same meaning. If the first target data value in the same bin has the same meaning, it means that equal-distance binning does not affect the meaning of the data. Therefore, equal-distance binning can be determined as the binning method corresponding to numerical data items.

[0180] In an optional embodiment of the present invention, after verifying whether the first target data values ​​in the same bin have the same meaning, step 103 may further include the following sub-steps:

[0181] Sub-step S31: If not, then divide the first target data values ​​of each sub-bin into equal-frequency sub-bins to obtain the third target data values ​​of each sub-bin.

[0182] Sub-step S32 verifies whether the third target data values ​​in the same bin have the same meaning;

[0183] Sub-step S33: If so, then the equidistant binning and the equal-frequency binning are determined as the binning methods corresponding to the numerical data items.

[0184] The data aggregation device can verify whether the first target data values ​​in the same bin have the same meaning. If the first target data values ​​in the same bin have different meanings, it indicates that equidistant binning affects the meaning of the data; that is, data with the same meaning are separated. Therefore, the first target data values ​​in each bin can be further binned at equal frequencies to obtain the third target data value for each bin. Equal-frequency binning refers to adjusting the data in each bin so that the number of data points in each bin is approximately equal, ensuring that data with the same meaning are in the same bin.

[0185] After equal-frequency binning, the data aggregation device can verify whether the third target data value in the same bin has the same meaning. If the third target data value in the same bin has the same meaning, it means that equal-frequency binning ensures the integrity of the data meaning. Therefore, equal-distance binning and equal-frequency binning can be combined to obtain a combination method, and this combination method is determined as the binning method corresponding to the numerical data item.

[0186] Sub-steps S23 to S25 and sub-steps S31 to S33 are specifically implemented using the following steps A to D:

[0187] A. Obtain the number of target quantiles m:

[0188] The target number of quantiles m can be set by the user. The target number of quantiles m refers to how many bins the user wants to divide the data into.

[0189] For example, if you want to divide the data into two bins, you can set one target quantile in the middle to divide the data into two parts, where m=1; if you want to divide the data into three bins, you can set two target quantiles in the middle to divide the data into three parts, where m=2.

[0190] Therefore, by setting the target number of quantiles m, multiple initial data values ​​can be divided into m+1 bins.

[0191] B. Total number of data points n all :

[0192] Total number of data n all This refers to the total number of initial data values.

[0193] C. Divide the sorted data into equal-interval bins:

[0194] The binning process can include equidistant binning and equal-frequency binning. The data aggregation device can equidistantly bin multiple sorted initial data values.

[0195] If n all The remainder of ÷ (m+1) is 0, meaning the quotient is an integer. This indicates the data can be divided into m+1 equal parts. Therefore, we can start from the first sorted data and divide n... all / m data values, divided into the first bin; from 1+n all n starting from / (m+1) data points all The data is divided into (m+1) bins, starting from 1+2*n. all n starting from / (m+1) data points all The data is divided into (m+1) bins, and so on, until all data are divided into m+1 bins.

[0196] If n all The remainder of / (m+1) is non-zero, meaning the quotient is a decimal, indicating the data cannot be divided into m+1 equal parts. Therefore, we can start from the first sorted data and divide by 1+n. all / (m+1) data values ​​are divided into the first bin; from 2+n all Starting with / (m+1) data points, 1+n all / (m+1) data points are divided into the second bin; from 3+2*n all Starting with / (m+1) data points, 1+n all The data is divided into (m+1) bins, and so on, until all data are divided into m+1 bins.

[0197] D. Adjust the binning method according to the equidistant binning of the groups to obtain the binning method corresponding to the numerical data items:

[0198] Step D is to ensure that the data in the same bin has the same meaning.

[0199] Therefore, the implementation process of step D is as follows:

[0200] a. Take two adjacent bins in sequence, select the last data value from the previous bin, and select the first data value from the next bin;

[0201] b. Determine whether the sub-data of the two data values ​​are the same in the group with the highest weight;

[0202] c. If they are not the same, it can be said that the data values ​​in the previous bin have the same meaning and the data values ​​in the next bin have the same meaning. This can further indicate that equal-distance binning does not affect the meaning of the data. Therefore, equal-distance binning can be determined as the binning method corresponding to numerical data items.

[0203] In one example, suppose the last data value of the previous bin is 13011112222, and the first data value of the next bin is 13633334444. Here, group 1 represents the first three digits of the phone number, group 2 represents the middle four digits, and group 3 represents the last four digits. Since the first score for group 1 is 0.9, the first score for group 2 is 0.7, and the first score for group 3 is 0.5, group 1 has the highest weight. Because the sub-data values ​​of the two data values ​​in group 1 are 130 and 136 respectively, it is clear that the sub-data values ​​of the two data values ​​in group 1 are different. This indicates that the data values ​​in the previous bin have the same meaning, and the data values ​​in the next bin have the same meaning. Therefore, it can be concluded that equidistant binning does not affect the meaning of the data. Thus, equidistant binning can be determined as the binning method corresponding to numerical data items.

[0204] d. If they are the same, it indicates that equidistant binning affects the meaning of the data; that is, data with the same meaning are separated. Therefore, the data in each bin can be further binned into equal-frequency bins. The implementation process of step d is as follows:

[0205] Determine the number of data points n1 in the previous bin that have the same data value as the last data point in the group with the largest weight, and determine the number of data points n2 in the next bin that have the same data value as the first data point in the group with the largest weight.

[0206] If n1 accounts for less than 1 / 4 of the total data in the previous bin and n1 / n2 < 0.5, then the data in the group with the largest weight in the previous bin that has the same data value as the last data value can be moved to the next bin, thus ensuring the integrity of the data meaning.

[0207] If n2 accounts for less than 1 / 4 of the total data in the next bin, and n1 / n2 > 1.5, then the data in the group with the largest weight in the next bin that has the same data value as the first data value can be moved to the previous bin, thus ensuring the integrity of the data meaning.

[0208] In addition, if adjusting the data would cause data imbalance, the example of this invention can discard the complete meaning to ensure the balance of data volume in bins without making any adjustments.

[0209] e. Determine the binning method obtained in step d as the binning method corresponding to the numerical data items;

[0210] Specifically, it involves combining equidistant binning and equal-frequency binning to obtain a combination method, and then determining this combination method as the binning method corresponding to the numerical data items.

[0211] In an optional embodiment of the present invention, step 103 may include the following sub-steps:

[0212] Sub-step S41: For the non-numerical data item, extract multiple second initial data values ​​from the non-numerical data item;

[0213] Sub-step S42: Classify the plurality of second initial data values ​​to obtain second initial data values ​​of different types;

[0214] Sub-step S43: Sort the second initial data values ​​of the different types respectively.

[0215] For non-numerical data items, the data aggregation device can extract multiple second initial data values ​​from the non-numerical data items, then classify these multiple second initial data values ​​to obtain different types of second initial data values, and then sort the second initial data values ​​of different types separately. Each second initial data value can be a specific data value for each data item. For example, gender data item 1 is female, gender data item 2 is female, gender data item 3 is male, education data item 1 is undergraduate, and education data item 2 is doctoral. Since gender data items 1-3 and education data items 1-2 are all non-numerical data items, multiple second initial data values ​​can be extracted from the non-numerical data items, including data values ​​such as female, female, male, undergraduate, and doctoral.

[0216] In an optional embodiment of the present invention, sub-step S43 may include the following sub-steps:

[0217] Sub-step S431: Determine the number of types involved in the different types of second initial data values;

[0218] Sub-step S432: For the second initial data value of the same type, calculate the first hash value corresponding to the data item description of the data item, calculate the second hash value corresponding to the number of the type, and calculate the third hash value corresponding to the second initial data value of the same type;

[0219] Sub-step S433: For the same type of second initial data value, determine the first hash value as the first identifier, the second hash value as the second identifier, and the third hash value as the third identifier;

[0220] Sub-step S434: For the same type of second initial data value, determine the second score corresponding to the first identifier, the second identifier, and the third identifier respectively;

[0221] Sub-step S435: Sort the second initial data values ​​of the same type based on the second score.

[0222] In this embodiment of the invention, the data aggregation device can determine the number of types involved in different types of second initial data values. For example, for gender data items, based on the data values, it can be known that they are either female or male. Therefore, the second initial data values ​​of gender data items can be divided into two categories: one category contains all female data values, and the other category contains all male data values. Thus, the number of types involved can be determined to be 2.

[0223] For a second initial data value of the same type, the data aggregation device can calculate a first hash value corresponding to the data item description of the data item to which it belongs, a second hash value corresponding to the type quantity, and a third hash value corresponding to the second initial data value of the same type. Then, the first hash value can be determined as a first identifier, the second hash value can be determined as a second identifier, and the third hash value can be determined as a third identifier.

[0224] In one example, for a gender data item of type "female", the first hash value corresponding to the data item description of the gender data item containing the female class can be calculated. Since the number of types involved in the gender data item is 2, the second hash value corresponding to 2 can be calculated. In addition, the third hash value corresponding to "female" can be calculated. Then, the first hash value can be determined as the first identifier X, the second hash value can be determined as the second identifier Y, and the third hash value can be determined as the third identifier Z. Therefore, the identifier XYZ of the female class = the hash value corresponding to the data item description + the hash value corresponding to 2 + the hash value corresponding to "female".

[0225] For the same type of second initial data value, the data aggregation device can divide the identifiers XYZ into three groups: the first identifier X, the second identifier Y, and the third identifier Z. Then, the second score corresponding to each group (X, Y, and Z) can be determined, which means determining the second score corresponding to each identifier (X, Y, and Z). The second score can be used to characterize the importance of each identifier. Specifically, in this embodiment, the importance of groups X, Y, and Z can be preset, such that: importance of group X > importance of group Z > importance of group Y. Therefore, the second score of group X > the second score of group Z > the second score of group Y.

[0226] After determining the second score for each group, the data aggregation device can sort the second initial data values ​​of the same type based on the second score. Specifically, based on the second score of each group, the data aggregation device can determine the importance of each group, and thus sort them from highest to lowest importance.

[0227] In the specific implementation, since the importance of group X > the importance of group Z > the importance of group Y, the data values ​​or hash values ​​in group X can be sorted first. If all the data in group X have different values, then only the data values ​​or hash values ​​in group X need to be sorted. If there are data with the same value in group X, then the data values ​​or hash values ​​in group Z can be sorted. If the data in group X and group Z are different, then only the data in group X and group Z need to be sorted. If the data in group X and group Z are the same, then the data values ​​or hash values ​​in group Y can be sorted to obtain the final sequence.

[0228] The above sorting scheme enables sorting to represent the attributes of data, that is, to represent the meaning of data, making sorting more valuable.

[0229] Sub-step S44: Divide the sorted second initial data values ​​into equal-interval bins to obtain the second target data values ​​for each bin;

[0230] Sub-step S45 verifies whether the second target data values ​​in the same bin have the same meaning;

[0231] Sub-step S46: If so, then the equidistant binning is determined as the binning method corresponding to the non-numerical data item.

[0232] In this embodiment of the invention, after sorting the second initial data values ​​of different types respectively, the data aggregation device can divide the sorted second initial data values ​​into equal-distance bins, thereby obtaining the second target data value of each bin.

[0233] After binning at equal intervals, the data aggregation device can verify whether the second target data values ​​in the same bin have the same meaning. If the second target data values ​​in the same bin have the same meaning, it means that binning at equal intervals does not affect the meaning of the data. Therefore, binning at equal intervals can be determined as the binning method corresponding to non-numerical data items.

[0234] In an optional embodiment of the present invention, after verifying whether the second target data values ​​in the same bin have the same meaning, step 103 may further include the following sub-steps:

[0235] Sub-step S51: If not, then perform equal-frequency binning on the second target data values ​​of each bin to obtain the fourth target data values ​​of each bin.

[0236] Sub-step S52 verifies whether the fourth target data values ​​in the same bin have the same meaning;

[0237] Sub-step S53: If so, then the equidistant binning and the equal-frequency binning are determined as the binning methods corresponding to the non-numerical data items.

[0238] The data aggregation device can verify whether the second target data values ​​in the same bin have the same meaning. If the second target data values ​​in the same bin have different meanings, it means that the equidistant binning affects the meaning of the data, that is, the data with the same meaning is separated. Therefore, the second target data values ​​of each bin can be further binned at the same frequency to obtain the fourth target data value of each bin.

[0239] After equal-frequency binning, the data aggregation device can verify whether the fourth target data value in the same bin has the same meaning. If the fourth target data value in the same bin has the same meaning, it means that equal-frequency binning ensures the integrity of the data meaning. Therefore, equal-distance binning and equal-frequency binning can be combined to obtain a combination method. This combination method is determined as the binning method corresponding to non-numerical data items.

[0240] Sub-steps S44 to S46 and sub-steps S51 to S53 are implemented in the same way as steps A to D in the numerical data items to determine the binning method. For details, please refer to steps A to D, which will not be repeated here.

[0241] The above steps can determine the binning method for numerical data items, as well as for non-numerical data items. For example, data 1 is {gender: female, age: 20}, data 2 is {gender: female, age: 40}, and data 3 is {gender: male, age: 30}. The binning for the gender data item is {{data 1, data 2}, {data 3}}, and the binning for the age data item is {{data 2, data 3}, {data 4}}.

[0242] Step 104: Determine the target binning method whose stability meets the preset conditions from the binning methods corresponding to the different types of data items.

[0243] In this embodiment of the invention, after obtaining the binning method corresponding to each type of data item, the data aggregation device can determine the target binning method whose stability meets the preset conditions.

[0244] In an optional embodiment of the present invention, the bin sorting method includes a numerical first bin sorting method and a non-numerical second bin sorting method; step 104 may include the following sub-steps:

[0245] Sub-step S61: Calculate the first stability corresponding to the first binning method, and calculate the second stability corresponding to the second binning method;

[0246] Sub-step S62: Determine whether the first stability is within the target prediction capability range, and determine whether the second stability is within the target prediction capability range;

[0247] Sub-step S63: If the first stability is within the target prediction capability range, while the second stability is not within the target prediction capability range, then the first binning method is determined as the target binning method.

[0248] Sub-step S64: If the second stability is within the target prediction capability range, while the first stability is not within the target prediction capability range, then the second binning method is determined as the target binning method.

[0249] This invention can determine the most stable binning method as the final target binning method by evaluating the stability of all bins. Specifically, this invention can use the binning method corresponding to the determined numerical data items as the first binning method, and the binning method corresponding to the determined non-numerical data items as the second binning method. Therefore, the obtained binning methods include the first binning method with numerical data and the second binning method with non-numerical data.

[0250] The data aggregation device can calculate the first stability IV1 corresponding to the first binning method, and can calculate the second stability IV2 corresponding to the second binning method. The specific calculation method of stability IV is as follows:

[0251]

[0252]

[0253] Where +i is the number of positive samples in the i-th bin, +ALL is the number of positive samples in all sample data, -i is the number of negative samples in the i-th bin, -ALL is the number of negative samples in all sample data, and α is the minimum value, which is used to prevent calculation errors caused by the denominator being 0.

[0254] The number of positive samples refers to the number of data points correctly binned using this binning method. The number of negative samples refers to the number of data points incorrectly binned using this binning method. Specifically, after determining the binning method, sample data can be entered, including manually labeled suggested binning results. After using this binning method, the sample data will be binned, and the current binning result will be compared with the suggested binning result. If the current binning result is the same as the suggested binning result, it is a positive sample; if the current binning result is different from the suggested binning result, it is a negative sample.

[0255] Sub-step S62: Determine whether the first stability is within the target prediction capability range, and determine whether the second stability is within the target prediction capability range;

[0256] Sub-step S63: If the first stability is within the target prediction capability range, while the second stability is not within the target prediction capability range, then the first binning method is determined as the target binning method.

[0257] Sub-step S64: If the second stability is within the target prediction capability range, while the first stability is not within the target prediction capability range, then the second binning method is determined as the target binning method.

[0258] In this embodiment of the invention, stability IV describes the predictive capability of the data. Therefore, after calculating the first stability IV1 and the second stability IV2, it can be determined whether the first stability IV1 is within the target predictive capability range, and whether the second stability IV2 is within the target predictive capability range. If the first stability IV1 is within the target predictive capability range, but the second stability IV2 is not, then the first binning method corresponding to the first stability IV1 can be determined as the target binning method; if the second stability IV2 is within the target predictive capability range, but the first stability IV1 is not, then the second binning method corresponding to the second stability IV2 can be determined as the target binning method.

[0259] Specifically, embodiments of the present invention may pre-set multiple prediction capability ranges, namely a first prediction capability range (0–0.02), a second prediction capability range (0.02–0.1), a third prediction capability range (0.1–0.3), a fourth prediction capability range (0.3–0.5), and a fifth prediction capability range (0.5–+∞). Stability IV describes the predictive capability of the data, specifically as follows:

[0260] If the stability IV is less than 0.02, it is not useful for prediction;

[0261] If the stability IV is between 0.02 and 0.1, the predictive ability is very weak.

[0262] If the stability IV is between 0.1 and 0.3, the predictive ability is of moderate strength.

[0263] If the stability IV is between 0.3 and 0.5, then the predictive ability is strong;

[0264] If the stability IV is greater than 0.5, the predictive ability is questionable and needs to be investigated.

[0265] Therefore, in this embodiment of the invention, the fourth prediction capability range (0.3 to 0.5) can be used as the target prediction capability range. If the first stability IV1 is within the target prediction capability range (0.3 to 0.5), while the second stability IV2 is not, it indicates that the prediction capability of the first binning method is strong, and the prediction capability of the second binning method is relatively weak or questionable. Therefore, the first binning method can be determined as the target binning method. If the second stability IV2 is within the target prediction capability range (0.3 to 0.5), while the first stability IV1 is not, it indicates that the prediction capability of the second binning method is strong, while the prediction capability of the first binning method is relatively weak or questionable. Therefore, the second binning method can be determined as the target binning method.

[0266] In an optional embodiment of the present invention, after determining whether the first stability is within the target prediction capability range and whether the second stability is within the target prediction capability range, step 104 may further include the following sub-steps:

[0267] Sub-step S71: If both the first stability and the second stability are within the target prediction capability range, then compare the first stability and the second stability.

[0268] Sub-step S72: If the first stability is greater than the second stability, then the first binning method is determined as the target binning method;

[0269] Sub-step S73: If the second stability is greater than the first stability, then the second binning method is determined as the target binning method.

[0270] If both the first stability IV1 and the second stability IV2 are within the target predictive capability range (0.3–0.5), it indicates that both the first and second binning methods have strong predictive capabilities. In this case, it is necessary to further determine the binning method with the most stable predictive capability, i.e., the binning method with the highest stability. Specifically, the data aggregation device can compare the first stability IV1 and the second stability IV2. If the first stability IV1 is greater than the second stability IV2, it indicates that the first binning method is the most stable, and therefore it can be determined as the target binning method. If the second stability IV2 is greater than the first stability IV1, it indicates that the second binning method is the most stable, and therefore it can be determined as the target binning method.

[0271] Step 105: The data to be stored is binned using the target binning method.

[0272] In this embodiment of the invention, after determining the final target binning method, the data aggregation device can adopt the target binning method to bin all data items of the data to be stored as a whole.

[0273] Step 106: Perform security checks on the data to be stored in each bin to obtain the check values ​​for each bin.

[0274] In this embodiment of the invention, after the data to be stored is binned, the data to be stored is divided into multiple bins. The data aggregation device can perform security detection on the data to be stored in each bin, thereby obtaining the detection value of each bin. Binning detection can improve the efficiency of data security detection.

[0275] In an optional embodiment of the present invention, step 106 may include the following sub-steps:

[0276] Sub-step S81: Extract the feature values ​​corresponding to the data to be stored in each bin;

[0277] Sub-step S82: Based on the feature values, determine the number of first data that meets the data requirements and the number of second data that does not meet the data requirements in each bin;

[0278] Sub-step S83: Using the first data quantity and the second data quantity, calculate the detection value of each sub-box.

[0279] In practical implementation, the data aggregation device can extract the feature value α corresponding to the data to be stored in each bin. j , where j is the bin identifier. The data aggregation device can process the characteristic value α of each bin. j Verification is performed to determine whether the data to be stored in each bin meets the data requirements. Specifically, the data aggregation device can perform this verification based on the characteristic value α of each bin. j Determine the number n of the first data points that meet the data requirements in each bin. j (+) and the number of second data points n that do not meet the data requirements j (-), thus allowing the use of the first data quantity n j (+) and the second data quantity n j (-), calculate the detection value β for each sub-box. j Among them, the detection value β j The calculation method is as follows:

[0280]

[0281] Among them, T nj(+) Let T be the mean of the features that meet the requirements in the j-th bin. j Let T be the characteristic mean of all data in the j-th bin. nj(-)Let be the mean value of the features that do not meet the requirements in the j-th bin.

[0282] Step 107: When the detection values ​​of each bin are greater than or equal to a preset threshold, the data to be stored is determined to have passed the security test, and the data to be stored that has passed the security test is stored.

[0283] In this embodiment of the invention, after calculating the detection value β of each sub-box... j Then, if the detection value β of all bins j If all values ​​are not less than the preset threshold, the data can be considered normal and the security test passes. That is, when the detection value β of each bin... j When all values ​​are greater than or equal to a preset threshold, the data aggregation device can determine that the data to be stored has passed the security test, and thus store the data that has passed the security test. Subsequently, security control can be applied to access this data. The preset threshold can be an empirical value or a value obtained through big data analysis; this embodiment of the invention does not limit its application.

[0284] In summary, in this embodiment of the invention, user-uploaded data to be stored is received, and a preset number of sample data are extracted from the data to be stored. The sample data includes multiple data items. These data items are classified based on data type to obtain different types of data items. A binning method corresponding to each type of data item is determined. From the binning methods corresponding to different types of data items, a target binning method whose stability meets preset conditions is determined. The target binning method is used to bin the data to be stored. Security checks are performed on the data to be stored in each bin to obtain the check value for each bin. When the check values ​​for each bin are greater than or equal to a preset threshold, the data to be stored is determined to have passed the security check, and the data that has passed the security check is stored. This embodiment of the invention strengthens data security control by performing security checks on the data before uploading it, ensuring the effectiveness of data security control. Furthermore, by binning the data and then performing security checks on the binned data, the efficiency of data security checks is improved.

[0285] refer to Figure 2 The diagram shows a structural block diagram of a data security detection device provided in an embodiment of the present invention, which may specifically include the following modules:

[0286] The data to be stored receiving module 201 is used to receive data to be stored uploaded by the user, and extract a preset number of sample data from the data to be stored; the sample data includes multiple data items;

[0287] The data item classification module 202 is used to classify the multiple data items based on data type to obtain data items of different types;

[0288] Binning method determination module 203 is used to determine the binning method corresponding to the different types of data items respectively;

[0289] The target binning method determination module 204 is used to determine the target binning method whose stability meets the preset conditions from the binning methods corresponding to the different types of data items;

[0290] The data binning module 205 is used to bin the data to be stored using the target binning method.

[0291] The data security detection module 206 is used to perform security detection on the data to be stored in each bin and obtain the detection value of each bin.

[0292] The data storage module 207 is used to determine that the data to be stored has passed the security test when the detection values ​​of each bin are greater than or equal to a preset threshold, so as to store the data to be stored that has passed the security test.

[0293] In an optional embodiment of the present invention, the data type includes numeric and non-numeric types; the data item classification module 202 may include:

[0294] The non-numeric determination submodule is used to determine, from the plurality of data items, data items that do not contain non-numeric values, and data items that do contain non-numeric values;

[0295] The data item classification submodule is used to identify data items that do not contain non-numeric values ​​as numeric data items, and to identify data items that contain non-numeric values ​​as non-numeric data items.

[0296] In an optional embodiment of the present invention, the bin-sorting method determination module 203 may include:

[0297] The first initial data value extraction submodule is used to extract multiple first initial data values ​​from the numerical data items.

[0298] The first initial data value sorting submodule is used to sort the various first initial data values;

[0299] The first initial data value equal-distance binning module is used to divide the sorted first initial data values ​​into equal-distance bins to obtain the first target data value of each bin.

[0300] The first target data value verification submodule is used to verify whether the first target data values ​​in the same bin have the same meaning.

[0301] The first binning method determination submodule is used to determine the equidistant binning as the binning method corresponding to the numerical data item if the condition is met.

[0302] In an optional embodiment of the present invention, after verifying whether the first target data values ​​in the same bin have the same meaning, the binning method determination module 203 may further include:

[0303] The first equal-frequency binning module is used to perform equal-frequency binning on the first target data value of each bin to obtain the third target data value of each bin if no.

[0304] The third target data value verification submodule is used to verify whether the third target data values ​​in the same bin have the same meaning.

[0305] The second binning method determination submodule is used to determine, if so, the equidistant binning and the equal-frequency binning as the binning method corresponding to the numerical data item.

[0306] Optionally, the first initial data value sorting submodule includes:

[0307] The encoding rule acquisition unit is used to acquire the encoding rule corresponding to the numerical data item;

[0308] A partitioning unit is used to partition each first initial data value according to the encoding rules to obtain multiple sets of sub-data corresponding to each first initial data value;

[0309] The first score acquisition unit acquires a first score for each group based on the first initial data value; the first score is used to characterize the correlation between each group of sub-data and the data item to which it belongs;

[0310] The first initial data value sorting unit is used to sort the multiple groups of sub-data based on the first score for each of the first initial data values.

[0311] Optionally, the bin-sorting method determination module includes:

[0312] The second initial data value extraction submodule is used to extract multiple second initial data values ​​from the non-numerical data items.

[0313] The second initial data value classification submodule is used to classify the plurality of second initial data values ​​to obtain different types of second initial data values;

[0314] The second initial data value sorting submodule is used to sort the second initial data values ​​of different types respectively;

[0315] The second equidistant binning module is used to divide the sorted multiple second initial data values ​​into equidistant bins to obtain the second target data value of each bin.

[0316] The second target data value verification submodule is used to verify whether the second target data values ​​in the same bin have the same meaning.

[0317] The third binning method determination submodule is used to determine the equidistant binning as the binning method corresponding to the non-numerical data item if the condition is met.

[0318] Optionally, after verifying whether the second target data values ​​in the same bin have the same meaning, the binning method determination module further includes:

[0319] The second equal-frequency binning module is used to perform equal-frequency binning on the second target data values ​​of each bin to obtain the fourth target data values ​​of each bin if no.

[0320] The fourth target data value verification submodule is used to verify whether the fourth target data values ​​in the same bin have the same meaning.

[0321] The fourth binning method determination submodule is used to determine the equidistant binning and the equal-frequency binning as the binning method corresponding to the non-numerical data item if the condition is met.

[0322] Optionally, the second initial data value sorting submodule includes:

[0323] A type quantity determination unit is used to determine the number of types involved in the second initial data values ​​of the different types;

[0324] The hash value calculation unit is used to calculate, for the same type of second initial data value, a first hash value corresponding to the data item description of the data item, a second hash value corresponding to the number of the type, and a third hash value corresponding to the same type of second initial data value;

[0325] The identifier determination unit is configured to, for a second initial data value of the same type, determine the first hash value as a first identifier, determine the second hash value as a second identifier, and determine the third hash value as a third identifier;

[0326] The second score determination unit is used to determine the second score corresponding to the first identifier, the second identifier, and the third identifier for the same type of second initial data value;

[0327] The second initial data value sorting unit is used to sort the second initial data values ​​of the same type based on the second score.

[0328] Optionally, the bin sorting method includes a numerical first bin sorting method and a non-numerical second bin sorting method; the target bin sorting method determination module includes:

[0329] The stability calculation submodule is used to calculate the first stability corresponding to the first binning method and the second stability corresponding to the second binning method.

[0330] The stability determination submodule is used to determine whether the first stability is within the target prediction capability range and whether the second stability is within the target prediction capability range.

[0331] The first target binning method determination submodule is used to determine the first binning method as the target binning method if the first stability is within the target prediction capability range, but the second stability is not within the target prediction capability range.

[0332] The second target binning method determination submodule is used to determine the second binning method as the target binning method if the second stability is within the target prediction capability range, but the first stability is not within the target prediction capability range.

[0333] Optionally, after determining whether the first stability is within the target prediction capability range and whether the second stability is within the target prediction capability range, the target binning method determination module further includes:

[0334] A stability comparison submodule is used to compare the first stability and the second stability if both the first stability and the second stability are within the target prediction capability range.

[0335] The third target binning method determination submodule is used to determine the first binning method as the target binning method if the first stability is greater than the second stability.

[0336] The fourth target binning method determination submodule is used to determine the second binning method as the target binning method if the second stability is greater than the first stability.

[0337] Optionally, the data security detection module to be stored includes:

[0338] The feature extraction submodule is used to extract the feature values ​​corresponding to the data to be stored in each bin;

[0339] The data quantity determination submodule is used to determine, based on the feature value, the first data quantity that meets the data requirements and the second data quantity that does not meet the data requirements in each bin;

[0340] The detection value calculation submodule is used to calculate the detection value of each sub-box using the first data quantity and the second data quantity.

[0341] In summary, in this embodiment of the invention, user-uploaded data to be stored is received, and a preset number of sample data are extracted from the data to be stored. The sample data includes multiple data items. These data items are classified based on data type to obtain different types of data items. A binning method corresponding to each type of data item is determined. From the binning methods corresponding to different types of data items, a target binning method whose stability meets preset conditions is determined. The target binning method is used to bin the data to be stored. Security checks are performed on the data to be stored in each bin to obtain the check value for each bin. When the check values ​​for each bin are greater than or equal to a preset threshold, the data to be stored is determined to have passed the security check, and the data that has passed the security check is stored. This embodiment of the invention strengthens data security control by performing security checks on the data before uploading it, ensuring the effectiveness of data security control. Furthermore, by binning the data and then performing security checks on the binned data, the efficiency of data security checks is improved.

[0342] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0343] This invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described data security detection method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here.

[0344] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described data security detection method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0345] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0346] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0347] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0348] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0349] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0350] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0351] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0352] The data security detection method, apparatus, electronic device, and computer-readable storage medium provided by this invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention. Regarding the apparatus embodiments, since they are basically similar to the method embodiments, the descriptions are relatively simple; relevant details can be found in the descriptions of the method embodiments.

Claims

1. A data security detection method, characterized in that, The method includes: Receive user-uploaded data to be stored, and extract a preset number of sample data from the data to be stored; the sample data includes multiple data items; The multiple data items are classified according to their data types to obtain different types of data items; Determine the binning method corresponding to each of the different types of data items; From the binning methods corresponding to the different types of data items, determine the target binning method whose stability meets the preset conditions; The data to be stored is binned using the target binning method described above. Perform security checks on the data to be stored in each bin and obtain the check values ​​for each bin; When the detection values ​​of each bin are greater than or equal to a preset threshold, the data to be stored is determined to have passed the security test, and the data to be stored that has passed the security test is stored. The data types include numeric and non-numeric types; the classification of the multiple data items based on data types to obtain different types of data items includes: From the plurality of data items, determine data items that do not contain non-numeric values, and determine data items that contain non-numeric values; The data items that do not contain non-numeric values ​​are defined as the numeric data items, and the data items that contain non-numeric values ​​are defined as the non-numeric data items.

2. The method according to claim 1, characterized in that, The step of determining the binning method corresponding to the different types of data items includes: For the numerical data item, extract multiple first initial data values ​​from the numerical data item; Sort the initial data values. The sorted initial data values ​​are binned at equal intervals to obtain the first target data value for each bin. Verify whether the first target data values ​​in the same bin have the same meaning; If so, then the equidistant binning is determined as the binning method corresponding to the numerical data item.

3. The method according to claim 2, characterized in that, After verifying whether the first target data values ​​in the same bin have the same meaning, the process further includes: If not, then the first target data values ​​of each sub-bin are divided into equal-frequency sub-bins to obtain the third target data values ​​of each sub-bin; Verify whether the third target data values ​​in the same bin have the same meaning; If so, then the equidistant binning and the equal-frequency binning are determined as the binning methods corresponding to the numerical data items.

4. The method according to claim 2, characterized in that, The sorting of the initial data values ​​includes: Obtain the encoding rules corresponding to the numerical data items; According to the encoding rules, each first initial data value is divided to obtain multiple sets of sub-data corresponding to each first initial data value; For each of the first initial data values, a first score is obtained for each group; the first score is used to characterize the correlation between each group of sub-data and the data item to which it belongs; For each of the first initial data values, the multiple groups of sub-data are sorted based on the first score.

5. The method according to claim 1, characterized in that, The step of determining the binning method corresponding to the different types of data items includes: For the non-numerical data item, extract multiple second initial data values ​​from the non-numerical data item; The plurality of second initial data values ​​are classified to obtain different types of second initial data values; Sort the second initial data values ​​of the different types respectively; The sorted initial data values ​​are binned at equal intervals to obtain the target data values ​​for each bin. Verify whether the second target data values ​​in the same bin have the same meaning; If so, then the equidistant binning is determined as the binning method corresponding to the non-numerical data item.

6. The method according to claim 5, characterized in that, After verifying whether the second target data values ​​in the same bin have the same meaning, the process further includes: If not, then the second target data values ​​of each sub-bin are divided into equal-frequency sub-bins to obtain the fourth target data values ​​of each sub-bin; Verify whether the fourth target data values ​​in the same bin have the same meaning; If so, then the equidistant binning and the equal-frequency binning are determined as the binning methods corresponding to the non-numerical data items.

7. The method according to claim 5, characterized in that, The sorting of the second initial data values ​​of the different types includes: Determine the number of types involved in the different types of second initial data values; For a second initial data value of the same type, calculate the first hash value corresponding to the data item description of the data item, calculate the second hash value corresponding to the number of the type, and calculate the third hash value corresponding to the second initial data value of the same type; For the same type of second initial data value, the first hash value is determined as the first identifier, the second hash value is determined as the second identifier, and the third hash value is determined as the third identifier; For the same type of second initial data value, determine the second score corresponding to the first identifier, the second identifier, and the third identifier respectively; The second initial data values ​​of the same type are sorted based on the second score.

8. The method according to claim 1, characterized in that, The bin sorting method includes a numerical first bin sorting method and a non-numerical second bin sorting method; determining the target bin sorting method whose stability meets preset conditions from the multiple bin sorting methods includes: Calculate the first stability corresponding to the first binning method, and calculate the second stability corresponding to the second binning method; Determine whether the first stability is within the target prediction capability range, and determine whether the second stability is within the target prediction capability range; If the first stability is within the target prediction capability range, while the second stability is not within the target prediction capability range, then the first binning method is determined as the target binning method; If the second stability is within the target prediction capability range, while the first stability is not within the target prediction capability range, then the second binning method is determined as the target binning method.

9. The method according to claim 8, characterized in that, After determining whether the first stability is within the target prediction capability range and whether the second stability is within the target prediction capability range, the method further includes: If both the first stability and the second stability are within the target prediction capability range, then compare the first stability and the second stability. If the first stability is greater than the second stability, then the first binning method is determined as the target binning method; If the second stability is greater than the first stability, then the second binning method is determined as the target binning method.

10. The method according to claim 1, characterized in that, The process of performing security checks on the data to be stored in each bin to obtain the detection value for each bin includes: Extract the feature values ​​corresponding to the data to be stored in each bin; Based on the characteristic values, determine the number of first data that meets the data requirements and the number of second data that does not meet the data requirements in each bin; The detection value of each sub-box is calculated using the first data quantity and the second data quantity.

11. A data security detection device, characterized in that, The device includes: The data to be stored receiving module is used to receive data to be stored uploaded by users and extract a preset number of sample data from the data to be stored; the sample data includes multiple data items; The data item classification module is used to classify the multiple data items based on data type to obtain different types of data items; The binning method determination module is used to determine the binning method corresponding to the different types of data items respectively; The target binning method determination module is used to determine the target binning method whose stability meets preset conditions from the binning methods corresponding to the different types of data items; The data binning module is used to bin the data to be stored using the target binning method. The data security detection module is used to perform security detection on the data to be stored in each bin and obtain the detection value of each bin. The data storage module is used to determine that the data to be stored has passed the security test when the detection values ​​of each bin are greater than or equal to a preset threshold, so as to store the data to be stored that has passed the security test. The data types include numeric and non-numeric types; the data item classification module includes: The non-numeric determination submodule is used to determine, from the plurality of data items, data items that do not contain non-numeric values, and data items that do contain non-numeric values; The data item classification submodule is used to identify data items that do not contain non-numeric values ​​as numeric data items, and to identify data items that contain non-numeric values ​​as non-numeric data items.

12. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the data security detection method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data security detection method as described in any one of claims 1 to 10.

Images