A method for identifying randomly combined uploaded fields

By constructing a function that represents the unreasonable degree of phrases, random combination upload fields in network attacks are identified and intercepted, which solves the problem that the existing technology is difficult to identify black industry attacks, and effectively prevents and controls the network.

CN112633009BActive Publication Date: 2024-12-06YANGZHOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011590192.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-29
Publication Date
2024-12-06
Estimated Expiration
2040-12-29

AI Technical Summary

Technical Problem

The existing technology is difficult to effectively identify and intercept network attacks carried out by black markets through random combination upload fields, resulting in difficulties in network security prevention and control.

Method used

By constructing a function that represents the unreasonable degree of phrases, using bag-of-word model and stem-stem extraction algorithm to preprocess the uploaded fields, calculate the unreasonable degree of each phrase, identify the phrases with safety hazards and intercept them.

Benefits of technology

It realizes the effective identification of abnormal upload fields under low computing complexity, identify and intercept black industry equipment, prevent network attacks, and improve data security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0002868740580000021
    Figure BDA0002868740580000021
  • Figure BDA0002868740580000041
    Figure BDA0002868740580000041
  • Figure BDA0002868740580000051
    Figure BDA0002868740580000051
Patent Text Reader

Abstract

The present invention discloses a method for identifying randomly combined uploaded fields, including 1) filtering out phrases or texts outside the scope of the algorithm, counting the number of text phrases and sorting them; 2) performing text preprocessing on the fields in the acquired phrases; 3) retaining phrases with the same number of elements, counting the number of occurrences of each phrase and the words in the preceding and succeeding terms of the phrases; 4) measuring the degree to which each phrase should not appear; 5) sorting the calculation results, and identifying the phrases ranked at the top as phrases with potential safety hazards. The present invention constructs a function that characterizes the unreasonable degree of phrases, and iteratively calculates each phrase with low computational complexity, so as to effectively mine abnormal uploaded fields, identify black production equipment, and thereby transform the system and intercept and deal with it in a timely manner.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an identification method, and in particular to an identification method for randomly combined upload fields. Background Art

[0002] At present, the well-known network security field has the problem of being vulnerable to black industry attacks. When the black industry conducts network attacks, in order to avoid being detected due to the same fields such as device and router names, they often use random text combinations to evade, such as lunch phone, helper delicious, etc., which contain both rare words and many common words, bringing a certain impact on network security prevention and control. However, if you look at the source, you can see the commonality of these uploaded fields - the essence is the random mechanical splicing of unrelated words, so if you look closely at each uploaded name or field, you will find that they have no practical meaning. The attacks of the black industry are trillions of levels, and it is very difficult to verify them by manual proofreading; At present, the main random detection methods include single-bit frequency detection, poker detection, matrix rank monitoring, etc. They mainly measure the randomness of the data itself at a significant level, and are mostly used in password randomness detection. Summary of the invention

[0003] The purpose of the present invention is to overcome the defects of the prior art and provide a method for identifying randomly combined uploaded fields. By constructing a function that characterizes the degree of unreasonableness of a phrase and iteratively calculating each phrase with lower computational complexity, abnormal uploaded fields can be effectively mined and black market equipment can be identified. The system can be transformed to intercept and deal with them in a timely manner.

[0004] The object of the present invention is achieved by: a method for identifying a random combination upload field, comprising the following steps:

[0005] Step 1) Screen out phrases or texts outside the scope of this algorithm, count the number of text phrases and sort them, use manual methods to remove common field phrases from high-frequency phrases, and form a whitelist library with the removed phrases for subsequent direct use;

[0006] Step 2) Perform text preprocessing on the fields in the acquired phrases, use a letter conversion function to unify the capitalization of letters, use a word restoration and stem extraction algorithm to remove word affixes; use a bag-of-words model to convert large-scale data sets into a mapping representation of numbers and simplified text;

[0007] Step 3) retain the phrases with the same number of elements, and count the number of occurrences of each phrase and the preceding and succeeding words of the phrase; if the number of elements is greater than 2, then count the phrases with the same number of elements in turn;

[0008] Step 4) Measure the degree to which each phrase should not appear. The larger the E value, the more unreasonable the combination is.

[0009] Step 5) Sort the calculation results in step 4), identify the top phrases as phrases with potential safety hazards, and mark them or intercept their subsequent operations at the system access layer.

[0010] As a further limitation of the present invention, the step 2) specifically comprises:

[0011] 2-1) Using the Porter stemming algorithm to remove the endings of common tenses in each word, simplifying the word into a stem;

[0012] 2-2) WordNet Lemmatizer algorithm uses the vocabulary knowledge base to restore the correct word form through semantic analysis, restores different forms of words to common basic forms, and removes the affixes of the words;

[0013] 2-3) The CountVectorizer algorithm is used to convert the bag-of-words model to effectively cope with large-scale data sets, reduce the amount of calculation, and convert the words in the text into a word frequency matrix containing statistical numbers, so that it can be converted into a mapping representation of numbers and simplified text.

[0014] As a further limitation of the present invention, the following formula is specifically used to measure the degree to which each phrase should not appear in step 4):

[0015]

[0016] Among them, p ab Measures the extent to which a and b appear at the same time when they should not; C ab Indicates the number of phrases containing both a and b, C a +C b -C ab The sum of the number of phrases containing a and b minus the number of phrases containing both a and b. ab Measures the frequency of co-occurrence of a and b, C total is the total number of phrases. To ensure that the denominator is not 0, it is added with 1. To reduce the deviation caused by the large total number of phrases, it is logarithmically processed.

[0017] As a further limitation of the present invention, in step 5), the subsequent operations will be marked or intercepted at the system access layer. Specifically, it is deployed at the system user access layer. When a large number of users with abnormal fields log in, the set threshold is triggered, and real-time alarms or interceptions are issued. The intercepted user field data results can be added to the high-risk login user label or blacklist library.

[0018] The present invention adopts the above technical solution, and compared with the prior art, the beneficial effects are: by constructing a function that characterizes the unreasonable degree of a phrase, and iteratively calculating each phrase with a lower computational complexity, it is possible to effectively mine abnormal upload fields, identify black market equipment, and thereby transform the system, intercept and deal with it in a timely manner, and transform the problem of identifying random and meaningless word combinations into identifying phrases that should not appear at the same time (irrational). The method of the present invention can identify the "random" and "unrelated" combination upload fields of black market gangs, and effectively intercept the black market's attacks on data security; it has low computational complexity and high practicality; it curbs the theft, order swiping and fraud of Internet information secrets by network black market at the source. DETAILED DESCRIPTION

[0019] A method for identifying randomly combined upload fields comprises the following steps:

[0020] Step 1) Screen out phrases or texts outside the scope of this algorithm, count the number of text phrases and sort them, use manual methods to remove common field phrases from high-frequency phrases, and form a whitelist library with the removed phrases for subsequent direct use;

[0021] Such as numbers (123), common nouns (such as my iphone) and special characters, only retain phrases consisting of English phrases, count the number of text phrases and sort them, use manual methods to remove common field phrases from high-frequency phrases, and form a whitelist library with the removed phrases for subsequent direct use. Use Python's Lower function to convert all the words in the phrase into lowercase letters.

[0022] Step 2) Perform text preprocessing on the fields in the acquired phrases, use a letter conversion function to unify the capitalization of letters, use a word restoration and stem extraction algorithm to remove word affixes; use a bag-of-words model to convert large-scale data sets into a mapping representation of numbers and simplified text;

[0023] 2-1) Use the Porter Stemmer algorithm in the Python Natural Language Toolkit (NLTK) to extract stems. The Porter stemming algorithm removes the endings of common tenses in each word and simplifies the words into stems. The algorithm provides tools such as plural judgment and vowel judgment to accurately extract stems;

[0024] 2-2) WordNet Lemmatizer algorithm uses the vocabulary knowledge base to restore the correct word form through semantic analysis, restores different forms of words to common basic forms, and removes the affixes of the words;

[0025] 2-3) Use the CountVectorizer module in the Python machine learning module sklearn to construct a bag-of-words model, and use the CountVectorizer algorithm to convert the bag-of-words model to effectively cope with large-scale data sets, reduce the amount of calculation, and convert the words in the text into a word frequency matrix containing statistical numbers, so that it can be converted into a mapping representation of numbers and simplified text; the stop word library is the stop words of each language provided by the sklearn module, the stop word library is English, and the filtering rule is to retain 2 or more letters. Table 1 shows the case preprocessing;

[0026] Table 1 Phrase preprocessing

[0027]

[0028]

[0029] Step 3) retain phrases with the same number of elements, and count the number of times each phrase and the words in the preceding and succeeding terms of the phrase appear; if the number of elements is greater than 2, then count the number of phrases with the same number of elements in turn; as shown in Table 2, in the last field name of all data sets visited, the phrase "ip helper" in sequence number 1 appeared 10 times at the same time, of which the preceding term "ip" appeared 12 times and the succeeding term "helper" appeared 12 times, and these three indicators of each phrase were calculated in this way.

[0030] Table 2 Phrase occurrence statistics

[0031] Serial number phrase Bag of Words Mapping Number of simultaneous occurrences Previous item number Number of subsequent items 1 [ip helper] (01 02)10 10 12 12 2 [helper delicious] (02 03)2 2 12 27 3 [delicious foods] (03 04)15 15 27 27 4 [delicious roast] (03 05)10 10 27 10 5 [good foods] (06 04)12 12 14 27 6 [ip martin] (01 07)2 2 12 2 7 [good milk] (06 08)2 2 14 2 8 [well milk] (09 08)2 2 2 4

[0032] Step 4) Measure the degree to which each phrase should not appear. The larger the E value, the more unreasonable the combination. The following formula is used to measure the degree to which each phrase should not appear:

[0033]

[0034] Among them, p ab Measures the extent to which a and b appear at the same time when they should not; C ab Indicates the number of phrases containing both a and b, C a +C b -C ab The sum of the number of phrases containing a and b minus the number of phrases containing both a and b. ab Measures the frequency of co-occurrence of a and b, C total is the total number of phrases. To ensure that the denominator is not 0, it is added with 1. To reduce the deviation caused by the large total number of phrases, it is logarithmically processed.

[0035] The sample data for calculation are shown in Table 3.ab With T ab , and take the product of them, the E value is the result after calculation:

[0036] For the phrase [ip helper] in sequence number 1, C a For 12, C b For 12, C ab 10, C total is 55, and we can calculate: p ab = (12+12-10) / 10=1.4, T ab It is ln(55 / (10+1))=1.609, and the E value is 2.25, which means the unreasonableness is relatively low;

[0037] For the phrase [helper delicious] in sequence 3, C a For 12, C b 27, C ab is 2, C total is 55, so we can calculate: p ab = (12+27-2) / 2 = 18.5, T ab It is ln(55 / (2+1))=2.909, and the E value is 53.81, which is quite unreasonable.

[0038] Table 3 Calculation process of unreasonable degree of phrases

[0039] Serial number phrase <![CDATA[p ab ]]> <![CDATA[T ab ]]> E Result 1 [ip helper] 1.400 1.609 2.25 Top 8 2 [helper delicious] 18.500 2.909 53.81 Top 1 3 [delicious foods] 2.600 1.235 3.21 Top 7 4 [delicious roast] 2.700 1.609 4.35 Top 5 5 [good foods] 2.417 1.442 3.49 Top 6 6 [ip martin] 6.000 2.909 17.45 Top 3 7 [good milk] 8.000 2.909 20.36 Top 2 8 [well milk] 2.000 2.909 5.82 Top 4

[0040] Step 5) Sort the results of step 4) and identify the phrases with the highest ranking as phrases with potential safety hazards. The E value of the phrase "helper delicious" ranked first (TOP1) in Table 3 is as high as 53.81. The preceding word helper appears 12 times and the following word delicious appears 27 times. When viewed separately, both words are common words, but they only appear twice together, which is exactly what we define as unreasonable phrases. The same is true for the second-ranked phrase "good milk" (TOP 2) and the third-ranked phrase "ip martin" (TOP 3); while the phrase "well milk" is an uncommon phrase, with two words appearing twice at the same time. Since the total number of occurrences of the preceding and following words is not large, the unreasonable degree is not as prominent as the former, and the E value is 5.82. It is a relatively normal combination, and the rest are normal phrases. It will be marked or its subsequent operations will be blocked at the system access layer. It will be deployed at the system user access layer. When a large number of users with abnormal fields log in, the set threshold will be triggered, and real-time alarms or interceptions will be issued. The intercepted user field data results can be added to the high-risk login user tag or blacklist library.

[0041] The present invention constructs a function that characterizes the degree of irrationality of a phrase and iteratively calculates each phrase with low computational complexity, thereby effectively mining abnormal upload fields and identifying black market devices, thereby transforming the system and intercepting and handling them in a timely manner.

[0042] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A method for identifying randomly combined upload fields, characterized in that: The following steps are involved: Step 1) Screen out phrases or texts outside the scope of this algorithm, count the number of text phrases and sort them, use manual methods to remove common field phrases from high-frequency phrases, and form a whitelist library with the removed phrases for subsequent direct use; Step 2) Perform text preprocessing on the fields in the acquired phrases, use a letter conversion function to unify the capitalization of letters, use a word restoration and stem extraction algorithm to remove word affixes; use a bag-of-words model to convert large-scale data sets into a mapping representation of numbers and simplified text; Step 3) retain the phrases with the same number of elements, and count the number of occurrences of each phrase and the preceding and succeeding words of the phrase; if the number of elements is greater than 2, then count the phrases with the same number of elements in turn; Step 4) Measure the degree to which each phrase should not appear. The larger the E value, the more unreasonable the combination. The following formula is used to measure the degree to which each phrase should not appear: Among them, p ab Measures the extent to which a and b appear at the same time when they should not; C ab Indicates the number of phrases containing both a and b, C a +C b -C ab is the sum of the number of phrases containing a and b, minus the number of phrases containing both a and b; T ab Measures the frequency of co-occurrence of a and b, C total is the total number of phrases. To ensure that the denominator is not 0, it is added with 1. To reduce the deviation caused by the large total number of phrases, it is logarithmically processed. Step 5) Sort the calculation results in step 4), identify the top phrases as phrases with potential safety hazards, and mark them or intercept their subsequent operations at the system access layer.

2. A method for identifying randomly combined upload fields according to claim 1, characterized in that: The step 2) specifically includes: 2-1) Using the Porter stemming algorithm to remove the endings of common tenses in each word, simplifying the word into a stem; 2-2) WordNet Lemmatizer algorithm uses the vocabulary knowledge base to restore the correct word form through semantic analysis, restores different forms of words to common basic forms, and removes the affixes of the words; 2-3) The CountVectorizer algorithm is used to convert the bag-of-words model to effectively cope with large-scale data sets, reduce the amount of calculation, and convert the words in the text into a word frequency matrix containing statistical numbers, so that it can be converted into a mapping representation of numbers and simplified text.

3. The method for identifying a random combination upload field according to claim 1, characterized in that: In the step 5), the subsequent operations will be marked or intercepted at the system access layer. Specifically, it is deployed at the system user access layer. When a large number of users with abnormal fields log in, the set threshold is triggered, and real-time alarms or interceptions are issued. The intercepted user field data results can be added to the high-risk login user tag or blacklist library.