Domain name recognition library updating method and system, domain name recognition method and server

Abstract

The application discloses a domain name recognition library updating method, a system, a domain name recognition method and a server, wherein the domain name recognition library updating method is used for comparing a to-be-recognized domain name with a domain name recognition library after mapping the to-be-recognized domain name, so as to determine which mapping domain name in the domain name recognition library is the same as the mapping domain name of the to-be-recognized domain name, and then determine whether the to-be-recognized domain name is legal according to the type of the domain name corresponding to the mapping domain name in the matching, so that the requirement of determining whether the to-be-recognized domain name is legal or not during automatic updating of the domain name recognition library can be effectively met; and the mapping of the domain name is performed through an ordered list and a mapping dictionary, so that the similarity detection of the domain name can be quickly performed, the development is facilitated, a complex mathematical principle is not needed, a complex calculation process is not needed, the calculation efficiency is high, the processing rate is fast, the recognition requirement of a large number of domain names can be met, and the updating rate of the domain name recognition library is improved.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a method, system, domain name identification method and server for updating a domain name identification database. Background Technology

[0002] A domain name consists of two or more groups of ASCII or other language characters, separated by periods. The rightmost group of characters is called the top-level domain or first-level domain, the second to last group is called the second-level domain, the third to last group is called the third-level domain, and so on.

[0003] Top-level domains (TLDs) are divided into three categories: first, country code top-level domains (ccTLDs), such as .cn for China and .jp for Japan; second, general top-level domains (GTLDs), such as .com for businesses and .net for internet service providers; and third, new top-level domains (NTLDs), such as the general .xyz and .top for "high-end" domains.

[0004] On the internet, there are often various illegal domain names or imitations of other people's domain names used to achieve illegal purposes. Therefore, it is crucial to check whether a domain name is legal.

[0005] When performing domain name identification, as shown in the patent document with authorization announcement number CN112751804B, the identification is usually performed by comparing all or part of the domain name to be identified with a pre-determined domain name database.

[0006] Therefore, the domain name database is the foundation for accurate identification and needs to be updated in a timely manner. However, the existing update methods can only add newly acquired domain names to the domain name database, but cannot automatically identify whether each newly added domain name is legitimate. Summary of the Invention

[0007] The purpose of this invention is to solve the above-mentioned problems existing in the prior art and to provide a domain name recognition database update method, system, domain name recognition method and server.

[0008] The objective of this invention is achieved through the following technical solution:

[0009] The method for updating the domain name recognition database includes the following steps:

[0010] S1, Obtain a batch of domain names to be identified;

[0011] S2, identify each domain name to be identified according to the following process;

[0012] S21, determine the domain name to be identified corresponding to a domain name to be identified based on the determined ordered list and mapping method;

[0013] S22, compare the domain name to be identified with the mapped domain names in the domain name identification database to determine whether the domain name to be identified matches a mapped domain name in the domain name identification database; if not, proceed to S23; if yes, proceed to S24;

[0014] S23, add the domain name to be identified to the domain name database to be classified;

[0015] S24, Determine whether the types of all domains corresponding to the mapped domains in the matching are consistent. If yes, proceed to S25; otherwise, proceed to S26. The type of a domain refers to whether a domain is a legal domain or an illegal domain.

[0016] S25, add the domain name to be identified to the domain name identification database, and determine that the type of the domain name to be identified is the type of all domain names corresponding to the mapped domain name in the matching;

[0017] S26, Determine whether the percentage of legitimate domains among all domains corresponding to the mapped domains in the matching process reaches the first threshold. If not, proceed to S27; if yes, proceed to S28.

[0018] S27, Add the domain name to be identified to the domain name database to be classified;

[0019] S28, add the domain name to be identified to the domain name identification database, determine that the type of the domain name to be identified is a legal domain name, and add a question mark to the domain name to be identified;

[0020] S3, after all domain names to be identified have been identified, confirm the proportion of suspicious domain names corresponding to each mapped domain name in the domain name identification database. The proportion of suspicious domain names corresponding to a mapped domain name is the proportion of domain names with suspicious tags among all domain names corresponding to the mapped domain name. When it is determined that the proportion of suspicious domain names corresponding to a mapped domain name exceeds the second threshold, transfer the domain names with suspicious tags among all domain names corresponding to the mapped domain name from the domain name identification database to the domain name database to be classified, and obtain the updated domain name database to be classified and the domain name identification database.

[0021] Preferably, the ordered list comprises letters and / or numbers and / or symbols arranged in sequence, with one and only one of each letter and / or number and / or symbol.

[0022] Preferably, determining the domain name to be identified corresponding to the domain name to be identified based on the determined ordered list and mapping method includes the following process:

[0023] S211, Establish an initial empty mapping dictionary for the domain name to be identified, which is updated according to a predetermined rule;

[0024] S212, take all or part of the character groups in the domain name to be identified as the domain name to be mapped, and map all the characters to be mapped in the domain name to be mapped except for the period in the order from left to right or from right to left to obtain the target mapping characters of each character to be mapped according to the ordered list and mapping dictionary.

[0025] S213, after mapping all the characters to be mapped in the domain name to be mapped to the corresponding target mapping characters, combine them with the original period in the domain name to be identified in their original order to obtain the domain name to be identified.

[0026] Preferably, in step S212, the top-level domain is removed from the domain name to be identified and the domain name to be mapped is obtained.

[0027] In step S213, after mapping all the characters to be mapped in the domain name to be mapped to the corresponding target mapping characters, they are combined with the original top-level domain and period in the domain name to be identified in their original order to obtain the domain name to be identified.

[0028] Preferably, before mapping, it is first determined whether the number of character types of the domain name to be mapped exceeds the number of characters in the ordered list. If so, the domain name to be mapped is truncated so that the number of character types of the truncated domain name does not exceed the number of characters in the ordered list.

[0029] Preferably, in step S212, when mapping each character to be mapped, it is first determined whether there is a mapping relationship corresponding to the character to be mapped in the mapping dictionary;

[0030] If so, according to the mapping relationship, the character to be mapped is mapped to the corresponding target mapped character;

[0031] If not, determine which first occurrence of the character to be mapped is in the domain name to be mapped, and select the corresponding character from the ordered list as the target mapping character of the character to be mapped, and map the character to be mapped to the target mapping character, and store the mapping relationship between the character to be mapped and the target mapping character in the mapping dictionary.

[0032] Preferably, in step S3, after obtaining the updated domain name database to be classified, the type of each domain name in the domain name database to be classified is determined using the second identification method;

[0033] Add the domain name whose type has been determined in the domain name classification database to the domain name identification database;

[0034] Domains whose types are not identified in the domain classification database will be retained in the database and compared with the updated domain identification database in the next comparison.

[0035] The domain name identification database update system includes:

[0036] The domain name acquisition unit is used to acquire a batch of domain names to be identified.

[0037] The identification unit, used to identify each domain name to be identified, includes:

[0038] The mapping module is used to determine the mapping domain name corresponding to a domain name to be identified based on a defined ordered list and mapping method.

[0039] The matching module is used to compare the domain name to be identified with the mapping domain names in the domain name identification database to determine whether the domain name to be identified matches a mapping domain name in the domain name identification database. If not, it communicates with the first classification module; if yes, it communicates with the second classification module.

[0040] The first classification module is used to add the domain name to be identified to the domain name database to be classified;

[0041] The second classification module is used to determine whether the types of all domains corresponding to the mapped domains in the matching are consistent. If yes, it communicates with the third classification module; otherwise, it communicates with the fourth classification module. The type of a domain refers to whether a domain is a legal domain or an illegal domain.

[0042] The third classification module is used to add the domain name to be identified to the domain name identification database and determine the type of the domain name to be identified as the type of all domain names corresponding to the mapped domain name in the matching.

[0043] The fourth classification module determines whether the proportion of legitimate domains among all domains corresponding to the mapped domains in the matching process reaches the first threshold. If not, it communicates with the fifth classification module; if so, it communicates with the sixth classification module.

[0044] The fifth classification module is used to add the domain name to be identified to the domain name database.

[0045] The sixth classification module is used to add the domain name to be identified to the domain name identification database, determine that the domain name to be identified is a legal domain name, and add a question mark to the domain name to be identified;

[0046] The questionable domain name identification unit is used to confirm the proportion of questionable domain names corresponding to each mapped domain name in the domain name identification database after all domain names to be identified have been identified. The proportion of questionable domain names corresponding to a mapped domain name is the proportion of domain names with questionable tags among all domain names corresponding to the mapped domain name. When it is determined that the proportion of questionable domain names corresponding to a mapped domain name exceeds a second threshold, the domain names with questionable tags among all domain names corresponding to the mapped domain name are transferred from the domain name identification database to the domain name database to be classified, thus obtaining an updated domain name database to be classified and a domain name identification database.

[0047] The domain name identification method uses the domain name identification database obtained by any of the domain name identification database update methods described above to identify the domain name accessed by the user.

[0048] The server includes a memory and a processor. The memory stores a program that can be executed by the processor. When the program is executed, it implements the domain name recognition database update method as described above and / or the domain name recognition method as described above.

[0049] The advantages of the technical solution of this invention are mainly reflected in:

[0050] This invention maps domain names using ordered lists and mapping dictionaries, enabling the identification of domain name naming patterns. Mapped domain names can be used to distinguish between illegal and legal domain names. Therefore, when updating the domain name identification database, the domain name to be identified can be mapped and compared with the database to determine which type of mapped domain name in the database matches the target domain name. The legality of the target domain name can then be determined based on the type of the matched mapped domain name. This effectively meets the requirement of determining the legality of the target domain name during automatic updates to the domain name identification database. Furthermore, mapping domain names using ordered lists and mapping dictionaries allows for rapid domain name similarity detection without requiring complex mathematical principles, making it easy to develop. It also eliminates the need for complex calculations, resulting in high computational efficiency and fast processing speed, meeting the needs of identifying large batches of domain names and improving the update speed of the domain name identification database.

[0051] When the domain name recognition library of this invention is used to identify the domain name accessed by a user, it can quickly find domain names of the same type by mapping domain names, thereby quickly determining whether the domain name accessed by the user is legitimate and performing corresponding access control, which greatly improves the processing speed. Attached Figure Description

[0052] Figure 1 This is a schematic diagram illustrating the process of updating the domain name recognition database according to the present invention;

[0053] Figure 2 This is a schematic diagram illustrating the process of mapping a domain name to be identified to obtain a mapped domain name in this invention;

[0054] Figure 3 This is a schematic diagram illustrating the process of mapping characters to be mapped and updating the mapping dictionary in this invention;

[0055] Figure 4 This is a schematic diagram illustrating the process of obtaining the mapped domain name by removing a portion of the character group from the domain name to be identified in this invention. Detailed Implementation

[0056] The objectives, advantages, and features of this invention will be illustrated and explained through the following non-limiting description of preferred embodiments. These embodiments are merely typical examples of applying the technical solutions of this invention, and all technical solutions formed by equivalent substitutions or equivalent transformations fall within the scope of protection claimed by this invention.

[0057] In the description of the solution, it should be noted that the terms "center," "upper," "lower," "left," "right," "front," "rear," "vertical," "horizontal," "inner," and "outer," etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings and are only for the convenience and simplification of description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0058] Example 1

[0059] The domain name identification database update method disclosed in this invention will be described below with reference to the accompanying drawings. Figure 1 As shown, it includes the following steps:

[0060] S1, Obtain a batch of domain names to be identified;

[0061] S2, identify each domain name to be identified according to the following process;

[0062] S21, determine the domain name to be identified corresponding to a domain name to be identified based on the determined ordered list and mapping method;

[0063] S22, compare the domain name to be identified with the mapping domain name in the domain name identification database to determine whether the domain name to be identified matches a mapping domain name in the domain name identification database; if not, proceed to S23; if yes, proceed to S24; the type of a domain name refers to whether a domain name is a legal domain name or an illegal domain name;

[0064] S23, add the domain name to be identified to the domain name database to be classified;

[0065] S24. Determine whether the types of all domains corresponding to the mapped domains in the match are consistent. If yes, proceed to S25; otherwise, proceed to S26.

[0066] S25, add the domain name to be identified to the domain name identification database, and determine that the type of the domain name to be identified is the type of all domain names corresponding to the mapped domain name in the matching;

[0067] S26, Determine whether the percentage of legitimate domains among all domains corresponding to the mapped domains in the matching process reaches the first threshold. If not, proceed to S27; if yes, proceed to S28.

[0068] S27, Add the domain name to be identified to the domain name database to be classified;

[0069] S28, add the domain name to be identified to the domain name identification database, determine that the type of the domain name to be identified is a legal domain name, and add a question mark to the domain name to be identified;

[0070] S3, after all domain names to be identified have been identified, confirm the proportion of suspicious domain names corresponding to each mapped domain name in the domain name identification database. The proportion of suspicious domain names corresponding to a mapped domain name is the proportion of domain names with suspicious tags among all domain names corresponding to the mapped domain name. When it is determined that the proportion of suspicious domain names corresponding to a mapped domain name exceeds the second threshold, transfer the domain names with suspicious tags among all domain names corresponding to the mapped domain name from the domain name identification database to the domain name database to be classified, and obtain the updated domain name database to be classified and the domain name identification database.

[0071] In step S1, the domain names to be identified can be obtained from the acquired traffic data at a certain period, for example, daily at a set time from the traffic data acquired throughout the day, such as at 24:00 every day, obtaining a batch of domain names to be identified from the daily traffic data. Specifically, when obtaining the domain names to be identified, a packet capture tool can be set up on the gateway. This tool can capture network packet data on the gateway and parse domain name information from the packet data. When performing various internet access operations under this gateway, the corresponding domain name information can be obtained, i.e., the domain names to be identified can be obtained. The domain names to be identified can also be domain names stored in a domain name classification database. After the domain name identification database is updated and expanded a predetermined number of times, the updated domain name identification database is compared again with the domain names in the domain name classification database, thereby adding the domain names in the domain name classification database to the domain name identification database.

[0072] In step S21, the determined ordered list includes letters and / or numbers and / or symbols arranged in sequence. The letters can be uppercase or lowercase of the 26 English letters, Greek letters, etc. The numbers can be Arabic numerals, Roman numerals, Chinese numerals, etc. The symbols can be mathematical symbols, punctuation marks, etc., and each letter and / or number and / or symbol has one and only one instance.

[0073] For example, the ordered list is defined as follows:

[0074] [a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,0,1,2,3,4,5,6,7,8,9].

[0075] As attached Figure 2 As shown, determining the corresponding mapped domain name based on the determined ordered list and mapping method includes the following process:

[0076] S211, Establish an initial empty mapping dictionary for the domain name to be identified, which is updated according to a predetermined rule;

[0077] S212, take all or part of the character groups in the domain name to be identified as the domain name to be mapped, and map all the characters to be mapped in the domain name to be mapped except for the period according to the ordered list and the mapping dictionary to obtain the target mapping characters of each character to be mapped.

[0078] As attached Figure 3 As shown, when mapping each character to be mapped in sequence, it is first determined whether there is a mapping relationship corresponding to the character to be mapped in the mapping dictionary;

[0079] If so, according to the mapping relationship, the character to be mapped is mapped to the corresponding target mapped character;

[0080] If not, determine which first occurrence of the character to be mapped is in the domain name to be mapped, and select the corresponding character from the ordered list as the target mapping character of the character to be mapped, and map the character to be mapped to the target mapping character, and store the mapping relationship between the character to be mapped and the target mapping character in the mapping dictionary.

[0081] S213, after mapping all the characters to be mapped in the domain name to be mapped to the corresponding target mapping characters, combine them with the original period in the domain name to be identified in their original order to obtain the domain name to be identified.

[0082] As attached Figure 4As shown, in order to reduce the mapping workload, in step S212, the top-level domain name in the domain name to be identified is removed and used as the domain name to be mapped.

[0083] In step S213, after mapping all the characters to be mapped in the domain name to be mapped to the corresponding target mapping characters, they are combined with the original top-level domain and period in the domain name to be identified in their original order to obtain the domain name to be identified.

[0084] For example, consider the following domain name to be identified: weixin.qq.com. An empty mapping dictionary {} is defined for this domain name. The mapping dictionary {} is updated based on the ordered list in the example above. Furthermore, after removing the top-level domain from the domain name to be identified, the corresponding domain name to be mapped is determined to be weixin.qq.

[0085] When mapping begins, the first character to be mapped in the domain name is determined to be "w". Since there is no corresponding mapping relationship for this character in the mapping dictionary {}, it is determined which first occurrence of the character "w" in the domain name. Based on the domain name, it can be determined that "w" is the first first occurrence of the character to be mapped. Then, the first character "a" is selected from the ordered list as its target mapping character, thereby establishing the mapping relationship between the character "w" and the target mapping character "a". The mapping relationship is then stored in the mapping dictionary {} to obtain the updated mapping dictionary {w:a}. The second character to be mapped is "e". This character also has no mapping relationship in the updated mapping dictionary. Since it is the second character to appear for the first time in the domain name to be mapped, the second character "b" is selected from the ordered list as the target mapping character for "e". This determines the mapping relationship between "e" and "b", and stores this mapping relationship in the mapping dictionary to obtain the updated mapping dictionary {w:a,e:b}. This process continues until the final updated mapping dictionary for the domain name is {w:a,e:b,i:c,x:d,n:e,q:f}. The mapped character is abcdceff. The mapped character, combined with the original period and top-level domain in their original order within the domain name to be identified, results in the domain name to be identified: abcdce.ff.com.

[0086] Furthermore, to reduce computational load, the number of characters in the ordered list is limited, preferably 36 characters as in the example above. Correspondingly, if a domain name to be mapped has more than 36 character types, the excess characters cannot be accurately mapped. Therefore, in S212, before mapping, it is first determined whether the number of character types in the domain name to be mapped exceeds the number of characters in the ordered list. If so, the domain name to be mapped is truncated so that the number of character types in the truncated domain name does not exceed the number of characters in the ordered list. That is, if the number of character types in the domain name to be mapped exceeds 36, it needs to be truncated to ensure that the number of character types does not exceed 36. When truncating the domain name to be mapped, second-level and third-level domain names are preferentially retained. This is because second-level and third-level domain names generally represent the company to which the domain name belongs. The domain name specification method of a single company is generally consistent, and the corresponding types should also be consistent. Therefore, retaining second-level and third-level domain names helps ensure a certain level of accuracy.

[0087] In step S22, in the initial state, the domain names stored in the domain name identification database are domain names that have been manually confirmed. That is, each domain name in the domain name identification database has been manually accessed and its type has been determined. The type of the domain name is whether it is a legal domain name or an illegal domain name. A legal domain name is a domain name whose content complies with legal requirements, and an illegal domain name is a domain name whose content does not comply with legal requirements.

[0088] The domain name identification database stores both legal and illegal domain names. Each domain name in the database has been mapped to its corresponding mapping domain name according to the above ordered list and mapping method. Each domain name corresponds to one mapping domain name, and one mapping domain name may correspond to multiple different domain names.

[0089] The domain name identification database stores information including domain names, their corresponding mapped domain names, types, and tag information.

[0090] Furthermore, the domain name identification database can also store the validity information corresponding to each mapped domain name. This validity information is determined based on the types of all domain names corresponding to that mapped domain name, for example:

[0091] If all domains corresponding to a mapped domain are valid domains, then the validity information of the mapped domain is valid; conversely, if all domains corresponding to a mapped domain are invalid domains, then the validity information of the mapped domain is invalid.

[0092] If a mapped domain name contains both legal and illegal domain names among all the domain names it corresponds to, then the legality information of the mapped domain name is the proportion of legal domain names and / or the proportion of illegal domain names.

[0093] Simultaneously, based on the legality information of each mapped domain, different domain identification databases can be created. For example, if the legality information of a mapped domain is invalid, then all such mapped domains and their corresponding domains are stored in an invalid domain identification database, meaning all domains in the invalid domain identification database are invalid. If the legality information of a mapped domain is valid, then all such mapped domains and their corresponding domains are stored in a valid domain identification database, meaning all domains in the valid domain identification database are valid. If the legality information of a mapped domain represents a percentage of valid domains and / or a percentage of invalid domains, then such mapped domains and their corresponding domains are stored in a suspicious domain identification database.

[0094] When identifying the domain name accessed by the user, the domain name to be identified can be matched with the illegal domain name identification database first. If no match is found, it can be matched with the suspicious domain name identification database. If no match is found again, it can be matched with the legal domain name identification database. This helps to determine whether the domain name accessed by the user is an illegal domain name more quickly.

[0095] The following example illustrates how to determine whether a domain name to be identified matches a domain name in the domain name identification database. For instance, the domain name to be identified, hhqq789.com, has a domain name to be identified as aabbcde.com. If the domain name identification database stores a domain name, yyll456.com, whose domain name is also aabbcde.com, then the domain name stored in the database matches the domain name to be identified. Therefore, the domain name to be identified as aabbcde.com matches a domain name in the domain name identification database.

[0096] For example, the domain name to be identified for the domain name weixin.qq.com mentioned above is abcdce.ff.com. If there is no mapping domain name in the domain name identification database that is the same as the domain name to be identified for abcdce.ff.com, then it is determined that the domain name to be identified for abcdce.ff.com does not match the mapping domain name in the domain name identification database.

[0097] In step S23, when adding the domain name to be identified to the domain name to be classified into the domain name database, the domain name to be identified and its corresponding mapping domain name are added to the domain name database.

[0098] In step S24, it is determined whether the types of all domains corresponding to the mapped domains in the matching are consistent, that is, it is determined which of the following situations applies to the types of all domains corresponding to the mapped domains in the matching:

[0099] (1) All are legitimate domain names;

[0100] (2) All of them are illegal domain names;

[0101] (3) There are both legal domain names and illegal domain names.

[0102] If all domains corresponding to the matched mapped domains are of the same type, or all are of the same type, then it is determined that all domains corresponding to the matched mapped domains are of the same type. Conversely, if all domains corresponding to the matched mapped domains include both legal and illegal domains, then it is determined that all domains corresponding to the matched mapped domains are of different types.

[0103] Correspondingly, in S25, assuming that all domains corresponding to the mapped domains in the domain name matching to be identified are valid domains, the type of the domain name to be identified is determined to be a valid domain name; assuming that all domains corresponding to the mapped domains in the domain name matching to be identified are invalid domains, the type of the domain name to be identified is determined to be an invalid domain name. When adding the domain name to be identified to the domain name identification database, the domain name to be identified, the mapped domain name, and the type are all added to the domain name identification database.

[0104] In step S26, the first threshold can be set as needed, for example, the first threshold is not less than 70%, and more preferably not less than 80%.

[0105] In step S28, the type of the domain name to be identified is determined to be a legitimate domain name, and a question mark is added to the domain name to be identified. This indicates that the type of the domain name to be identified is questionable and requires further observation and confirmation. Correspondingly, the domain name identification database also stores the marking information corresponding to each domain name. The marking information can distinguish different information through different symbols or characters. For example, 0 and 1 are used to indicate whether there is a question mark. Specifically, when it is determined that the type of a domain name to be identified is questionable, the marking information corresponding to the domain name to be identified is set to 1, which means that a question mark has been added to the domain name to be identified; conversely, when it is determined that the type of a domain name to be identified is not questionable, its corresponding marking information is 0.

[0106] In step S3, it is sequentially confirmed whether the proportion of domains with suspicious flags among all domains corresponding to each mapped domain in the domain identification database exceeds a second threshold. Specifically, this involves checking the proportion of domains with a flag of 1 among the domains corresponding to each mapped domain in the domain identification database. The second threshold can be designed as needed, for example, it can be between 10% and 20%. Taking 20% ​​as an example, if the proportion of domains with a flag of 1 among the domains corresponding to a mapped domain in the domain identification database exceeds 20%, then all domains with a flag of 1 and their related information are removed from the domain identification database and added to the domain classification database.

[0107] After completing the above steps, you will obtain the updated domain name identification database and the domain name database to be classified.

[0108] Furthermore, a second identification method is employed to identify the domains in the domain name database to be classified. This second identification method can be, for example, manual detection, whereby a person manually visits the relevant domain name and views its page content to determine the corresponding type. If the type can be determined, the domain name, its corresponding mapped domain name, and type are added to the domain name identification database. Otherwise, the domain name is retained in the domain name database to be classified for further comparison.

[0109] The second identification method can also determine the domain name type by crawling web page content, converting the crawled content into vector data, and then inputting it into a trained classification model. The method of training a classification model by crawling web page content and then classifying the content using that model is a known technique. For example, knowing the domain name, a script can be written using a programming language such as Python. When the script is executed, it simulates a human accessing the website corresponding to the domain name and saving the web page content. After crawling the web page content, the text content, IP address, IP location, and other information are converted into vector data. This vector data is then used to train a classification model from an existing model library in Python to obtain the classification model. The classification model could be, for example, the FastText model. A converged classification model can then be used for classification.

[0110] Furthermore, the domains in the domain name database to be classified can be compared with the updated domain name recognition database at certain intervals, such as once a week.

[0111] Example 2

[0112] This embodiment discloses a domain name recognition database update system, including:

[0113] The domain name acquisition unit is used to acquire a batch of domain names to be identified.

[0114] The identification unit, used to identify each domain name to be identified, includes:

[0115] The mapping module is used to determine the mapping domain name corresponding to a domain name to be identified based on a defined ordered list and mapping method.

[0116] The matching module is used to compare the domain name to be identified with the mapping domain names in the domain name identification database to determine whether the domain name to be identified matches a mapping domain name in the domain name identification database. If not, it communicates with the first classification module; if yes, it communicates with the second classification module.

[0117] The first classification module is used to add the domain name to be identified to the domain name database to be classified;

[0118] The second classification module is used to determine whether the types of all domain names corresponding to the mapped domain names in the matching are consistent. If yes, it communicates with the third classification module; otherwise, it communicates with the fourth classification module.

[0119] The third classification module is used to add the domain name to be identified to the domain name identification database and determine the type of the domain name to be identified as the type of all domain names corresponding to the mapped domain name in the matching.

[0120] The fourth classification module determines whether the proportion of legitimate domains among all domains corresponding to the mapped domains in the matching process reaches the first threshold. If not, it communicates with the fifth classification module; if so, it communicates with the sixth classification module.

[0121] The fifth classification module is used to add the domain name to be identified to the domain name database.

[0122] The sixth classification module is used to add the domain name to be identified to the domain name identification database, determine that the domain name to be identified is a legal domain name, and add a question mark to the domain name to be identified;

[0123] The questionable domain name identification unit is used to confirm the proportion of questionable domain names corresponding to each mapped domain name in the domain name identification database after all domain names to be identified have been identified. The proportion of questionable domain names corresponding to a mapped domain name is the proportion of domain names with questionable tags among all domain names corresponding to the mapped domain name. When it is determined that the proportion of questionable domain names corresponding to a mapped domain name exceeds a second threshold, the domain names with questionable tags among all domain names corresponding to the mapped domain name are transferred from the domain name identification database to the domain name database to be classified, thus obtaining an updated domain name database to be classified and a domain name identification database.

[0124] Example 3

[0125] This embodiment discloses a domain name identification method, which uses a domain name identification database obtained by the domain name identification database update method described above to identify the domain name accessed by the user.

[0126] Specifically, after obtaining the domain name accessed by the user, the corresponding mapping domain name to be identified is determined based on the established ordered list and mapping method. Then, the mapping domain name to be identified is compared with the mapping domain names in the domain name identification database to determine whether the mapping domain name to be identified matches a mapping domain name in the database. If not, other known domain name identification methods can be used to determine whether the domain name accessed by the user is legitimate, for example, the above-mentioned classification model can be used to determine the type of the domain name. If yes, the legitimacy of the domain name accessed by the user can be determined based on the situation of the domain name corresponding to the matching mapping domain name and / or the legitimacy information of the matching mapping domain name.

[0127] When determining whether the domain name accessed by the user is legitimate based on the domain name corresponding to the mapped domain name in the matching, the following process is followed:

[0128] Determine the type of the domain name corresponding to the mapped domain name in the match.

[0129] If it is determined that all domain names corresponding to the mapped domain names in the matching are valid domain names, then the domain name accessed by the user is determined to be a valid domain name;

[0130] If it is determined that all domain names corresponding to the mapped domain names in the matching are illegal domain names, then the domain name accessed by the user is determined to be an illegal domain name;

[0131] If it is determined that among all domains corresponding to the mapped domain in the matching process, there are both illegal and legal domains, the domain accessed by the user can be matched first with all illegal domains corresponding to the mapped domain in the matching process. If a match is found, the domain accessed by the user is determined to be illegal. If no match is found, the domain accessed by the user is then matched with all legal domains corresponding to the mapped domain in the matching process. If a match is found, the domain accessed by the user is determined to be legal. If no match is found, the legality of the domain accessed by the user can be determined based on the proportion of illegal and / or legal domains. For example, if the proportion of legal domains among the domains corresponding to the mapped domain in the matching process is greater than 80%, the domain accessed by the user is determined to be legal. Conversely, if the proportion is less than 80%, the domain accessed by the user is determined to be illegal.

[0132] Alternatively, if no valid domain name matches any of the mapped domain names, the type of domain name accessed by the user can be determined by crawling the web page content and classifying it using a classification model.

[0133] When determining the legitimacy of a domain name accessed by a user directly based on the legitimacy information corresponding to the mapped domain name in the matching, the following process is followed:

[0134] If the validity information corresponding to the mapped domain name in the match is determined to be valid, then the domain name accessed by the user is determined to be a valid domain name;

[0135] If the legality information corresponding to the mapped domain name in the match is determined to be illegal, then the domain name accessed by the user is determined to be an illegal domain name;

[0136] If the legality information corresponding to the mapped domain name is determined to be the percentage of legal domain names and / or the percentage of illegal domain names, then the domain name accessed by the user can be matched with all illegal domain names corresponding to the matched mapped domain names first. If a match is found, the domain name accessed by the user is determined to be an illegal domain name; if no match is found, the domain name accessed by the user is then matched with all legal domain names corresponding to the matched mapped domain names. If a match is found, the domain name accessed by the user is determined to be a legal domain name; if no match is found, the legality of the domain name accessed by the user is determined based on the percentage of legal domain names and / or the percentage of illegal domain names. For example, if the percentage of legal domain names is greater than 80%, the domain name accessed by the user is confirmed to be a legal domain name; otherwise, the domain name accessed by the user is determined to be an illegal domain name.

[0137] This approach can achieve higher efficiency. Of course, both methods can be used for identification and cross-verification to ensure accuracy.

[0138] If the domain name accessed by the user is determined to be a legitimate domain name, the user is allowed to access it. Conversely, if the domain name accessed by the user is an illegitimate domain name, the user is blocked from accessing the domain name through DNS interception or other means, and a corresponding prompt is issued.

[0139] This method can quickly identify illegal domain names because most illegal domain names use a fixed naming pattern. The ordered list and mapping method described above can quickly identify the naming pattern of illegal domain names. For example, assuming the domain name identification database stores the domain name hhqq789.com as an example above, which is an illegal domain name and its corresponding mapping domain name is aabbcde.com, if the domain name accessed by the user is yyll456.com, and its corresponding mapping domain name to be identified is also aabbcde.com, then after the mapping domain name to be identified is matched with the domain name identification database, it can be quickly confirmed that the domain name yyll456.com accessed by the user is an illegal domain name.

[0140] Example 4

[0141] This embodiment discloses a server, including a memory and a processor. The memory stores a program that can be executed by the processor. When the program is executed, it implements the domain name identification database update method as described above.

[0142] This invention has many other embodiments, and all technical solutions formed by equivalent transformation or equivalent transformation fall within the protection scope of this invention.

Claims

1. A method for updating a domain name recognition database, characterized in that, Includes the following steps: S1, Obtain a batch of domain names to be identified; S2, identify each domain name to be identified according to the following process; S21, Determine the domain name to be identified corresponding to a domain name to be identified based on the determined ordered list and mapping method, which includes the following process: S211, Establish an initial empty mapping dictionary for the domain name to be identified, which is updated according to a predetermined rule; S212, take all or part of the character groups in the domain name to be identified as the domain name to be mapped, and map all the characters to be mapped in the domain name to be mapped except for the period in the order from left to right or from right to left to obtain the target mapping characters of each character to be mapped according to the ordered list and mapping dictionary. In step S212, when mapping each character to be mapped, it is first determined whether there is a mapping relationship corresponding to the character to be mapped in the mapping dictionary; If so, according to the mapping relationship, the character to be mapped is mapped to the corresponding target mapped character; If not, determine which first occurrence of the character to be mapped is in the domain name to be mapped, and select the character of the corresponding order from the ordered list as the target mapping character of the character to be mapped, and map the character to be mapped to the target mapping character, and store the mapping relationship between the character to be mapped and the target mapping character in the mapping dictionary; S213, after mapping all the characters to be mapped in the domain name to be mapped to the corresponding target mapping characters, combine them with the original period in the domain name to be identified in their original order to obtain the domain name to be identified; S22, compare the domain name to be identified with the mapped domain names in the domain name identification database to determine whether the domain name to be identified matches a mapped domain name in the domain name identification database; if not, proceed to S23; if yes, proceed to S24; S23, add the domain name to be identified to the domain name database to be classified; S24, Determine whether the types of all domains corresponding to the mapped domains in the matching are consistent. If yes, proceed to S25; otherwise, proceed to S26. The type of a domain refers to whether a domain is a legal domain or an illegal domain. S25, add the domain name to be identified to the domain name identification database, and determine that the type of the domain name to be identified is the type of all domain names corresponding to the mapped domain name in the matching; S26, Determine whether the percentage of legitimate domains among all domains corresponding to the mapped domains in the matching process reaches the first threshold. If not, proceed to S27; if yes, proceed to S28. S27, Add the domain name to be identified to the domain name database to be classified; S28, add the domain name to be identified to the domain name identification database, determine that the type of the domain name to be identified is a legal domain name, and add a question mark to the domain name to be identified; S3, after all the domain names to be identified have been identified, confirm the proportion of suspicious domain names corresponding to each mapped domain name in the domain name identification database. The proportion of suspicious domain names corresponding to a mapped domain name is the proportion of domain names with suspicious tags among all the domain names corresponding to the mapped domain name. When the proportion of suspicious domains corresponding to a mapped domain exceeds the second threshold, the domains with suspicious tags among all the domains corresponding to the mapped domain are transferred from the domain identification database to the domain to be classified database, thus obtaining the updated domain to be classified database and domain identification database.

2. The domain name identification database update method according to claim 1, characterized in that: The ordered list includes letters and / or numbers and / or symbols arranged in sequence, with one and only one of each letter and / or number and / or symbol.

3. The domain name recognition database update method according to claim 1, characterized in that: In step S212, the top-level domain name is removed from the domain name to be identified and then used as the domain name to be mapped. In step S213, after mapping all the characters to be mapped in the domain name to be mapped to the corresponding target mapping characters, they are combined with the original top-level domain and period in the domain name to be identified in their original order to obtain the domain name to be identified.

4. The domain name identification database update method according to claim 1, characterized in that: Before mapping, it is first determined whether the number of character types of the domain name to be mapped exceeds the number of characters in the ordered list. If so, the domain name to be mapped is truncated so that the number of character types of the truncated domain name does not exceed the number of characters in the ordered list.

5. The domain name identification database update method according to any one of claims 1-4, characterized in that: In step S3, after obtaining the updated domain name database to be classified, the second identification method is used to determine the type of each domain name in the domain name database to be classified. Add the domain name whose type has been determined in the domain name classification database to the domain name identification database; Domains whose types are not identified in the domain classification database will be retained in the database and compared with the updated domain identification database in the next comparison.

6. A domain name identification database update system, characterized in that, include: The domain name acquisition unit is used to acquire a batch of domain names to be identified. The identification unit, used to identify each domain name to be identified, includes: The mapping module is used to determine the mapping domain name corresponding to a given domain name based on a defined ordered list and mapping method; it includes the following sub-modules: The dictionary creation submodule is used to create an initial empty mapping dictionary for the domain name to be identified, which is updated according to a predetermined rule; The mapping submodule is used to take all or part of the character groups in the domain name to be identified as the domain name to be mapped, and map all the characters to be mapped in the domain name to be mapped except for the period in order from left to right or from right to left to obtain the target mapping characters for each character to be mapped according to the ordered list and the mapping dictionary. When mapping each character to be mapped, it is first determined whether there is a mapping relationship corresponding to the character to be mapped in the mapping dictionary. If so, the character to be mapped is mapped to the corresponding target mapping character according to the mapping relationship. If not, it is determined which first occurrence of the character to be mapped is in the domain name to be mapped, and the character of the corresponding order is selected from the ordered list as the target mapping character of the character to be mapped and the character to be mapped is mapped to the target mapping character. The mapping relationship between the character to be mapped and the target mapping character is stored in the mapping dictionary. The domain name reorganization submodule is used to map all the characters to be mapped in the domain name to be mapped to the corresponding target mapping characters and then combine them with the original period in the domain name to be identified in the original order of their original positions in the domain name to be identified to obtain the domain name to be identified. The matching module is used to compare the domain name to be identified with the mapping domain names in the domain name identification database to determine whether the domain name to be identified matches a mapping domain name in the domain name identification database. If not, it communicates with the first classification module; if yes, it communicates with the second classification module. The first classification module is used to add the domain name to be identified to the domain name database to be classified; The second classification module is used to determine whether the types of all domains corresponding to the mapped domains in the matching are consistent. If yes, it communicates with the third classification module; otherwise, it communicates with the fourth classification module. The type of a domain refers to whether a domain is a legal domain or an illegal domain. The third classification module is used to add the domain name to be identified to the domain name identification database and determine the type of the domain name to be identified as the type of all domain names corresponding to the mapped domain name in the matching. The fourth classification module determines whether the proportion of legitimate domains among all domains corresponding to the mapped domains in the matching process reaches the first threshold. If not, it communicates with the fifth classification module; if so, it communicates with the sixth classification module. The fifth classification module is used to add the domain name to be identified to the domain name database. The sixth classification module is used to add the domain name to be identified to the domain name identification database, determine that the domain name to be identified is a legal domain name, and add a question mark to the domain name to be identified; The questionable domain name identification unit is used to confirm the proportion of questionable domain names corresponding to each mapped domain name in the domain name identification database after all domain names to be identified have been identified. The proportion of questionable domain names corresponding to a mapped domain name is the proportion of domain names with questionable tags among all domain names corresponding to the mapped domain name. When it is determined that the proportion of questionable domain names corresponding to a mapped domain name exceeds a second threshold, the domain names with questionable tags among all domain names corresponding to the mapped domain name are transferred from the domain name identification database to the domain name database to be classified, thus obtaining an updated domain name database to be classified and a domain name identification database.

7. A domain name identification method, characterized in that: The domain name identification database obtained by the domain name identification database update method as described in any one of claims 1-5 is used to identify the domain name accessed by the user.

8. A server, comprising a memory and a processor, the memory storing a program executable by the processor, characterized in that: When the program is executed, it implements the domain name identification database update method as described in any one of claims 1-5 and / or the domain name identification method as described in claim 7.

Images