A website digital certificate recognition method and system based on decision tree

Abstract

The application is a method and system for improving the detection speed of website digital certificate by accumulating characteristic variation probability through sample and regular expression, and constructing decision tree. The method and system for identifying website digital certificate comprises: detecting whether the website state is normal, and obtaining the website source code if normal; preprocessing the source code, and extracting the source code content related to digital certificate; analyzing the extracted characteristic according to the source code content of digital certificate, and constructing decision tree; combining the obtained decision tree with traditional verification method, and constructing digital certificate identification system. Through traditional method verification, because the number of websites is large, the concurrency is large, and the source code detection process is complex, there are the disadvantages of long time consumption and high CPU occupancy rate. Therefore, the application analyzes the characteristic variation probability through regular expression, obtains the decision tree, and constructs the identification system, so as to accurately determine the range of digital certificate verification result, greatly improve the detection speed, and effectively reduce the CPU occupancy rate.

Description

Technical Field

[0001] This invention relates to the field of website source code analysis and processing technology, and a processing method for extracting features based on regular expressions to build a decision tree model. Background Technology

[0002] With the rapid development of internet technology, people have built a thriving internet society using the network as a platform. In this unique society, digital certificates are paperless electronic licenses, created by legally established, qualified third-party organizations in accordance with relevant registration laws and regulations, and containing enterprise registration information.

[0003] The emergence of website digital certificates has made the Internet more secure and trustworthy. However, in order to urge online operators to implement the relevant provisions of the "E-commerce Law" and the "Administrative Measures for Online Transactions", strengthen the supervision of online transactions, and improve the credit level of e-commerce, relevant departments need to test the display of digital certificates.

[0004] The sheer number of websites and the diverse formats of digital certificates make detection increasingly difficult for relevant authorities. Therefore, this patent provides a website digital certificate identification method and system based on decision trees. By using a decision tree model to accurately determine the range of digital certificate verification results and optimize the detection process, this invention facilitates the detection of digital certificate display status by relevant authorities for online business entities. Summary of the Invention

[0005] The purpose of this invention is to provide a website digital certificate identification method and system based on decision trees, which optimizes the traditional digital certificate detection process, facilitates relevant departments in detecting the display status of digital certificates of network operators, and improves regulatory efficiency.

[0006] This invention discloses a website digital certificate recognition method based on decision trees, including...

[0007] 1. Collect the source code related to website digital certificates and perform preprocessing based on similarity;

[0008] 2. Perform feature extraction processing on the preprocessed digital certificate source code;

[0009] 3. Create corresponding regular expressions for the digital certificate source code after feature extraction;

[0010] 4. Using partially categorized and labeled digital certificate source code and detection results as training samples, regular expressions are used to perform feature detection on the preprocessed source code to build a feature detection result library;

[0011] 5. Calculate the feature hit probability based on the feature detection results, and construct several decision tree models with high accuracy;

[0012] 6. Test the detection accuracy of the decision tree model using a large amount of experimental data, and obtain the decision tree with the highest accuracy;

[0013] 7. Statistically analyze the characteristics of erroneous digital certificates detected by the decision tree model, and optimize the decision tree;

[0014] Prior to this, the similarity preprocessing of the digital certificate-related source code described in step 1 is characterized by classification preprocessing based on the source code format, including what type of digital certificate the source code is, whether the source code is a JS script that generates a digital certificate, or whether the certificate is added manually via an tag, etc.

[0015] Prior to step 2, feature extraction processing is performed on the preprocessed source code, including information such as domain names and parameters contained in the source code, to construct a source code feature library.

[0016] Prior to step 3, the establishment of a corresponding regular expression library for the digital certificate source code after feature processing is to establish a corresponding regular expression based on the features and source code constructed in step 2. This regular expression can be used to detect whether a certain feature exists in a certain source code.

[0017] Prior to step 4, the feature detection database is constructed by using regular expressions to detect the collected website source code, recording the website domain name, the features present, and the digital certificate results detected by the traditional detection process, and constructing a feature detection result database.

[0018] Prior to this, the decision tree construction method described in step 5 involves calculating the probability of whether a single or combined feature has an impact on the result based on the feature detection results obtained in step 4, ranking the priority of feature impact, and simulating the construction of several more accurate decision tree models.

[0019] Prioritize the use of a large amount of experimental data to test the detection results of the decision tree model in step 6, and compare them with the detection results of the traditional process to obtain the accuracy and error of the decision tree detection, and obtain the decision tree with the highest accuracy.

[0020] Prior to this, step 7 analyzes the characteristics of the error results detected in step 6, and optimizes the model constructed in step 5 or performs data filtering based on the results.

[0021] The present invention also discloses a website digital certificate recognition system based on decision tree, which consists of five parts: data storage module, source code crawling module, source code preprocessing module, decision tree prediction module, and result verification module.

[0022] The data storage submodule is used to store and retrieve website-related information and digital certificate detection results;

[0023] The source code crawling module is used to crawl the source code of the page according to the website URL and transmit the results to the source code preprocessing module;

[0024] The source code preprocessing module is used to extract digital certificate-related source code from the web page source code obtained by the source code crawling module, perform classification preprocessing according to the format, extract source code parameter features, and transmit the results to the decision tree prediction module.

[0025] The decision tree prediction module is used to predict the display of the website's digital certificate based on the results obtained from the source code preprocessing module, and to obtain the detection prediction result.

[0026] The result verification module is used to verify the results based on the source code preprocessing module and the decision tree prediction module, obtain the accurate result displayed by the website digital certificate, and transmit the result to the data storage module.

[0027] This invention combines a decision tree model with traditional verification methods, which can predict the results of website digital certificate display, reduce identification steps, and improve detection efficiency. Attached Figure Description

[0028] Figure 1 This is a flowchart of a website digital certificate recognition method based on decision trees according to the present invention;

[0029] Figure 2 This is a flowchart of the code related to the classification and processing of website digital certificates in this invention.

[0030] Figure 3 It is a process of detecting and statistically analyzing results based on one or more features.

[0031] Figure 4 This is a schematic diagram of the structure of a website digital certificate recognition system based on a decision tree according to the present invention.

[0032] Figure 5 This invention constructs a decision tree based on information gain as an example.

[0033] Figure 6 This invention constructs a decision tree based on information gain as an example. Detailed Implementation

[0034] To better illustrate the above-mentioned objectives, features, and advantages of the present invention and to make it more easily understood, the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0035] Implementation Examples

[0036] One embodiment of the present invention discloses a website digital certificate recognition method based on decision trees, which mainly includes the following steps (the following examples use website electronic identification and business licenses):

[0037] See attached document Figure 1 This invention relates to the construction of a website digital certificate recognition method and system based on decision trees. The overall flowchart may include the following steps:

[0038] Step 1: Collect the source code related to website digital certificates and classify them according to similarity. In Step 1, based on the website address regulated by the provincial online commodity transaction supervision, the website source code is crawled, the source code related to displaying electronic certificates and licenses is extracted, and the similarity is calculated using Levenshtein Distance for classification. Based on the existing electronic certificate and business license issuance status and the display status of enterprise certificates and licenses, they can be roughly divided into the following categories:

[0039] 1) A tag-based electronic business license, where showType indicates the type of electronic license used for display, serial is the electronic business license number minus the filing number, and signData is the signature value formed by signing the filing information on the electronic business license.

[0040] For example, the source code for the electronic business license labeling of a certain e-commerce website is as follows:

[0041]

[0042] / businessCheck / verifKey.do?showType=p&serial=3201009132010056286146a50a42001001-SAIC_SHOW_32000020151207140312284&signData=MEQCID9pnGVdjVWkL18LwwKD2iNZ / 2Cb2k4crEll9ozNZxmOAiBbuNKkz3ADohGHuH4gcds4LEI0PGsnWEA+Lqs0QtcDaw==">

[0043] xxxxxx (The middle source code contains icon styles, etc.)

[0044]

[0045] 2) a) Electronic identifier in label format, where siteId is the electronic identifier number.

[0046] For example, the source code for an electronic labeling system on a company's website is as follows:

[0047] ​<a href="http: / / Domain Name

[0048] / mbm / entweb / elec / certView.shtml?siteId=c6c3b59937244daab699e08ac8787f63"target="_blank">

[0049] xxxxxx(The middle source code is in the form of an icon, etc.)

[0050]

[0051] 3) Electronic identifier in the form of js, where siteId is the electronic identifier number, and this js can generate the code related to the electronic identifier

[0052] For example, the source code of the electronic identifier label of an official website of a business company is as follows:

[0053] <script id="jsgovicon"src="http: / / Domain Name

[0054] / mbm / app / main / electronic / js / govicon.js?siteId=3ffedfdda8c7434ebeb1e6baca1a0706&width=32&height=45&type=1"type="text / javascript"charset="utf-8">

[0055] Step 2: Perform feature extraction on the preprocessed digital certificate source code; this step is a further data preprocessing work based on Step 1, and features can be obtained according to the appropriate distance of the string (Levenshtein Distance).

[0056] The following features (partial selection) can be collected according to Step 1:

[0057] Serial Number feature Feature Description 1 url Website domains that issue electronic identifiers or electronic business licenses 2 showType Displaying the license using the electronic license type 3 serial Electronic Business License Number - Filing Number 4 signData The signature value generated by signing the filing information on the electronic business license 5 siteId Electronic Identification Number 6 businessCheck / verifKey The electronic business license with a label type has a relatively fixed code. 7 mbm / entweb / elec / certView A type of electronic identification has a relatively fixed code. 8 jsgovicon The JS-based electronic identifier has a relatively fixed encoding.

[0058] Table 1

[0059] Step 3: Establish a corresponding regular expression for the digital certificate source code after feature extraction; that is, establish a regular expression according to the features, and through this expression, it can be judged whether a certain feature exists in the source code of a certain website. For example:

[0060] 1) According to the feature businessCheck / verifKey.do in Step 2, a regular expression (java format) can be constructed:

[0061] <a\\s+(?:[^>]+? \\s*)? businessCheck / verifKey\\.do\\? [\\w\\d]+[^>]+? >

[0062] 2) Based on the features in step 2, mbm / entweb / elec / certView.shtml can be used to construct a regular expression (Java format):

[0063] <a\\s+(?:[^> ]+? \\s*)? mbm / entweb / elec / certView\\.shtml\\? [\\w\\d]+[^>]+? >

[0064] 3) Based on the feature jsgovicon from step 2, a regular expression (Java format) can be constructed:

[0065] <script\\s+(?:[^> ]+? \\s*)? id\\s*=\\s*['\"]?\\s*jsgovicon\\s*['\"]? [^>]+? >

[0066] Step 4: Using partially categorized and labeled digital certificate source code and detection results as training samples, regular expressions are used to perform feature detection on the preprocessed source code, and a feature detection result library is built. This step uses the regular expressions built in Step 3 to examine the website source code, and the following data is compiled (showing some experimental data and some features; for security reasons, all URLs are shown as codes):

[0067]

[0068] Table 2

[0069] Remark:

[0070] For the above experimental data, 0 indicates non-existence; 1 indicates existence.

[0071] The above verification results are as follows: a: No application submitted; b: No label affixed; c: Correct label / photograph displayed; d: Incorrect label / photograph displayed; e: Unauthorized label / photograph displayed.

[0072] Step 5: Calculate the feature hit probability based on the feature detection results, and construct several decision tree models with high accuracy.

[0073] First, based on the results obtained in step 4, the impact of the existence of one or more features on the verification results is statistically analyzed, as shown in the figure below (only partial experimental data is shown):

[0074] Feature showType probability a probability b probability c d probability probability 1 0.0264235 0.0294835 0.5465978 0.3216537 0.0758415 0 0.2128469 0.2395786 0.2967825 0.1648986 0.0858934

[0075] Table 3

[0076] Feature siteId probability a probability b probability c d probability probability 1 0 0 0.4569872 0.3562468 0.186766 0 0.1849567 0.2064895 0.2648275 0.1843562 0.1593701

[0077] Table 4

[0078]

[0079] Table 5

[0080] Remark:

[0081] For the above experimental data, 0 indicates non-existence; 1 indicates existence.

[0082] The above verification results are as follows: a: No application submitted; b: No label affixed; c: Correct label / photograph displayed; d: Incorrect label / photograph displayed; e: Unauthorized label / photograph displayed.

[0083] Analyzing the above examples reveals that, according to the feature detection probabilities in Table 3, while the presence of `showType` increases the probability of correct labeling, the probabilities of producing results a, b, and c are similar when it is absent. Therefore, the presence or absence of `showType` has little impact on the accuracy of the results. According to the feature detection probabilities in Table 4, when `siteId` exists, the probabilities of a and b are both 0, but when `siteId` is absent, the uncertainty of the detection results is greater. According to the feature detection probabilities in Table 5, when either `signData` or `jsgovicon` exists, the probabilities of a and b are both 0, but their impact on c, d, and e is relatively small.

[0084] Based on a comprehensive analysis of the features cited in the appeal, and according to information gain, the following structure can be constructed: Figure 5 Appendix Figure 6 The decision tree shown is an example. The experimental data shown in the figure is 0: not present; 1: present; the verification results are a: not applied for; b: not labeled; c: correctly labeled / photographed; d: incorrectly labeled / photographed; e: misused label / photographed.

[0085] Step 6: Test the detection accuracy of the decision tree model using a large amount of experimental data to obtain the decision tree with the highest accuracy. Since the gain rates of some features are similar, several better decision trees were constructed in Step 5. In order to obtain the optimal decision tree, a large amount of experimental data was used to detect each decision tree in order using regular expressions to obtain the optimal decision tree.

[0086] Step 7: Statistically analyze the characteristics of the erroneous digital certificates detected by the decision tree model and optimize the processing results; Based on the results obtained in Step 6, it can be found that it is difficult to distinguish between cases of no application and no labeling. Therefore, the electronic identification ID is obtained from the database as a new feature and added to the decision tree model constructed in Step 6 to improve the accuracy.

[0087] Step 8: Combine the decision tree model obtained in Step 7 with traditional recognition methods. Traditional methods require individual verification for each case, while the decision tree obtained in Step 7 accurately reflects the website's digital certificate display. Verification is performed directly based on the decision tree results, eliminating the need for additional verification steps and significantly improving efficiency.

[0088] An embodiment of the present invention also discloses a website digital certificate recognition system based on a decision tree, as shown in the appendix. Figure 4 As shown, it includes the following steps (this embodiment uses an e-commerce website as an example):

[0089] Step 1: Obtain the URL https: / / www.suning.com from the data storage module.

[0090] Step 2: Based on the URL obtained from the data storage module, as shown below, crawl the page source code and transmit the results to the source code preprocessing module.

[0091] <!DOCTYPE html>

[0092]

[0093]

[0094] <meta charset＝"utf-8">

[0095] <link rel＝"shortcut icon"href＝" www.suning.com favicon.ico"type＝"image x-icon">

[0096] <meta http-equiv＝"Content-Type"content＝"text html；charset＝utf-8">

[0097] <meta name＝"keywords"content＝"XXXX网上商城,苏宁电器,Suning,手机,电脑,冰箱,洗衣机,相机,数码,家居用品,鞋帽,化妆品,母婴用品,图书,食品,正品行货">

[0098] <meta name＝"description"content＝"XXXX-综合网上购物平台，商品涵盖家电、手机、电脑、超市、母婴、服装、百货、海外购等品类。送货更准时、价格更超值、上新货更快，正品行货、全国联保、可门店自提，全网更低价，让您放心去喜欢！">

[0099] <title> XXXX (Suning.com) - More punctual delivery, better prices, and faster new arrivals.< / title>

[0100] <meta name＝"apple-itunes-app"content＝"app-id＝537508092">

[0101] <meta http-equiv＝"X-UA-Compatible"content＝"IE＝edge">

[0102] <link rel＝"canonical"href＝"http: www.suning.com">

[0103] <meta property＝"wb:webmaster"content＝"3addc532fa0c656e">

[0104] <meta property＝"qc:admins"content＝"165746643563561676375">

[0105] <meta name＝"mobile-agent"content＝"format＝html5；url＝http: m.suning.com">

[0106] <meta name＝"viewport"content＝"width＝device-width,initial-scale＝1.0">

[0107] <meta name＝"baidu-site-verification"content＝"x0HfZwVU6x">

[0108] <meta content＝"true"name＝"autoclick">

[0109] <meta content＝"d488778a"name＝"siteid">

[0110] <meta content＝"homepage1"name＝"pageid">

[0111] <script type＝"text javascript">

[0112] 步骤3：源代码预处理模块对步骤2所获得的网页源代码提取数字证书相关源代码，如下所示。

[0113] <a

[0114] href＝"https: / / zzlz.gsxt.gov.cn / businessCheck / verifKey.do？showType＝p&serial＝913200005668848108-SAIC_SHOW_1000009132000056688481081572311357911&signData＝MEQCII+7OGallh6hxToamA3VfhexsTQw4fBcArcI7NexQaS0AiA8CqvkBVtdxtB / yiHMrNo9FWipRU161IH5hhB5BlBi2g＝＝"target＝"_blank"rel＝"nofollow"name＝"public0_none_wb_zs0305">

[0115] <img src＝" / / res.suning.cn / public / v3 / images / dianzizhizhao.png？v＝01"height＝"24"width＝"24"alt＝"电子营业执照">

[0116] 根据源代码格式进行分类预处理，提取源代码参数特征，例如：showType、serial等，并传输至决策树预判模块。

[0117] 步骤4：根据步骤3所得结果，通过决策树进行预判，概率较大的结果为"正确亮照”。将决策树预判结果以及步骤3获得的数字证书相关源代码传输至结果校验模块，优先校验"正确亮照”检测步骤，可得知改网站数字证书显示结果符合"正确亮照”相关标准，故而无需再检测其他结论所需流程。

[0118] 本系统将决策树模型与传统校验方法相结合，可预判网站数字证书显示情况结果，缩减识别步骤，提高检测效率。< / script>

Claims

1. A website digital certificate recognition method based on decision trees, characterized in that, The method includes: 1) Collect the source code related to the website's digital certificate, use string edit distance, and perform preprocessing based on similarity; We collected source code related to digital certificates from a large number of websites, categorized and preprocessed the source code according to its format, kept interfering source code, and removed invalid source code. 2) Perform feature extraction processing on the preprocessed digital certificate source code; Based on the preprocessed source code, common features of the source code are extracted using string edit distance; 3) Create a corresponding regular expression for the digital certificate source code after feature extraction; 4) Using partially categorized and labeled digital certificate source code and detection results as training samples, regular expressions are used to perform feature detection on the preprocessed source code to build a feature detection result library; 5) Calculate the feature hit probability based on the feature detection results, and construct several decision tree models with an accuracy greater than the threshold; 6) Test the detection accuracy of the decision tree model using a large amount of experimental data to obtain the decision tree with the highest accuracy; 7) Statistical analysis of the characteristics of erroneous digital certificates detected by the decision tree model, and optimization of the decision tree.

2. The website digital certificate recognition method based on decision tree according to claim 1, characterized in that, include: Based on feature extraction from the source code, design matching rules and construct corresponding regular expressions; The collected digital certificate source code was inspected using the constructed regular expressions and compared with materials including registration information of online business entities to build a feature detection result library.

3. The website digital certificate recognition method based on decision tree according to claim 1, characterized in that, include: For feature detection results, calculate the probability that individual and combined features may affect the results. Based on the magnitude of the influence probability, construct several decision trees with an accuracy greater than a set value. Collect a large amount of website source code, preprocess it with regular expressions, and then use the decision tree model for detection. Compare the detection results with the results obtained by the original detection method to obtain the accuracy, thereby confirming the decision tree with the highest accuracy.

4. The website digital certificate recognition method based on decision tree according to claim 1, characterized in that, include: In step 7, based on the model test results, the characteristics of the detected error source code are analyzed, the decision tree is optimized, and the detection accuracy is improved.

5. A decision tree-based website digital certificate recognition system, used to implement the decision tree-based website digital certificate recognition method as described in claim 1, characterized in that, It consists of five parts: data storage module, source code crawling module, source code preprocessing module, decision tree prediction module, and result verification module. The data storage module is used to store website-related information and digital certificate detection results; The source code crawling module is used to crawl the source code of the page according to the website URL and transmit the results to the source code preprocessing module; The source code preprocessing module is used to extract digital certificate-related source code from the web page source code obtained by the source code crawling module, perform classification preprocessing according to the format, extract source code parameter features, and transmit the results to the decision tree prediction module. The decision tree prediction module is used to predict the display of the website's digital certificate based on the results obtained from the source code preprocessing module, and to obtain the detection prediction result. The result verification module is used to verify the results based on the source code preprocessing module and the decision tree prediction module, obtain the accurate result displayed by the website digital certificate, and transmit the result to the data storage module.

Images