Mobile phone number capturing method and system
By processing HTTP data extraction feature combination and calling effective feature library matching, the problem of difficulty in obtaining mobile phone numbers in the existing technology is solved, and efficient and accurate mobile phone number crawling is achieved.
Patent Information
- Application Number
- CN202411935847.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-25
AI Technical Summary
In the prior art, it is difficult to obtain a mobile phone number by analyzing the email username information, and it is difficult to effectively obtain the required second mobile phone number.
By obtaining HTTP data, processing and extracting feature combinations, calling the valid feature library for matching, and extracting mobile phone numbers that meet the conditions. This method combines feature recognition based on cookies, URLs, and Referer contents of HTTP data, avoiding dependence on mailbox data.
It improves the crawling range and crawling ability of mobile phone numbers, reduces development costs and difficulty, improves development efficiency and mobile phone number crawling accuracy, and can capture multiple mobile phone numbers at one time.
Smart Images
Figure CN119884455A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of Internet technology, and in particular to a method and system for capturing mobile phone numbers. Background Art
[0002] Deep packet inspection technology has developed to a very mature level. Telecommunications operators have collected and accumulated massive amounts of traffic data through the DPI platform. Extracting mobile phone numbers from this traffic data is very important for telecommunications operators.
[0003] The invention patent with authorization announcement number CN113127767B discloses a usable method for extracting mobile phone numbers.
[0004] This method requires parsing the email user name information to obtain the second mobile phone number required for filtering, but the email information is often difficult to obtain. Summary of the invention
[0005] The purpose of the present invention is to solve the above problems existing in the prior art and to provide a method and system for capturing mobile phone numbers.
[0006] The purpose of the present invention is achieved through the following technical solutions:
[0007] The method for capturing a mobile phone number includes the following steps:
[0008] S1, obtain HTTP data of the mobile phone number to be captured;
[0009] S2, processing the HTTP data of the mobile phone number to be captured to obtain a batch of feature combinations to be extracted, wherein the feature combinations to be extracted include a key field value to be identified and its corresponding host field value to be identified, a parameter type field value to be identified, and a value field value to be identified;
[0010] S3, calling a valid feature library, the valid feature library includes a set of valid feature combinations, each valid feature combination includes a key field value and its corresponding host field value and a parameter type field value; the parameter type field includes Cookie, URL, Referer, each valid feature combination in the valid feature library is a feature combination to be verified after being verified by verification parameters and meeting the requirements; a set of verification parameters includes mobile phone number segment matching rate, mobile phone number terminal distribution rate, mobile phone number family distribution rate, mobile phone number efficiency, single number device coverage and single number area coverage;
[0011] S4, matching each of the feature combinations to be extracted with the valid feature combinations in the valid feature library. If a feature combination to be extracted matches a valid feature combination in the valid feature library, extracting the value field value to be identified of the feature combination to be extracted as the mobile phone number corresponding to the feature combination to be extracted.
[0012] Preferably, the mobile phone number segment matching rate is the ratio of the number of value field values that meet the requirements and are successfully matched with the mobile phone number segment library among all value field values that meet the requirements extracted under a feature combination to be verified to the total number of value field values that meet the requirements extracted under the feature combination to be verified, and the value field values that meet the requirements include at least 11 consecutive digits.
[0013] Preferably, the mobile phone number terminal distribution rate is the ratio of the number of all value field values that meet the requirements extracted from a feature combination to be verified to the number of terminal devices on which the feature combination to be verified appears.
[0014] Preferably, the mobile phone number household distribution rate is the ratio of the number of all value field values that meet the requirements extracted from a feature combination to be verified to the number of home gateways where the feature combination to be verified appears.
[0015] Preferably, the mobile phone number effectiveness rate is the ratio of the number of qualified value field values that meet the mobile phone number regularity to the total number of qualified value field values extracted under a feature combination to be verified.
[0016] Preferably, the single-number device coverage number is the maximum value of each value field value that meets the requirements extracted under a feature combination to be verified and appears in the terminal device within a period of time; the single-number area coverage number is the maximum value of each value field value that meets the requirements extracted under a feature combination to be verified and appears in the area within the period of time.
[0017] Preferably, when it is determined that the mobile phone number segment matching rate of the value field value that meets the requirements corresponding to a feature combination to be verified is not less than 98%, the mobile phone number efficiency is not less than 98%, the mobile phone number terminal distribution rate is between 0.7-1.3, the mobile phone number household distribution rate is between 0.7-1.3, the single-number device coverage number is less than 10, and the single-number area coverage number is less than 5, the feature combination to be verified is determined to be a valid feature combination; otherwise, the feature combination to be verified is added to the invalid feature library.
[0018] Preferably, whether each valid feature combination in the valid feature library is valid is verified regularly or irregularly through new HTTP data. If a valid feature combination is verified to be valid, the valid feature combination is retained in the valid feature library. Otherwise, the valid feature combination is transferred to the invalid feature library as an invalid feature combination.
[0019] And / or, regularly or irregularly verify whether each invalid feature combination in the invalid feature library is valid through new HTTP data; if an invalid feature combination is verified to be valid, the valid feature combination is transferred to the valid feature library as a valid feature combination; otherwise, the invalid feature combination is retained in the invalid feature library.
[0020] Preferably, in S4, after extracting the mobile phone number corresponding to the feature combination to be extracted, the mobile phone number corresponding to the feature combination to be extracted is compared with the mobile phone number segment library. If it is determined that the mobile phone number corresponding to the feature combination to be extracted matches a mobile phone number in the mobile phone number segment library, the mobile phone number corresponding to the feature combination to be extracted is output; if it is determined that the mobile phone number corresponding to the feature combination to be extracted does not match the mobile phone number in the mobile phone number segment library, the mobile phone number corresponding to the feature combination to be extracted is not output.
[0021] Mobile phone number capture system, including:
[0022] A data acquisition unit, used to acquire HTTP data of the mobile phone number to be captured;
[0023] A feature extraction unit, used for processing the HTTP data of the mobile phone number to be captured to obtain a batch of feature combinations to be extracted, wherein the feature combinations to be extracted include a key field value to be identified and its corresponding host field value to be identified, a parameter type field value to be identified, and a value field value to be identified;
[0024] A feature library calling unit is used to call a valid feature library, wherein the valid feature library includes a set of valid feature combinations, each of which includes a key field value and its corresponding host field value and a parameter type field value; the parameter type field includes Cookie, URL, Referer, and each of the valid feature combinations in the valid feature library is a feature combination to be verified and obtained after the verification parameter verification meets the requirements; each of the valid feature combinations in the valid feature library is a feature combination to be verified and obtained after the verification parameter verification meets the requirements; a set of verification parameters includes a mobile phone number segment matching rate, a mobile phone number terminal distribution rate, a mobile phone number household distribution rate, a mobile phone number efficiency rate, a single number device coverage number, and a single number area coverage number;
[0025] An extraction unit is used to match each of the feature combinations to be extracted with a valid feature combination in the valid feature library. If a feature combination to be extracted matches a valid feature combination in the valid feature library, the value corresponding to the key field value to be identified of the feature combination to be extracted is extracted as the mobile phone number corresponding to the feature combination to be extracted.
[0026] The advantages of the technical solution of the present invention are mainly reflected in:
[0027] The present invention identifies feature combinations based on the three aspects of Cookie, URL and Referer of HTTP data, and can obtain an effective feature library with more effective feature combinations, which is beneficial to improving the capture range and capture ability of mobile phone numbers. At the same time, there is no need to identify email data, which is beneficial to reducing development costs and development difficulty, and improving development efficiency. Moreover, when screening feature combinations, a set of verification parameters are used for selection, which can effectively improve the accuracy of the effective feature library, thereby helping to improve the capture accuracy of mobile phone numbers, and can capture multiple mobile phone numbers at one time from the HTTP data of the mobile phone numbers to be captured, which is beneficial to improving the capture rate.
[0028] After the mobile phone number is extracted, it is further matched with the mobile phone number segment library, which can effectively avoid misrecognition caused by human error in entering the mobile phone number and improve accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a schematic diagram of the process of determining the valid feature library and the invalid feature library of the present invention;
[0030] Figure 2 It is a flow chart of the method for capturing mobile phone numbers of the present invention;
[0031] Figure 3 It is a flow chart of the process of re-matching the mobile phone number segment library in the mobile phone number capture method of the present invention. DETAILED DESCRIPTION
[0032] The purpose, advantages and features of the present invention will be illustrated and explained by the non-limiting description of the following preferred embodiments. These embodiments are only typical examples of the application of the technical solution of the present invention, and any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection claimed by the present invention.
[0033] In the description of the scheme, it should be noted that the terms "center", "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience and simplification of description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.
[0034] Example 1
[0035] The following is an explanation of the method for capturing mobile phone numbers disclosed in the present invention in conjunction with the accompanying drawings. The method for capturing mobile phone numbers is based on the collected historical HTTP data to determine an effective feature library as a basis for subsequent mobile phone number capture. The effective feature library is determined according to the following process:
[0036] As attached Figure 1 As shown, historical HTTP data for a period of time is obtained, and the period of time can be selected as needed, preferably 1 week. The historical HTTP data can be collected and analyzed by a known DPI (deep packet inspection) system, which will not be described here.
[0037] The historical HTTP data acquired over a period of time is formatted through software programming to obtain a feature combination relationship table, which includes at least a host (host name) field, a parameter type field, a key field, and a value field, etc. The parameter type field includes Cookie, URL, Referer, and the value field contains at least 11 consecutive digits. Table 1 provides an example of partial data in a feature combination relationship table.
[0038] Table 1:
[0039]
[0040] Each row in the feature combination relationship table is defined as a record. When each record is extracted from the historical HTTP data, the value field value containing at least 11 consecutive digits is extracted through regular matching. Each extracted value field value containing at least 11 consecutive digits is defined as a value field value that meets the requirements. All value field values that meet the requirements and their corresponding key field values, host field values, and parameter type field values are added to the feature combination relationship table.
[0041] The key field value, host field value, and parameter type field value of each record are defined as a feature combination to be verified. A value field value that meets the requirements corresponds to a feature combination to be verified, and a feature combination to be verified can correspond to multiple different value field values that meet the requirements.
[0042] Subsequently, the effective feature combination for capturing the mobile phone number can be determined based on the value field value that meets the requirements corresponding to each feature combination to be verified. When determining the effective feature combination for capturing the mobile phone number, it is specifically determined whether the value field value that meets the requirements corresponding to each feature combination to be verified meets a set of verification parameters, and the verification parameters include the mobile phone number segment matching rate, mobile phone number terminal distribution rate, mobile phone number household distribution rate, mobile phone number effectiveness rate, single-number device coverage number and single-number area coverage number.
[0043] (1) The mobile phone number segment matching rate is the ratio of the number of value field values that meet the requirements and are successfully matched with the mobile phone number segment library among all value field values that meet the requirements extracted under a feature combination to be verified to the total number of value field values that meet the requirements extracted under the feature combination to be verified.
[0044] When all the value field values that meet the requirements extracted under a feature combination to be verified are matched with the mobile phone number segment library, the first 7 digits (counting from left to right) of each value field value that meets the requirements are matched with the first 7 digits of the mobile phone number in the mobile phone number segment library. The mobile phone number segment library can be obtained in advance and stored in the system for call. The first 7 digits of each mobile phone number reflect the operator and province and city information of a mobile phone number, which can effectively reflect the possibility that a value field value that meets the requirements is a mobile phone number. When the first 7 digits of the value field value that meets the requirements are the same as the first 7 digits of a mobile phone number in the mobile phone number segment library, it is considered that the extracted value field value that meets the requirements is matched successfully.
[0045] The higher the mobile phone number segment matching rate, the higher the possibility that the value field value corresponding to the feature combination is a mobile phone number. Conversely, the lower the mobile phone number segment matching rate, the lower the possibility that the value field value corresponding to the feature combination is a mobile phone number. For example, the number of value field values that meet the requirements extracted under a feature combination to be verified is 1000, but the number of value field values that meet the requirements and are successfully matched with the mobile phone number segment library is 10, then the mobile phone number segment matching rate is 0.01, which means that the possibility that the value field value that meets the requirements extracted under this feature combination is a mobile phone number is very low. Therefore, such a feature combination to be verified cannot be used for mobile phone number capture.
[0046] (2) The mobile phone number terminal distribution rate reflects how many terminal devices all the value field values that meet the requirements extracted from a feature combination to be verified appear on within the period of time. It is the ratio of the number of value field values that meet the requirements extracted from a feature combination to be verified to the number of terminal devices on which the feature combination to be verified appears.
[0047] Since a mobile phone number usually does not appear in multiple terminal devices, the higher the terminal distribution rate of the mobile phone number, the smaller the possibility that the value field value that meets the requirements corresponding to the feature combination to be verified is a mobile phone number. Conversely, the lower the terminal distribution rate of the mobile phone number, the higher the possibility that the value field value that meets the requirements corresponding to the feature combination to be verified is a mobile phone number.
[0048] (3) The mobile phone number household distribution rate reflects how many households all the value field values that meet the requirements extracted from a feature combination to be verified appear in the period of time. It is the ratio of the number of all the value field values that meet the requirements extracted from a feature combination to be verified to the number of home gateways where the feature combination to be verified appears.
[0049] Usually, a mobile phone number will not appear in multiple households within a period of time. Therefore, the higher the household distribution rate of the mobile phone number, the less likely it is that the value field value that meets the requirements of the feature combination to be verified is a mobile phone number. Conversely, the lower the household distribution rate of the mobile phone number, the more likely it is that the value field value that meets the requirements of the feature combination to be verified is a mobile phone number.
[0050] (4) The mobile phone number efficiency is the ratio of the number of value field values that meet the requirements and conform to the mobile phone number regularity to the total number of value field values that meet the requirements extracted under a feature combination to be verified.
[0051] The higher the mobile phone number efficiency, the higher the possibility that the value field value that meets the requirements and corresponds to the feature combination to be verified is a mobile phone number; conversely, the lower the mobile phone number efficiency, the lower the possibility that the value field value that meets the requirements and corresponds to the feature combination to be verified is a mobile phone number.
[0052] (5) The single-number device coverage number is the maximum value of each value field value that meets the requirements and is extracted from a feature combination to be verified, and appears in the terminal devices within the period of time. The number of terminal devices in which each value field value that meets the requirements appears in the period of time is specifically obtained by determining the number of terminal MAC addresses in which each value field value that meets the requirements appears.
[0053] The larger the number of single-number device coverage, the lower the possibility that the value field value that meets the requirements of the feature combination to be verified is a mobile phone number. Conversely, the smaller the number of single-number device coverage, the higher the possibility that the value field value that meets the requirements of the feature combination to be verified is a mobile phone number. Taking into account the impact of one-to-many scenarios such as random MAC addresses and router hanging (connecting one router to another router or switch), when the single-number device coverage is less than 10, the possibility that the value field value that meets the requirements of the feature combination to be verified is a mobile phone number is higher.
[0054] (6) The single number area coverage number reflects the maximum value of each value field value that meets the requirements extracted by a feature combination to be verified and appears in the area within the period of time. The area where each value field value that meets the requirements appears can be determined according to the following process:
[0055] First, determine from which gateway device the record to which the value field value that meets the requirements belongs obtains the data;
[0056] Then, the area where the gateway device is located is determined according to the IP information of the gateway device as the area where the value field value that meets the requirements appears.
[0057] The area is generally a province and / or city. The larger the single-number area coverage number, the lower the possibility that the value field value that meets the requirements of the feature combination to be verified is a mobile phone number. Conversely, the smaller the single-number area coverage number, the higher the possibility that the value field value that meets the requirements of the feature combination to be verified is a mobile phone number. Taking into account the impact of one-to-many scenarios such as random MAC and router hanging, when the single-number area coverage number is less than 5, the possibility that the value field value that meets the requirements of the feature combination to be verified is a mobile phone number is higher.
[0058] By determining whether the value field value that meets the requirements corresponding to each feature combination to be verified meets the above verification parameters, it is determined whether the feature combination to be verified can be used to capture the mobile phone number. Specifically, if the mobile phone number segment matching rate of the value field value that meets the requirements corresponding to a feature combination to be verified is not less than 98%, the mobile phone number efficiency is not less than 98%, the mobile phone number terminal distribution rate is between 0.7-1.3, the mobile phone number family distribution rate is between 0.7-1.3, the number of single-number device coverage is less than 10, and the number of single-number area coverage is less than 5, then the possibility that the vlue parameter value corresponding to the feature combination to be verified is a mobile phone number is high, so the feature combination to be verified is stored in the valid feature library as a valid feature combination, and at the same time, the extraction time of each valid feature combination is recorded in the valid feature library; on the contrary, when at least one of the verification parameters corresponding to a feature combination to be verified does not meet the above requirements, the possibility that the value field value that meets the requirements extracted by its corresponding value parameter value is a mobile phone number is low, so the feature combination to be verified is stored in the invalid feature library as an invalid feature combination, and the extraction time of the invalid feature combination is recorded at the same time. Even better, the mobile phone number segment matching rate is not less than 99%, and the mobile phone number efficiency is not less than 99%.
[0059] After obtaining the valid feature library, the valid feature library can be used to capture the mobile phone number of the HTTP data of the mobile phone number to be captured, as shown in the attached Figure 2 As shown, it includes the following steps:
[0060] S1, obtaining HTTP data of the mobile phone number to be captured; the HTTP data of the mobile phone number to be captured may be, for example, HTTP data of one hour or HTTP data of one day, which can be selected according to the needs.
[0061] S2, processing the HTTP data of the mobile phone number to be captured to obtain a set of feature combinations to be extracted, each of which includes a key field value to be identified and its corresponding host field value to be identified, a parameter type field value to be identified, and a value field value to be identified. The method for obtaining the feature combination to be extracted can refer to the process of extracting each record when building a valid feature library, which will not be repeated here.
[0062] S3, calling the effective feature library;
[0063] S4, matching each of the feature combinations to be extracted with the valid feature combinations in the valid feature library. If a feature combination to be extracted matches a valid feature combination in the valid feature library, extracting the value field value to be identified of the feature combination to be extracted as the mobile phone number corresponding to the feature combination to be extracted.
[0064] After determining that all feature combinations to be extracted are matched with the valid feature library, the mobile phone number corresponding to the extracted feature combination to be extracted is output. Of course, after extracting the mobile phone number corresponding to the feature combination to be extracted, the mobile phone number corresponding to the extracted feature combination to be extracted can be matched to determine whether there are repeated mobile phone numbers. If there are, when outputting the mobile phone number, only the number of times it appears is output.
[0065] Further, as attached Figure 3 As shown, in S4, after extracting the mobile phone number corresponding to the feature combination to be extracted, the mobile phone number corresponding to the feature combination to be extracted is compared with the mobile phone number segment library. If it is determined that the mobile phone number corresponding to the feature combination to be extracted matches a mobile phone number in the mobile phone number segment library, the mobile phone number corresponding to the feature combination to be extracted is output; if it is determined that the mobile phone number corresponding to the feature combination to be extracted does not match the mobile phone number in the mobile phone number segment library, the mobile phone number corresponding to the feature combination to be extracted is not output. The reason for this operation is that there may be a situation where the user enters an incorrect mobile phone number, which can effectively ensure the accuracy of the output mobile phone number.
[0066] Furthermore, during use, in order to ensure the reliability of the valid feature library, it is necessary to iteratively update the valid feature library and the invalid feature library. Specifically, the latest acquired HTTP data is used to verify whether each valid feature combination in the valid feature library is still valid and whether each invalid feature combination in the invalid feature library is still invalid. For example, the newly collected HTTP data every day, week or month can be used for verification.
[0067] When newly acquired HTTP data is used to verify a valid feature combination in a valid feature library, a value field value that meets the requirements and corresponds to the valid feature combination is extracted from the latest acquired HTTP data, and the newly extracted value field value that meets the requirements is combined with the previously extracted value field value that meets the requirements to re-determine whether it meets the above-mentioned verification parameters, thereby determining whether a valid feature combination still meets the requirements. If the newly extracted and previously extracted value field values that meet the requirements corresponding to a valid feature combination still meet the requirements after re-verification with the verification parameters, it is determined that the valid feature combination is still valid and the valid feature combination is retained in the valid feature library. Conversely, if the newly extracted and previously extracted value field values that meet the requirements corresponding to a valid feature combination do not meet the requirements after re-verification with the verification parameters, it is determined that the valid feature combination is invalid and the valid feature combination is transferred to the invalid feature library as an invalid feature combination.
[0068] When newly acquired HTTP data is used to check an invalid feature combination in the invalid feature library, a value field value that meets the requirements and corresponds to the invalid feature combination is extracted from the latest acquired HTTP data, and the newly extracted value field value that meets the requirements is combined with the previously extracted value field value that meets the requirements to re-determine whether it meets the above-mentioned verification parameters, thereby determining whether an invalid feature combination still does not meet the requirements. If the newly extracted and previously extracted value field values that meet the requirements corresponding to an invalid feature combination still do not meet the requirements after re-verification with the verification parameters, it is determined that the invalid feature combination is still invalid and the feature combination is retained in the invalid feature library; conversely, if the newly extracted and previously extracted value field values that meet the requirements corresponding to an invalid feature combination meet the requirements after re-verification with the verification parameters, it is determined that the invalid feature combination is valid and the invalid feature combination is transferred to the valid feature library as a valid feature combination.
[0069] In order to distinguish valid feature combinations from invalid feature combinations, a valid state label is configured for each valid feature combination, and an invalid state label is configured for each invalid feature combination. For example, the valid state label can be set to 1, and the invalid state label can be set to 0. In addition, the verification time of each feature combination is recorded during each verification.
[0070] Example 2
[0071] This embodiment discloses a mobile phone number capture system, including:
[0072] A data acquisition unit, used to acquire HTTP data of the mobile phone number to be captured;
[0073] A feature extraction unit, used for processing the HTTP data of the mobile phone number to be captured to obtain a batch of feature combinations to be extracted, wherein the feature combinations to be extracted include a key field value to be identified and its corresponding host field value to be identified, a parameter type field value to be identified, and a value field value to be identified;
[0074] A feature library calling unit is used to call a valid feature library, wherein the valid feature library includes a set of valid feature combinations, each of which includes a key field value and its corresponding host field value and a parameter type field value; the parameter type field includes Cookie, URL, Referer, and each of the valid feature combinations in the valid feature library is a feature combination to be verified and obtained after the verification parameter verification meets the requirements; each of the valid feature combinations in the valid feature library is a feature combination to be verified and obtained after the verification parameter verification meets the requirements; a set of verification parameters includes a mobile phone number segment matching rate, a mobile phone number terminal distribution rate, a mobile phone number household distribution rate, a mobile phone number efficiency rate, a single number device coverage number, and a single number area coverage number;
[0075] An extraction unit is used to match each of the feature combinations to be extracted with a valid feature combination in the valid feature library. If a feature combination to be extracted matches a valid feature combination in the valid feature library, the value corresponding to the key field value to be identified of the feature combination to be extracted is extracted as the mobile phone number corresponding to the feature combination to be extracted.
[0076] There are many implementation methods of the present invention, and all technical solutions formed by equivalent transformation or equivalent transformation fall within the protection scope of the present invention.
Claims
1. A method for capturing mobile phone numbers, characterized in that , including the following steps: S1, obtain HTTP data of the mobile phone number to be captured; S2, processing the HTTP data of the mobile phone number to be captured to obtain a batch of feature combinations to be extracted, wherein the feature combinations to be extracted include a key field value to be identified and its corresponding host field value to be identified, a parameter type field value to be identified, and a value field value to be identified; S3, calling a valid feature library, the valid feature library includes a set of valid feature combinations, each valid feature combination includes a key field value and its corresponding host field value and a parameter type field value; the parameter type field includes Cookie, URL, Referer, each valid feature combination in the valid feature library is a feature combination to be verified after being verified by verification parameters and meeting the requirements; a set of verification parameters includes mobile phone number segment matching rate, mobile phone number terminal distribution rate, mobile phone number family distribution rate, mobile phone number efficiency, single number device coverage and single number area coverage; S4, matching each of the feature combinations to be extracted with the valid feature combinations in the valid feature library. If a feature combination to be extracted matches a valid feature combination in the valid feature library, extracting the value field value to be identified of the feature combination to be extracted as the mobile phone number corresponding to the feature combination to be extracted.
2. The method for capturing mobile phone numbers according to claim 1, characterized in that: The mobile phone number segment matching rate is the ratio of the number of value field values that successfully match the mobile phone number segment library among all value field values that meet the requirements extracted under a feature combination to be verified to the total number of value field values that meet the requirements extracted under the feature combination to be verified, and the value field values that meet the requirements include at least 11 consecutive digits.
3. The method for capturing mobile phone numbers according to claim 2, characterized in that: The mobile phone number terminal distribution rate is the ratio of the number of all value field values that meet the requirements extracted from a feature combination to be verified to the number of terminal devices where the feature combination to be verified appears.
4. The method for capturing mobile phone numbers according to claim 2, characterized in that: The mobile phone number household distribution rate is the ratio of the number of all value field values that meet the requirements extracted from a feature combination to be verified to the number of home gateways where the feature combination to be verified appears.
5. The method for capturing mobile phone numbers according to claim 2, characterized in that: The mobile phone number efficiency is the ratio of the number of qualified value field values that meet the mobile phone number regularity to the total number of qualified value field values extracted under a feature combination to be verified.
6. The method for capturing mobile phone numbers according to claim 2, characterized in that: The single-number device coverage number is the maximum value of each value field value that meets the requirements and appears in the terminal devices within a period of time extracted under a feature combination to be verified; The single-number area coverage number is the maximum value of the area in which each value field value that meets the requirements and is extracted from a feature combination to be verified appears within the period of time.
7. The method for capturing mobile phone numbers according to claim 2, characterized in that: When it is determined that the mobile phone number segment matching rate of the value field value that meets the requirements corresponding to a feature combination to be verified is not less than 98%, the mobile phone number efficiency is not less than 98%, the mobile phone number terminal distribution rate is between 0.7-1.3, the mobile phone number household distribution rate is between 0.7-1.3, the single-number device coverage number is less than 10, and the single-number area coverage number is less than 5, the feature combination to be verified is determined to be a valid feature combination; otherwise, the feature combination to be verified is added to the invalid feature library.
8. The method for capturing mobile phone numbers according to claim 1, characterized in that: Verify whether each valid feature combination in the valid feature library is valid through new HTTP data at regular or irregular intervals. If a valid feature combination is verified to be valid, the valid feature combination is retained in the valid feature library. Otherwise, the valid feature combination is transferred to the invalid feature library as an invalid feature combination. And / or, regularly or irregularly verify whether each invalid feature combination in the invalid feature library is valid through new HTTP data; if an invalid feature combination is verified to be valid, the valid feature combination is transferred to the valid feature library as a valid feature combination; otherwise, the invalid feature combination is retained in the invalid feature library.
9. The method for capturing mobile phone numbers according to claims 1-8, characterized in that: In S4, after extracting the mobile phone number corresponding to the feature combination to be extracted, the mobile phone number corresponding to the feature combination to be extracted is compared with the mobile phone number segment library. If it is determined that the mobile phone number corresponding to the feature combination to be extracted matches a mobile phone number in the mobile phone number segment library, the mobile phone number corresponding to the feature combination to be extracted is output; if it is determined that the mobile phone number corresponding to the feature combination to be extracted does not match the mobile phone number in the mobile phone number segment library, the mobile phone number corresponding to the feature combination to be extracted is not output.
10. Mobile phone number capture system, characterized by ,include: A data acquisition unit, used to acquire HTTP data of the mobile phone number to be captured; A feature extraction unit, used for processing the HTTP data of the mobile phone number to be captured to obtain a batch of feature combinations to be extracted, wherein the feature combinations to be extracted include a key field value to be identified and its corresponding host field value to be identified, a parameter type field value to be identified, and a value field value to be identified; A feature library calling unit is used to call a valid feature library, wherein the valid feature library includes a set of valid feature combinations, each of which includes a key field value and its corresponding host field value and a parameter type field value; the parameter type field includes Cookie, URL, Referer, and each of the valid feature combinations in the valid feature library is a feature combination to be verified and obtained after the verification parameter verification meets the requirements; each of the valid feature combinations in the valid feature library is a feature combination to be verified and obtained after the verification parameter verification meets the requirements; a set of verification parameters includes a mobile phone number segment matching rate, a mobile phone number terminal distribution rate, a mobile phone number household distribution rate, a mobile phone number efficiency rate, a single number device coverage number, and a single number area coverage number; An extraction unit is used to match each of the feature combinations to be extracted with a valid feature combination in the valid feature library. If a feature combination to be extracted matches a valid feature combination in the valid feature library, the value corresponding to the key field value to be identified of the feature combination to be extracted is extracted as the mobile phone number corresponding to the feature combination to be extracted.
Citation Information
Patent Citations
Mobile phone number extraction methods, devices, electronic equipment and storage media
CN113127767B