Method and system for analyzing and mining Internet specific behavior object based on big data modeling

Abstract

The invention discloses a method and system for analyzing and mining a specific behavior object of the Internet based on big data modeling, and the method comprises the steps: accessing Internet data in real time, and verifying a domain name and related actions of the accessed data based on a pre-constructed domain name keyword knowledge base by using a streaming processing technology to determine whether the domain name and related actions accord with a preset rule; the method comprises the following steps of: storing data which is verified to accord with a preset rule into an offline database, extracting data in a preset time range from the offline database, selecting key attributes to carry out deep analysis and calculation, carrying out grouping statistics on information of related attributes, and combining a preset integral calculation rule, comprehensively evaluating and pushing Internet access broadband data with different risk level identifiers; deducing active network virtual account data associated with the data association logic through the data association logic; and based on a preset multi-dimensional label rule, analyzing and marking various key information in the offline streaming data, and pushing risk active Internet access broadband and active network virtual account data.

Description

Technical Field

[0001] The present invention relates to the technical field of offline data modeling and analysis, and in particular to a method and system for mining Internet specific behavior objects based on big data modeling and analysis. Background Art

[0002] With the rapid development of Internet technology, people can generate, obtain and transmit data in various ways. In the current big data information age, a huge amount of data is generated every day, such as financial, news, chat, video and other types of website and online activity data. Similarly, some people are also spreading and guiding some bad behaviors through secret network behaviors. Existing data collection technologies include server logs, browser cookies, web crawlers and other means. These technologies can track and record user behavior in the network environment, such as browsing history, click-through rate, purchase behavior, etc. The collected data is usually stored in a large database or data warehouse, such as Hadoop or NoSQL database, for further processing and analysis. Traditional data processing methods involve preprocessing steps such as data cleaning and data conversion, aiming to convert raw data into structured data for analysis. Analysis methods include statistical analysis, cluster analysis, classification analysis, etc., which are used to extract useful information from data. Currently popular technologies include using R language, pandas library in Python for data processing, and using SAS and SPSS for statistical analysis.

[0003] The quality of Internet data varies greatly, and contains erroneous, redundant or incomplete data, which seriously affects the accuracy and reliability of analysis results. The existing technology is not very automated and intelligent in data preprocessing, and still requires a lot of manual intervention, which is inefficient. The diversity and complexity of Internet behavioral data make it difficult to capture deep patterns and associations using traditional methods. Although machine learning provides some solutions, the effectiveness and interpretability of existing algorithms when processing high-dimensional, heterogeneous data are still challenged. Summary of the invention

[0004] In order to solve the above technical problems existing in the prior art, the present invention proposes a method and system for mining specific Internet behavior objects based on big data modeling and analysis to solve the above technical problems.

[0005] According to a first aspect of the present invention, a method for mining Internet specific behavior objects based on big data modeling analysis is proposed, comprising:

[0006] S1: Access Internet data in real time and use streaming processing technology to verify the domain names and related actions of the accessed data based on a pre-built domain name keyword knowledge base to determine whether they comply with the predetermined rules;

[0007] S2: Store the data that has been verified to meet the predetermined rules in the offline database, extract the data within the predetermined time range from the offline database, select key attributes for in-depth analysis and calculation, group and count the information of related attributes, and combine the preset integral calculation rules to comprehensively evaluate and push out the Internet broadband data with different risk level identifications;

[0008] S3: Based on the Internet broadband data, the active network virtual account data associated with it is derived through data association logic;

[0009] S5: Based on the preset multi-dimensional label rules, various key information in the offline streaming data is analyzed and marked, and the risky active Internet broadband and active network virtual account data are pushed out.

[0010] In some specific embodiments, S1 performs a hash comparison between the domain names and actions in the streaming data and the collected domain name knowledge base.

[0011] In some specific embodiments, the integral calculation in S2 specifically includes:

[0012] Match the domain names visited by the broadband account and the actions taken when visiting the domain names with the scores corresponding to the score calculation rules to obtain the basic score, and calculate the monthly score through the logarithmic function. The formula is: Monthly score = log (number of visits to domain name 1, 1.2) * basic score of domain name action 1 + log (number of visits to domain name 2, 1.2) * basic score of domain name action 2 + ... + log (number of visits to domain name N, 1.2) * basic score of domain name action N;

[0013] The monthly score of the current month and the monthly score of history are calculated comprehensively, and different score proportions are taken into account according to the distance between the months. The calculation formula is: Comprehensive score = 0.7 monthly score of the current month + 0.2 monthly score of the previous month + 0.1*monthly score of the month before that.

[0014] In some specific embodiments, the active network virtual account data in S3 is specifically: the active network virtual account is launched based on the preset activity determination days requirement through the associated network virtual account according to the Internet broadband data.

[0015] In some specific embodiments, the analysis and marking of various key information in the offline streaming data in S5 specifically includes:

[0016] Based on the preset multi-dimensional label rules, action labels are assigned according to the accessed domain name and action;

[0017] Assign active labels to time periods based on the time periods of Internet access;

[0018] Assign penetration tool tags based on the use of special tools to access the Internet;

[0019] Assign an aggregated broadband label to multiple virtual accounts of the same type of network associated with broadband.

[0020] In some specific embodiments, the various types of key information in S5 include information related to APP installation, website access information, mailbox usage information, file content characteristics, and network actions.

[0021] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, the above method is implemented.

[0022] According to a third aspect of the present invention, a system for mining Internet specific behavior objects based on big data modeling analysis is proposed, comprising:

[0023] A data access and verification unit is configured to access Internet data in real time, and verify the domain name and related actions of the accessed data based on a pre-built domain name keyword knowledge base using a streaming processing technology to determine whether it complies with a predetermined rule;

[0024] The broadband data modeling and analysis unit is configured to store the data verified to meet the predetermined rules in an offline database, extract the data within a predetermined time range from the offline database, select key attributes for in-depth analysis and calculation, and comprehensively evaluate and push out the Internet broadband data with different risk level identifications by grouping and counting the information of related attributes and combining with the preset integral calculation rules;

[0025] A virtual account association derivation unit configured to derive active network virtual account data associated therewith based on the Internet broadband data through data association logic;

[0026] The tag analysis and screening unit is configured to analyze and mark various key information in the offline streaming data based on preset multi-dimensional tag rules, and push out risky active Internet broadband and active network virtual account data.

[0027] In some specific embodiments, the data access and verification unit performs a hash comparison between the domain names and actions in the streaming data and the collected domain name knowledge base.

[0028] In some specific embodiments, the integral calculation in the broadband data modeling and analysis unit specifically includes:

[0029] Match the domain names visited by the broadband account and the actions taken when visiting the domain names with the scores corresponding to the score calculation rules to obtain the basic score, and calculate the monthly score through the logarithmic function. The formula is: Monthly score = log (number of visits to domain name 1, 1.2) * basic score of domain name action 1 + log (number of visits to domain name 2, 1.2) * basic score of domain name action 2 + ... + log (number of visits to domain name N, 1.2) * basic score of domain name action N;

[0030] The monthly score of the current month and the monthly score of history are calculated comprehensively, and different score proportions are taken into account according to the distance between the months. The calculation formula is: Comprehensive score = 0.7 monthly score of the current month + 0.2 monthly score of the previous month + 0.1*monthly score of the month before that.

[0031] In some specific embodiments, the active network virtual account data in the virtual account association derivation unit is specifically: the active network virtual account is derived based on the preset activity determination days requirement through the associated network virtual account according to the Internet broadband data.

[0032] In some specific embodiments, the various key information in the tag analysis and screening unit includes information related to the installation of APP, website access information, mailbox usage information, file content characteristics, and network actions; analyzing and marking various key information in offline streaming data specifically includes:

[0033] Based on the preset multi-dimensional label rules, action labels are assigned according to the accessed domain name and action;

[0034] Assign active labels to time periods based on the time periods of Internet access;

[0035] Assign penetration tool tags based on the use of special tools to access the Internet;

[0036] Assign an aggregated broadband label to multiple virtual accounts of the same type of network associated with broadband.

[0037] The present invention proposes a method and system for mining Internet specific behavior objects based on big data modeling analysis, which uses an offline database as a container for filtering analysis and calculation, and collects domain name knowledge bases, label rules, and integral rules as calculation and analysis conditions. The calculation container attributes include distributed cluster servers and data storage cycles. The calculation and analysis conditions include writing qualified streaming data to an offline database based on rules, and constructing hierarchical rules for data result data based on integral calculation and business labels, so as to realize the ability of dynamically analyzing and mining specific behavior objects according to customized rules in the face of big data and massive data. It can meet the needs of analyzing and mining specific behavior data that meet the conditions in massive data, and can rely on the corresponding business rules to analyze and calculate active broadband accounts and network virtual accounts to provide corresponding warning objects. In various industries, through the analysis and mining of massive data, it is possible to quickly grasp the situation and make various warnings and judgments in advance, which can be of great significance in various fields such as network detection, event development, and risk prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and together with the description are used to explain the principles of the present invention. Other embodiments and many expected advantages of the embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0039] Figure 1 This is a flow chart of mining Internet specific behavior objects based on big data modeling and analysis in one embodiment of the present application;

[0040] Figure 2 It is a flowchart of a specific embodiment of the present application for mining Internet specific behavior objects based on big data modeling and analysis;

[0041] Figure 3 It is a schematic diagram of a specific embodiment of a matching rule between domain names and action attributes in streaming data and a collected domain name knowledge base in a specific embodiment of the present application;

[0042] Figure 4 is a flowchart of offline data analysis calculation of a specific embodiment of the present application;

[0043] Figure 5 is a flowchart of launching active virtual accounts based on calculation in a specific embodiment of the present application;

[0044] Figure 6This is a flowchart of a specific embodiment of the present application for marking data based on preset rules;

[0045] Figure 7 It is a flowchart of launching Internet broadband accounts and active virtual accounts based on a method of mining Internet specific behavior objects based on big data modeling analysis in a specific embodiment of the application;

[0046] Figure 8 This is a system architecture diagram for mining Internet specific behavior objects based on big data modeling and analysis in one embodiment of the present application;

[0047] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0048] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0049] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] Figure 1 FIG. 1 shows a flowchart of mining Internet specific behavior objects based on big data modeling and analysis according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0051] S101: Access Internet data in real time, and use streaming processing technology to verify the domain name and related actions of the accessed data based on a pre-built domain name keyword knowledge base to determine whether it complies with predetermined rules.

[0052] In a specific embodiment, network data is transmitted in real time in a streaming program. When data arrives, the program quickly extracts the domain name and related action information in the data, and performs a hash comparison with the pre-collected and constructed domain name keyword knowledge base. For example, if the knowledge base contains domain name keywords of a specific risk category, when the domain name in the streaming data matches it, it is preliminarily determined that the data may meet the subsequent analysis rules, otherwise it does not meet the rules. The non-compliant data will be screened out, and only the compliant data will enter the next step.

[0053] S102: Store the data verified to meet the predetermined rules in the offline database, extract the data within the predetermined time range from the offline database, select key attributes for in-depth analysis and calculation, group and count the information of related attributes, and combine with the preset point calculation rules to comprehensively evaluate and push out the Internet broadband data with different risk level identifications.

[0054] In a specific embodiment, data that has been verified to comply with the rules is continuously written to the offline database. The offline database has a specific data storage cycle and can store a large amount of historical data for analysis. Taking monthly analysis as an example, at the beginning of each month, data within the range of the previous month is extracted from the offline database, and key attributes such as time (accurate to days, hours, etc.) and network behavior (type of websites visited, network services used, etc.) are selected as key attributes. Through group statistics, for example, the number of times each broadband account visits a specific type of website by day is counted, and information such as the number of days of visit is calculated. Then, combined with the points calculation rules, the domain names visited by the broadband account and the actions taken when accessing the domain names are matched with the corresponding scores according to the points calculation rules to obtain the basic score, and the monthly score is calculated through the logarithmic function (the formula is: monthly score = log (domain name 1 visits, 1.2) domain name action 1 basic score + log (domain name 2 visits, 1.2) domain name action 2 basic score + ... + log (domain name N visits, 1.2) domain name action N basic score), and then the monthly score of the current month and the monthly score of the history are comprehensively calculated (comprehensive score = 0.7 current month's monthly score + 0.2 last month's monthly score + 0.1 the previous month's monthly score), and the Internet broadband data with different risk levels such as high, medium and low are evaluated and pushed out according to the final score.

[0055] S103: Based on the Internet broadband data, the active network virtual account data associated therewith is derived through data association logic.

[0056] In this step, based on the Internet broadband data obtained in the previous step, the pre-set data association logic is used, such as the login and usage time association between the broadband account and the network virtual account, to find all network virtual accounts associated with the broadband account. Then, according to the preset active determination days requirement, such as more than 15 days in a month are considered active, the active network virtual account data is filtered out from these associated accounts.

[0057] S104: Based on the preset multi-dimensional label rules, various key information in the offline streaming data is analyzed and labeled, and risky active Internet broadband and active network virtual account data are pushed out.

[0058] In a specific embodiment, various types of key information include information related to the installation of APP (such as the name and source of the APP), website access information (domain name and URL of the website visited), mailbox usage information (addresses for sending and receiving emails, keywords in the subject of emails, etc.), file content features (such as file type, specific keywords contained in the file, etc.), and network actions (upload, download, browse, and other action types). Based on the preset multi-dimensional labeling rules, for example, if the domain name visited belongs to a specific risk category website, a corresponding risk action label is assigned; an active label is assigned to the time period according to the Internet access time period (such as frequent Internet access late at night); if the use of special tools (such as proxy servers, etc.) to access the Internet is detected, a penetration tool label is assigned; and an aggregated broadband label is assigned to multiple network virtual accounts of the same type (such as multiple game accounts) associated with the broadband. Through such analysis and labeling, the risky active Internet broadband and active network virtual account data are eventually pushed out for more accurate screening and classification management.

[0059] Figure 2 It is a flowchart of a specific embodiment of the present application for mining Internet specific behavior objects based on big data modeling analysis, such as Figure 2 As shown, the specific steps include:

[0060] S1: Internet data is filtered and matched against specific domain names based on streaming processing. In this step, the streaming processing program continuously monitors Internet data. Once the data flows in, the domain name part is immediately parsed and quickly compared with the rules in the collected domain name knowledge base. For example, domain names containing keywords related to illegal or high-risk behaviors, such as certain gambling and illegal file sharing domain names, are accurately identified and filtered and matched. Only data whose domain names meet the knowledge base rules will enter the next processing flow, and those that do not meet the rules will be directly excluded, ensuring that the data for subsequent analysis has a certain degree of pertinence and potential risk relevance.

[0061] S2: Write the current qualified data into the offline database. After filtering and matching in step S1, the qualified data is written into the offline database in order. The offline database adopts a distributed cluster server architecture, has efficient data storage and management capabilities, and can cope with the storage needs of massive data. Data storage has a certain cycle, during which data can be extracted at any time for subsequent analysis and calculation, providing a stable data source foundation for long-term data analysis and behavior object mining.

[0062] S3: Based on the modeling analysis of the offline database, the number of times, days, and network behaviors are calculated by grouping some of the attributes (time, network behavior), and the Internet broadband data is pushed out. Extract data from the offline database according to a predetermined time range, and group them with time (such as daily, weekly, monthly and other time granularities) and network behavior (such as visiting specific types of websites, performing specific network operations, etc.) as key attributes. For example, the behaviors of the same broadband account visiting different websites in one day are grouped together, and the number of visits, number of days visited, and other information are counted. Then, combined with the points calculation rules, the integral value of the Internet broadband data is obtained according to the above-mentioned integral calculation process, and the risk level is divided according to the integral value, and the Internet broadband data containing the risk level identifier is pushed out for subsequent analysis of the associated network virtual account situation.

[0063] S4: Based on the warning broadband data of S3, the active network virtual account data under broadband is introduced. According to the warning broadband data with risk level identification obtained in step S3, the network virtual account information associated with it is deeply mined. By analyzing the relationship between the broadband account and the network virtual account, such as login time and usage frequency, the network virtual accounts that are frequently used within a certain time range (such as within a month) are screened out and determined as active network virtual account data. For example, if a network virtual account is logged in and used for more than 20 days in a month through a specific warning broadband, it will be identified as an active network virtual account, providing a basis for subsequent precise monitoring and management.

[0064] S5: Based on preset rules, analyze offline streaming data for different tags such as APP installation domain name, website, mailbox, file content, action, etc., and push out risky active Internet broadband and active network virtual accounts that are easy to filter and classify. For the rich information in offline streaming data, such as domain name information of APP installation (to determine whether APP is downloaded from illegal channels), type of visited website (whether it is a high-risk website), mailbox usage (whether there is abnormal email sending and receiving), file content characteristics (whether it contains sensitive information) and network actions (whether there is abnormal upload and download behavior), etc., comprehensive analysis and marking are performed according to preset label rules. For example, if a specific type of APP is detected to be downloaded from an unauthorized domain name, the broadband account and related virtual accounts are marked with corresponding risk tags. Through such label marking, risky active Internet broadband and active network virtual accounts can be easily filtered and classified, and precise positioning and management of specific Internet behavior objects can be achieved.

[0065] Figure 3 FIG. 1 is a schematic diagram of a specific embodiment of a matching rule between a domain name and an action attribute in streaming data and a collected domain name knowledge base in a specific embodiment of the present application, such as Figure 3 As shown, the specific process is as follows:

[0066] 301. Streaming data: The process starts with receiving streaming data. These streaming data may be real-time data from the network, such as network traffic data.

[0067] 302. Calculate the HASH value of the domain name action: After receiving the streaming data, the next step is to calculate the HASH value of the domain name action. The purpose of this step is to convert the domain name and related actions into a fixed-length hash value through the HASH algorithm to facilitate subsequent fast comparison operations. For example, for each domain name and corresponding operation (such as access, download, etc.) in the data, the corresponding HASH value is calculated.

[0068] 303. Compare with the domain name rules in the collection knowledge base: After calculating the HASH values, compare these HASH values ​​with the domain name rules in the collection knowledge base. The collection knowledge base may contain pre-set domain name rules, such as a legal domain name list, an illegal domain name list, etc. By comparing the HASH values, it is possible to quickly determine whether the domain names and actions in the current streaming data comply with the rules in the knowledge base.

[0069] 304, comparison: This step is a decision node, which determines the comparison result of the previous step. If the comparison result is "yes", that is, the domain name and action in the data meet the rules in the knowledge base, the data flows to 305, the offline library; if the comparison result is "no", that is, the data does not meet the rules, the data flows to 306, clearing. The offline library can be used for subsequent data analysis, mining and other operations, such as further analyzing the behavior patterns and risk levels of the data stored in the offline library. This process effectively screens out data that meets the requirements and stores them in the offline library by performing HASH value calculation and rule comparison on streaming data, while clearing data that does not meet the rules, ensuring the accuracy and efficiency of data processing.

[0070] Figure 4 is a flowchart of offline data analysis calculation of a specific embodiment of the present application, such as Figure 4 As shown, the specific process is as follows:

[0071] 401, Offline streaming data: The process starts with obtaining offline streaming data. This data may be network data that has been preliminarily screened or stored and is already offline.

[0072] 402. Domain name comparison: First, perform a domain name comparison operation on the offline streaming data. This step may involve comparing the domain names in the data with specific domain name rules or whitelists or blacklists to determine the legitimacy or relevance of the domain names.

[0073] 403. Grouping action: After the domain name comparison, the data is processed by grouping action. The data is grouped according to the action type (such as access, download, upload, etc.) in the data, so that different types of actions can be processed and analyzed separately later.

[0074] 404. Calculate days and times: For the grouped action data, calculate the days and times that each action occurs. For example, counting the days and total times that a specific action (such as visiting a certain type of website) occurs within a certain time range helps quantify the user's network behavior.

[0075] 405. Scoring: Based on the data such as the number of days and times calculated in the previous step, the data is scored according to the preset scoring rules. The scoring rules may comprehensively consider factors such as the frequency and duration of the action to assess the risk level or importance of the network behavior.

[0076] 406. Release of broadband data: Based on the scoring results, broadband data may be released. Such broadband data may include processed and evaluated network behavior information, which can be used for further network analysis, monitoring or management, such as identifying high-risk broadband users or network behavior patterns.

[0077] The process performs a series of processing operations on offline streaming data, including domain name comparison, action grouping, behavior quantification and scoring, and ultimately obtains broadband data with analytical value, which helps analyze and manage network behavior.

[0078] Figure 5 is a flowchart of launching active virtual accounts based on calculation in a specific embodiment of the present application, such as Figure 5 As shown, the specific process is as follows:

[0079] 501. Broadband account data: The process starts with obtaining broadband account data. This data may contain relevant information of broadband users, such as account identification, usage records, etc.

[0080] 502. Cross collision: Perform cross collision operation on broadband account data. This step may involve comparing, matching and associating broadband account data with other related data, and finding other information associated with the broadband account through specific algorithms or rules.

[0081] 503. Network virtual accounts: After the cross-collision operation, network virtual accounts are derived. These network virtual accounts may be virtual accounts used by users on the Internet associated with broadband accounts, such as online game accounts, social media accounts, etc.

[0082] This process can effectively mine related network virtual accounts from broadband account data by performing cross-collision processing on broadband account data, which is helpful for network behavior analysis, user portrait construction, and network security monitoring.

[0083] Figure 6 It is a flowchart of a specific embodiment of the present application for marking data based on preset rules, such as Figure 6 As shown, the specific process is as follows:

[0084] Offline streaming data: The process starts with obtaining offline streaming data. These offline streaming data may be previously processed or stored network data, which contains various network behavior information of users.

[0085] Label calculation: Offline streaming data enters the label calculation stage. Through specific algorithms and rules, the data is analyzed and calculated to determine different types of labels.

[0086] Tag results: After the tag calculation, multiple tag results are obtained, including: Action tags: may reflect the user's specific operation behavior, such as visiting a specific website, performing a specific network operation, etc. Behavior tags: may cover a wider range of user behavior patterns, such as frequent nighttime Internet access, peak network usage during a specific time period, etc. Penetration tools: used to identify whether the user has used tools such as proxy servers, VPNs, etc. that may be used to break through network protection. Communication characteristics: may reflect the connection and interaction characteristics of the user with other network entities (such as other users, network services, etc.).

[0087] This process can extract rich user behavior labels from the data by performing label calculations on offline streaming data, which is helpful for network behavior analysis, network security monitoring, and user profile construction.

[0088] Figure 7 It is a flowchart of a method for launching Internet broadband accounts and active virtual accounts based on big data modeling and analysis of a specific embodiment of the application, such as Figure 7 As shown, the specific process is as follows:

[0089] 701. Broadband account: The process starts with broadband account data, which may include broadband user account information, usage records and other related data.

[0090] 702. Network virtual account: further obtain network virtual account data from the broadband account data. This data may be a virtual account used by a user on the Internet associated with the broadband account, such as an online game account, a social media account, etc.

[0091] 703. Points Assessment: Perform points assessment on broadband account and network virtual account data. This step may involve calculation and evaluation based on the user's network behavior (such as access frequency, access content, etc.) according to preset points rules to determine the account's risk level or activity level and other indicators.

[0092] 704. Label data: Label the data according to the score analysis results of the previous step and other relevant rules. For example, high-risk accounts are labeled with specific risk labels, and active accounts are labeled with activity labels, etc., to facilitate subsequent screening and analysis.

[0093] 705. Data screening and classification: Screening and classification operations are performed based on tagged data. By classifying data with different tags, account data with similar characteristics or risk levels are grouped together for further analysis and management.

[0094] 706. Broadband and network virtual account result data: After the above series of processing steps, broadband and network virtual account result data are finally obtained. These result data can be used for network management, risk monitoring, user behavior analysis, etc. For example, a comprehensive analysis report of user network behavior is provided to network operators to help them formulate more accurate service strategies and security measures.

[0095] This process can effectively extract valuable result data from the original data by performing operations such as scoring, labeling, data screening and classification on broadband account and network virtual account data, which helps in the refined management and analysis of network behavior.

[0096] Figure 8 A system architecture diagram of mining Internet specific behavior objects based on big data modeling analysis according to an embodiment of the present application is shown as follows: Figure 8As shown, the system includes a data access and verification unit 801, a broadband data modeling and analysis unit 802, a virtual account association derivation unit 803, and a label analysis and screening unit 804. The data access and verification unit 801 is configured to access Internet data in real time, and use streaming processing technology to verify the domain name and related actions of the accessed data based on a pre-built domain name keyword knowledge base to determine whether it meets the predetermined rules; the broadband data modeling and analysis unit 802 is configured to store the data that has been verified to meet the predetermined rules in an offline database, extract data within a predetermined time range from the offline database, select key attributes for in-depth analysis and calculation, and comprehensively evaluate and push out Internet broadband data with different risk level identifiers by grouping and counting the information of related attributes, and combining with the preset integral calculation rules; the virtual account association derivation unit 803 is configured to derive the active network virtual account data associated with the Internet broadband data through data association logic; the label analysis and screening unit 804 is configured to analyze and mark various key information in the offline streaming data based on the preset multi-dimensional label rules, and push out risk active Internet broadband and active network virtual account data.

[0097] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application. Fig. 9 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0098] like Fig. 9 As shown, the computer system includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0099] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0100] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

[0101] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The modules involved in the embodiments of the present application may be implemented by software or by hardware.

[0104] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: accesses Internet data in real time, uses streaming processing technology, and verifies the domain name and related actions of the accessed data based on a pre-built domain name keyword knowledge base to determine whether it complies with the predetermined rules; stores the data verified to comply with the predetermined rules in an offline database, extracts data within a predetermined time range from the offline database, selects key attributes for in-depth analysis and calculation, and comprehensively evaluates and pushes Internet broadband data with different risk level identifiers by grouping and counting the information of related attributes, combined with the preset integral calculation rules; derives the associated active network virtual account data through data association logic; analyzes and marks various key information in the offline streaming data based on the preset multi-dimensional label rules, and pushes out risk active Internet broadband and active network virtual account data.

[0105] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for mining Internet specific behavior objects based on big data modeling and analysis, characterized in that: include: S1: Access Internet data in real time and use streaming processing technology to verify the domain names and related actions of the accessed data based on a pre-built domain name keyword knowledge base to determine whether they comply with the predetermined rules; S2: storing the data verified to meet the predetermined rules in an offline database, extracting the data within a predetermined time range from the offline database, selecting key attributes for in-depth analysis and calculation, statistically analyzing the information of relevant attributes by grouping, and combining with the preset integral calculation rules, comprehensively evaluating and pushing out the Internet broadband data with different risk level identifications; S3: Based on the Internet broadband data, derive the active network virtual account data associated therewith through data association logic; S5: Based on the preset multi-dimensional label rules, various key information in the offline streaming data is analyzed and marked, and the risky active Internet broadband and active network virtual account data are pushed out.

2. The method for mining Internet specific behavior objects based on big data modeling and analysis according to claim 1 is characterized in that: In S1, the domain names and actions in the streaming data are hashed and compared with the collected domain name knowledge base.

3. The method for mining Internet specific behavior objects based on big data modeling and analysis according to claim 1 is characterized in that: The integral calculation in S2 specifically includes: Match the domain names visited by the broadband account and the actions taken when visiting the domain names with the scores corresponding to the score calculation rules to obtain the basic score, and calculate the monthly score through the logarithmic function. The formula is: Monthly score = log (number of visits to domain name 1, 1.2) * basic score of domain name action 1 + log (number of visits to domain name 2, 1.2) * basic score of domain name action 2 + ... + log (number of visits to domain name N, 1.2) * basic score of domain name action N; The monthly score of the current month and the monthly score of history are calculated comprehensively, and different score proportions are taken into account according to the distance between the months. The calculation formula is: Comprehensive score = 0.7 monthly score of the current month + 0.2 monthly score of the previous month + 0.1*monthly score of the month before that.

4. The method for mining Internet specific behavior objects based on big data modeling and analysis according to claim 1 is characterized in that: The active network virtual account data in S3 is specifically: the active network virtual account is launched based on the preset activity determination days requirement through the associated network virtual account according to the Internet broadband data.

5. The method for mining Internet specific behavior objects based on big data modeling and analysis according to claim 1 is characterized in that: The analysis and marking of various key information in the offline streaming data in S5 specifically includes: Based on the preset multi-dimensional label rules, action labels are assigned according to the accessed domain name and action; Assign active labels to time periods based on the time periods of Internet access; Assign penetration tool tags based on the use of special tools to access the Internet; Assign an aggregated broadband label to multiple virtual accounts of the same type of network associated with broadband.

6. The method for mining Internet specific behavior objects based on big data modeling and analysis according to claim 1 is characterized in that: The various types of key information in S5 include information related to APP installation, website access information, email usage information, file content characteristics, and network actions.

7. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that: When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 6 is implemented.

8. A system for mining Internet specific behavior objects based on big data modeling and analysis, characterized in that: include: A data access and verification unit is configured to access Internet data in real time, and verify the domain name and related actions of the accessed data based on a pre-built domain name keyword knowledge base using a streaming processing technology to determine whether it complies with a predetermined rule; The broadband data modeling and analysis unit is configured to store the data verified to meet the predetermined rules in an offline database, extract the data within a predetermined time range from the offline database, select key attributes for in-depth analysis and calculation, and comprehensively evaluate and push out the Internet broadband data with different risk level identifiers by grouping and counting the information of related attributes and combining with the preset integral calculation rules; A virtual account association derivation unit configured to derive active network virtual account data associated therewith based on the Internet broadband data through data association logic; The tag analysis and screening unit is configured to analyze and mark various key information in the offline streaming data based on preset multi-dimensional tag rules, and push out risky active Internet broadband and active network virtual account data.

9. The system for mining Internet specific behavior objects based on big data modeling and analysis according to claim 8 is characterized in that: The data access and verification unit performs a hash comparison between the domain name and action in the streaming data and the collected domain name knowledge base.

10. The system for mining Internet specific behavior objects based on big data modeling and analysis according to claim 8 is characterized in that: The integral calculation in the broadband data modeling and analysis unit specifically includes: Match the domain names visited by the broadband account and the actions taken when visiting the domain names with the scores corresponding to the score calculation rules to obtain the basic score, and calculate the monthly score through the logarithmic function. The formula is: Monthly score = log (number of visits to domain name 1, 1.2) * basic score of domain name action 1 + log (number of visits to domain name 2, 1.2) * basic score of domain name action 2 + ... + log (number of visits to domain name N, 1.2) * basic score of domain name action N; The monthly score of the current month and the monthly score of history are calculated comprehensively, and different score proportions are taken into account according to the distance between the months. The calculation formula is: Comprehensive score = 0.7 monthly score of the current month + 0.2 monthly score of the previous month + 0.1*monthly score of the month before that.

11. The system for mining Internet specific behavior objects based on big data modeling and analysis according to claim 8 is characterized in that: The active network virtual account data in the virtual account association derivation unit is specifically: the active network virtual account is derived based on the preset activity determination days requirement through the associated network virtual account according to the Internet broadband data.

12. The system for mining Internet specific behavior objects based on big data modeling and analysis according to claim 8 is characterized in that: The various key information in the tag analysis and screening unit includes information related to the installation of APP, website access information, mailbox usage information, file content characteristics, and network actions; The analysis and marking of various key information in offline streaming data include: Based on the preset multi-dimensional label rules, action labels are assigned according to the accessed domain name and action; Assign active labels to time periods based on the time periods of Internet access; Assign penetration tool tags based on the use of special tools to access the Internet; Assign an aggregated broadband label to multiple virtual accounts of the same type of network associated with broadband.

Images