Information mining method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a method and device for information mining, electronic equipment and storage medium, relates to the technical field of data processing, and particularly relates to the technical field of big data. The specific implementation scheme is as follows: obtaining user basic information and behavior related information of each user from log information; generating a graph database based on the user basic information and the behavior related information, the graph database comprising a plurality of nodes and connection relationships between the nodes, the plurality of nodes comprising a to-be-reviewed node and an abnormal node; dividing the nodes included in the graph database into a plurality of communities based on a community discovery algorithm; and for each to-be-reviewed node in the graph database, if the closeness of the to-be-reviewed node to abnormal nodes in the community to which the to-be-reviewed node belongs is greater than a preset threshold, determining that the to-be-reviewed node is an abnormal node. In this way, potential abnormal users can be identified.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more particularly to the field of big data technology. Background Technology

[0002] In the process of promoting an application (App), in order to encourage users to bring in new users and thus drive traffic to the App, users can receive corresponding rewards after bringing in new users. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, and storage medium for information mining.

[0004] In a first aspect, embodiments of this disclosure provide an information mining method, including:

[0005] Obtain basic user information and behavior-related information for each user from the log information;

[0006] A graph database is generated based on the user's basic information and the behavior-related information. The graph database includes multiple nodes and the connection relationships between the nodes. The multiple nodes include nodes to be reviewed and abnormal nodes.

[0007] Based on the community detection algorithm, the nodes included in the graph database are divided into multiple communities;

[0008] For each node to be reviewed in the graph database, if the density between the node to be reviewed and the abnormal node in its community is greater than a preset threshold, then the node to be reviewed is determined to be an abnormal node.

[0009] Secondly, embodiments of this disclosure provide an information mining apparatus, comprising:

[0010] The acquisition module is used to obtain basic user information and behavior-related information for each user from log information;

[0011] The generation module is used to generate a graph database based on the user's basic information and the behavior-related information. The graph database includes multiple nodes and the connection relationships between the nodes. The multiple nodes include nodes to be reviewed and abnormal nodes.

[0012] The partitioning module is used to divide the nodes included in the graph database into multiple communities based on a community detection algorithm.

[0013] The determination module is used to determine that each node to be reviewed in the graph database is an abnormal node if the closeness between the node to be reviewed and the abnormal node in its community is greater than a preset threshold.

[0014] Thirdly, this disclosure provides an electronic device, including:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0018] Fourthly, this disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in the first aspect.

[0019] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 This is a flowchart of the first information mining method provided in this embodiment of the disclosure;

[0023] Figure 2 This is an exemplary schematic diagram of node data details provided in an embodiment of this disclosure;

[0024] Figure 3 This is an exemplary schematic diagram of a node connection relationship provided in an embodiment of this disclosure;

[0025] Figure 4 This is a flowchart of the second information mining method provided in this disclosure embodiment;

[0026] Figure 5 This is a flowchart of the third information mining method provided in this embodiment of the disclosure;

[0027] Figure 6 This is a schematic diagram of the structure of an information mining device provided in an embodiment of this disclosure;

[0028] Figure 7 This is a block diagram of an electronic device used to implement the information mining method of the embodiments of this disclosure. Detailed Implementation

[0029] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0030] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0031] First, the application scenario of this application embodiment will be introduced. In various apps, in order to encourage registered users to bring in new users to use the app, rewards can be given to registered users who successfully bring in new users. For ease of description, in this application embodiment, registered users who successfully bring in new users are referred to as "new user referrers".

[0032] To prevent users from fraudulently acquiring new users to obtain rewards, reward distribution conditions can be set. Taking a cloud storage app as an example, the reward distribution condition could be that the number of videos uploaded and played by the users referred by the new user reaches a certain threshold. However, fraudulent user acquisition methods are becoming increasingly sophisticated and have reached a large scale. Abnormal users deliberately register multiple fake accounts and use automated scripts to keep these fake accounts active to meet the reward distribution conditions. Currently, it is difficult to identify such potentially abnormal users. In this embodiment, users engaging in fraudulent user acquisition behavior and users registering fake accounts can both be referred to as abnormal users.

[0033] This disclosure provides an information mining method that can be applied to electronic devices, such as terminals or servers. Figure 1 As shown, the method includes:

[0034] S101. Obtain basic user information and behavior-related information for each user from the log information.

[0035] Among them, behavior-related information is used to represent information related to various user behaviors within the app. For example, user behaviors within the app may include user acquisition, login, registration, file upload, file download, and withdrawal behaviors.

[0036] S102. Generate a graph database based on user basic information and behavior-related information.

[0037] The graph database includes multiple nodes and the connections between them, including nodes under review and abnormal nodes.

[0038] Nodes awaiting review refer to all nodes in the graph database except for abnormal nodes. Abnormal nodes are nodes that have been identified as belonging to abnormal users. It is understandable that for some obvious abnormal behaviors, existing anomaly identification methods or manual judgment can be used to identify whether a user has been identified as an abnormal user, and the abnormal user's user ID has been marked in the log record or recorded in a preset blacklist. Electronic devices can mark each node as an abnormal node when generating the graph database.

[0039] The graph database includes various types of nodes. For example, in the case of an app that functions as a cloud storage service, the graph database may include user nodes, device nodes, and file nodes. The node types in this embodiment are merely examples; in actual implementations, the node types are not limited to these.

[0040] As an example, such as Figure 2 As shown, the fields in the user node include vertex type, vertex ID, whether it is abnormal (is_black), whether it is a website administrator (is_webmaster), and user ID.

[0041] Here, vertex type indicates node type. Figure 2 The node shown is of type user.

[0042] Vertex ID is used to represent the ID of the node itself, for example, vertex ID is 1:20076612355;

[0043] The "is_black" field is used to mark whether the node is an abnormal node. If the user node is not an abnormal node, the value of the "is_black" field is no (false).

[0044] The "Is the webmaster?" option indicates whether the user corresponding to this node is a webmaster. If the user is a webmaster, the "Is the webmaster?" option will be marked as "true".

[0045] User ID is used to represent the unique identifier of a user in the APP, for example, user ID is 20076612355.

[0046] Understandably, nodes in the graph database whose is_black field is false are all abnormal nodes.

[0047] S103. Based on the community detection algorithm, the nodes included in the graph database are divided into multiple communities.

[0048] Among them, the community detection algorithm can be either the label propagation algorithm or the Louvain community algorithm.

[0049] Taking the label propagation algorithm as an example, the label propagation algorithm can use the labels of labeled nodes to predict the labels of unlabeled nodes. In this embodiment, each abnormal node can be labeled first, and then the labels of other nodes can be generated by using the labels of abnormal nodes and the association between nodes in the graph database through the label propagation algorithm. Then, nodes with the same labels are divided into the same community.

[0050] S104. For each node to be reviewed in the graph database, if the density between the node to be reviewed and the abnormal node in its community is greater than a preset threshold, then the node to be reviewed is determined to be an abnormal node.

[0051] Here, density represents the degree of correlation between the node under review and the abnormal node. The preset threshold can be set based on experience.

[0052] Using the above method, electronic devices generate a graph database based on user basic information and behavior-related information. Then, based on a community detection algorithm, the nodes in the graph database are divided into multiple communities. By calculating the density between each node under review and abnormal nodes in its community, it is determined whether the node under review is an abnormal node. Since nodes grouped into the same community according to the community detection algorithm have high similarity, and the density between the node under review and abnormal nodes in its community reflects the degree of association between them, when the density is greater than a preset threshold, it indicates a strong association between the node under review and abnormal nodes, and the node under review is more likely to be an abnormal node. Thus, abnormal nodes can be accurately identified. Furthermore, by dividing the data into communities, nodes with high similarity in the entire graph database can be grouped into the same community, which can uncover the complex relationships among multiple participants in abnormal activities, thereby identifying potential abnormal nodes and improving the accuracy of abnormal node identification.

[0053] In some embodiments of this application, the electronic device can obtain the user's basic information and behavior-related information from the log records in the APP's log information.

[0054] User basic information includes at least one of the following: user ID, registered account, and whether the user is an abnormal user.

[0055] The registered account can be the account entered by the user when registering an account, such as a username or mobile phone number, or other information used to identify the user. This application embodiment does not limit this.

[0056] Behavioral information includes at least one of the following:

[0057] In the user acquisition process, determine the ID of the user who acquires the new user, the ID of the user who is acquired, and whether the user who acquires the new user and the user who is acquired are abnormal users.

[0058] The login process includes the device identifier of the device being logged in, the IP address of the data center being logged in, and whether the device being logged in is a risky device.

[0059] The registration process includes checking the data center IP address accessed during registration, the device identifier of the device used during registration, and whether the device used during registration is a risky device.

[0060] The uploaded file name, device identifier of the device logged in during upload, data center IP address, and whether the uploaded file is an invalid file are included in the file upload process.

[0061] The download file name, the device identifier of the device logged in when downloading the file, the IP address of the logged-in data center, and whether the downloaded file is an invalid file are all considered valid information in the download process.

[0062] Optionally, the behavior-related information may also include the withdrawing user ID, the withdrawing financial account, the data center IP accessed during the withdrawal, the device identifier, and whether the withdrawing user is an abnormal user. This application embodiment does not limit the form of the financial account.

[0063] It should be noted that the above data center IP can specifically refer to the IP address of the data center or the city where the data center is located.

[0064] The aforementioned basic user information and behavior-related information can reflect various user behaviors in the APP. After generating a graph database, the edges of the graph database can reflect the relationships between nodes. As the relationships between nodes in the graph database become clear, complex potential relationships between nodes can be discovered, which is more conducive to discovering potential abnormal nodes.

[0065] In some embodiments of this disclosure, when the user's basic information includes a registered account, the above-mentioned step S102, generating a graph database based on the user's basic information and behavior-related information, can be implemented as follows:

[0066] Extract the account prefixes of the registered accounts of newly acquired users, establish a mapping relationship between the registered accounts and the prefixes, and generate a graph database based on user basic information, user behavior-related information, and the mapping relationship.

[0067] In this context, the account prefix refers to the content shared by multiple registered accounts, excluding any common elements. Taking the username as an example, if there are three newly added users with usernames AAA1, AAA2, and AAA3, the account prefix AAA can be extracted. Furthermore, a mapping relationship can be established between usernames AAA1, AAA2, and AAA3 and the account prefix AAA.

[0068] Furthermore, electronic devices can also use account prefixes as nodes in the graph database, and based on the above mapping relationship, establish a connection between account prefix nodes and user nodes in the graph database.

[0069] like Figure 3 As shown, Figure 3 This is an exemplary schematic diagram of a graph database provided in this disclosure embodiment. The graph database includes user nodes 1 to 4, device node 1, financial account node 1, prefix node 1, data center IP node 1, file node 1, and the connection relationships between each node.

[0070] In this system, user node 1 is connected to device node 1, and the connection line between them indicates that the registered account corresponding to user node 1 was registered on the device corresponding to device node 1.

[0071] User node 1 is connected to financial account node 1, and the connection line between the two indicates that the earnings obtained by the registered account corresponding to user node 1 are withdrawn through the financial account corresponding to financial account node 1.

[0072] User node 1 is connected to file node 1, and the connection line between them indicates that the registered account corresponding to user node 1 has downloaded the file corresponding to file node 1.

[0073] User node 1 is connected to user node 3, and the connection line between them indicates that there is an invitation relationship between the registered account corresponding to user node 1 and the registered account corresponding to user node 3. That is, the user corresponding to user node 1 has invited the user corresponding to user node 3 to become a new user.

[0074] User node 1 is connected to data center IP node 1, and the connection line between the two indicates that the registered account corresponding to user node 1 obtains resources from the data center corresponding to data center IP node 1.

[0075] User node 2 is connected to device node 1, and the connection line between them indicates that the registered account corresponding to user node 2 is logged in on the device corresponding to device node 1.

[0076] User node 4 is connected to file node 1, and the connection line between them indicates that the registered account corresponding to user node 4 uploaded the file corresponding to file node 1.

[0077] User node 3 is connected to data center IP node 1, and the connection line between the two indicates that the registered account corresponding to user node 3 has checked in at the data center corresponding to data center IP node 1.

[0078] User node 1 and user node 2 are both connected to prefix node 1, indicating that the registered account corresponding to user node 1 and the registered account corresponding to user node 2 have the same account prefix.

[0079] Since abnormal users may register multiple accounts with the same account prefix in batches, establishing a mapping relationship between registered accounts and account prefixes, and using this mapping relationship when generating the graph database, allows us to represent the relationship between user nodes and account prefixes in the graph database. This enables subsequent community detection algorithms to uncover potential relationships between user nodes with the same account prefix, and even potential relationships between other nodes connected to these user nodes, thus identifying potential abnormal users.

[0080] In some embodiments of this disclosure, in step S104 above, for each node to be reviewed in the graph database, if the closeness between the node to be reviewed and the abnormal nodes in its community is greater than a preset threshold, then the node to be reviewed is determined to be an abnormal node. This includes the following two implementation methods:

[0081] In one implementation, for each node to be reviewed in the graph database, the number of connections between the node to be reviewed and abnormal nodes in its community is determined; if the number of connections is greater than a preset connection threshold, the node to be reviewed is determined to be an abnormal node.

[0082] The aforementioned density can be represented by the number of connections between the node under review and the abnormal nodes in its community. The number of connections refers to the number of abnormal nodes connected to the node under review within its community.

[0083] Understandably, if the number of connections between the node under review and abnormal nodes in the community exceeds a preset connection threshold, it indicates that the node under review is too closely linked to abnormal nodes in the community, suggesting that the node under review is also an abnormal node. By mining the abnormal nodes in the graph database through the connection relationships between the node under review and abnormal nodes, potential abnormal nodes can be identified.

[0084] In another implementation, the electronic device can determine the first number of connections between each node to be reviewed and the abnormal nodes in its community for each node to be reviewed in the graph database, and determine the total number of connections of the node to be reviewed in its community; if the ratio between the first number of connections and the total number of connections is greater than a preset weight threshold, then the node to be reviewed is determined to be an abnormal node.

[0085] The aforementioned density can also be expressed as the ratio between the first number of connections and the total number of connections. The first number of connections is the number of abnormal nodes connected to the node under review within the community to which the node under review belongs. The total number of connections refers to the number of nodes connected to the node under review within the community to which the node under review belongs.

[0086] Understandably, within the community to which the node under review belongs, it can be connected to multiple nodes, and these connected nodes may include anomalous nodes. The electronic device can calculate the proportion of the first connection number between the node under review and anomalous nodes within its community to the total number of connections the node under review has within that community. When this proportion exceeds a preset threshold, it indicates that a large number of anomalous nodes are connected to the node under review, the node under review is too closely linked to anomalous nodes, and the node under review is highly likely to be an anomalous node itself. This allows for the identification of potential anomalous nodes based on the connection relationships between nodes in the graph database.

[0087] In some embodiments of this disclosure, the electronic device can also determine whether a newly acquired user is an abnormal user based on a graph database. A newly acquired user refers to a user who acquires new users. Therefore, after generating the graph database based on user basic information and behavior-related information in S102 above, abnormal information can also be determined using the following methods, such as... Figure 4 As shown, the method also includes S401-S402.

[0088] It should be noted that after S102, S103-S104 can also be executed, that is, S103-S104 and S401-S402 are parallel implementations. The implementation of this application does not limit the execution order of these two parallel implementations. For example, S103-S104 can be executed first, or S401-S402 can be executed first, or they can be executed simultaneously.

[0089] S401. For each new user, retrieve from the graph database the number of new users recruited by that user each day, the number and quality of videos uploaded by each user recruited each day, and the number of videos played by each user recruited each day.

[0090] Here, the number of new users acquired refers to the number of users that the acquiring user brings in each day. Electronic devices can retrieve the number of connections in the graph database representing new users acquired by that user, and use this number as the number of new users acquired. For example, Figure 3 If user node 1 only recruited new users from user node 3, then the number of new users recruited by user node 1 is 1.

[0091] It is understandable that files uploaded and downloaded by users can include text files, image files, audio files, or video files. In the case of video files, the file nodes in the graph database can include video quality, which is used to indicate whether the file is an unqualified file.

[0092] It should be noted that the app itself can determine whether a video file is unqualified based on its size or format, and record the determination result in the log information. This disclosure does not limit the method by which the app determines whether a video file is unqualified.

[0093] If the log information in S101 is the APP's log information for one day, the electronic device can use the graph database to determine the new user corresponding to the new user. Then, for each new user, it can determine the number of file nodes with upload relationships between them and the user node where the new user is located, and use this number as the number of files uploaded by the new user. Similarly, the electronic device can also determine the number of file nodes with playback relationships between them and the user node where the new user is located, and use this number as the number of files played by the new user.

[0094] If the log information in S101 is log information recorded by the APP for multiple days, you can divide the number obtained above by the number of days the log was recorded to get the number of new users acquired, the number of files downloaded, and the number of files played per day.

[0095] S402. Based on the number of new users recruited by the user each day, the number and quality of videos uploaded by each user recruited by the user each day, and the number of videos played by each user recruited each day, determine whether the user recruited by the user is an abnormal user.

[0096] Understandably, if the users recruited by the new user exhibit abnormal behavior, then those users are considered abnormal users. If the number of abnormal users recruited by the new user is relatively large, then the new user recruiter is more likely to be an abnormal user.

[0097] For example, let's say User A is a new user referrer. User A refers to users 1 through 30 within a single day. Users 1 through 30 upload a large number of videos daily, all of which are substandard. Furthermore, users 1 through 30 also watch a large number of videos daily. All three of these behaviors are considered abnormal. Since users 1 through 30 exhibit all three behaviors, they can be classified as abnormal users. Therefore, User A, as a referrer of users 1 through 30, is also considered an abnormal user.

[0098] Using the above method, the number and quality of videos uploaded daily by each user recruited by the new user, as well as the number of videos played daily by each recruited user, can indicate whether the recruited users exhibit abnormal behavior. When most of the users recruited by the new user exhibit abnormal behavior, and the number of new users recruited by the new user is large, even if the new user does not exhibit obvious abnormal behavior, the new user is a beneficiary of the abnormal behavior of the recruited users. In this case, the electronic device can identify the new user as an abnormal user. It is evident that this method can identify abnormal users without obvious abnormal behavior.

[0099] The following describes how an electronic device can identify abnormal users based on the number of new users a user recruits each day, the number and quality of videos uploaded by each user recruited by that user each day, and the number of videos played by each recruited user each day. Specifically, S402 above, determining whether a user recruited by that user is an abnormal user based on their daily number of new users recruited, the number and quality of videos uploaded by each user recruited by that user each day, and the number of videos played by each recruited user each day, can be implemented as follows:

[0100] Step 1: Determine whether the number of new users referred by the user each day is greater than the first threshold.

[0101] The first threshold can be the median number of new users acquired by each user per day, or it can be preset based on experience.

[0102] Step 2: Determine whether the number of videos uploaded per day by each user invited by the new user exceeds the second threshold.

[0103] The second threshold can be the median number of videos uploaded per day by each user referred by each new user, or the second threshold can be preset based on experience.

[0104] Step 3: Determine whether the number of videos played by each user per day by the user who brought in the new user is greater than the third threshold.

[0105] The third threshold is the median number of videos played per day by each user brought in by each new user, or the third threshold can be preset based on experience.

[0106] Step 4: Determine whether the failure rate of the videos uploaded by the users recruited by the new user is greater than the preset failure rate threshold.

[0107] It should be noted that the execution order of steps 1-4 above is only an example, and the embodiments of this application do not limit the execution order of steps 1-4 above.

[0108] As an example, the default failure rate can be 20%. If user A pulls user B to upload 50 videos in one day, and 40 of them are unqualified, then the failure rate of the videos uploaded by user B is 90%, and user B is likely an abnormal user.

[0109] Step 5: If all judgment results are yes, then the newly acquired user is determined to be an abnormal user; if any judgment result is no, then the newly acquired user is determined to be not an abnormal user.

[0110] Understandably, the first, second, and third thresholds mentioned above represent the median number of new users acquired, the median number of uploaded videos, and the median number of videos played, respectively, for multiple users who acquired new users. These thresholds accurately reflect the user behavior habits of most users. Furthermore, if the video quality failure rate of the users acquired by the acquiring user exceeds a preset failure rate threshold, and the number of videos played daily by the users acquired by the acquiring user also exceeds the third threshold, it indicates that the behavior of the users acquired by the acquiring user does not conform to normal user behavior habits; that is, most of the users acquired by the acquiring user are abnormal users. Since the acquiring user acquired a large number of abnormal users, it indicates that the acquiring user is also an abnormal user. Thus, even when the acquiring user does not exhibit obvious abnormal behavior, it is possible to accurately identify potential abnormal acquiring users.

[0111] In some embodiments of this disclosure, the electronic device can also determine whether a registered account is an abnormal node based on the risk device and cloud data center IP corresponding to the registered account. After obtaining the user's basic information and behavior-related information from the log information in S101 above, the electronic device can also determine abnormal accounts through the following methods, such as... Figure 5 As shown, the method also includes: S501-S502.

[0112] It should be noted that, Figure 1 , Figure 4 and Figure 5 The corresponding embodiments are three parallel schemes, and the execution order of the unique steps of each embodiment is not limited in the embodiments of this application.

[0113] S501. For each registered account, obtain the number of devices, data center IPs, risky devices, and cloud data center IPs corresponding to that registered account.

[0114] In one implementation, the electronic device can further calculate the number of devices, data center IPs, risky devices, and cloud data center IPs for each registered account from the acquired user basic information and behavior-related information.

[0115] In another implementation, S501 can be executed after S102, and the electronic device can take the number of device nodes connected to the user node corresponding to the registered account in the graph database as the number of devices corresponding to the registered account, and take the number of device nodes marked as risky devices among these device nodes as the number of risky devices corresponding to the registered account; take the number of data center IP nodes connected to the user node corresponding to the registered account in the graph database as the number of data center IPs corresponding to the registered account, and take the number of data center IP nodes marked as cloud data center IPs among these data center IP nodes as the number of cloud data center IPs corresponding to the registered account.

[0116] S502. If the number of devices exceeds the device number threshold, the number of data center IPs exceeds the data center IP number threshold, the ratio of the number of risky devices to the total number of devices exceeds the risky device percentage threshold, and the ratio of the number of cloud data center IPs to the total number of data center IPs exceeds the cloud IP percentage threshold, then the registered account is determined to be an abnormal account.

[0117] Among them, the thresholds for the number of devices, the number of data center IPs, the proportion of risky devices, and the proportion of cloud data center IPs can be preset based on experience.

[0118] Using the embodiments of this application, the number of devices, data center IPs, percentage of risky devices, and percentage of cloud IPs corresponding to a registered account can be statistically obtained. The percentage of risky devices and cloud IPs is only meaningful when both the number of devices and data center IPs corresponding to a registered account are relatively large. Since normal users typically log in from their own devices and a few fixed locations (such as their home or office), if it is determined that the percentage of risky devices under a registered account is greater than or equal to the risky device percentage threshold, and the percentage of cloud data center IPs is greater than or equal to the cloud IP percentage threshold, it indicates that the registered account has logged in on a large number of risky devices and a large number of cloud data center IPs, which does not conform to the behavioral characteristics of normal users. Therefore, the registered account can be identified as an abnormal account. In this way, abnormal accounts can be accurately identified.

[0119] If there are obvious abnormal behaviors in the information corresponding to the nodes to be examined in the graph database, the electronic device can first identify the abnormal information based on a preset strategy, and then use the Online Transaction Processing (OLTP) algorithm to determine the abnormal nodes. After generating the graph database based on user basic information and behavior-related information in S102, the method further includes:

[0120] If a specified type of abnormal information has been identified using a preset strategy, then the OLTP algorithm and the specified type of abnormal information are used to identify the nodes associated with the abnormal information from the graph database, and the identified nodes are taken as abnormal nodes.

[0121] The specified type of abnormal information refers to abnormal information with obvious clustering abnormal characteristics identified using a preset strategy. For example, the preset strategy could be that the number of user nodes connected to the same financial account node in the graph database exceeds a first preset threshold, or the number of user nodes connected to the same prefix node in the graph database exceeds a second preset threshold, or the number of user nodes connected to the same device node in the graph database exceeds a third preset threshold. Information identified using the above three preset strategies can all be used as specified type of abnormal information. Specified type of abnormal information can also be other abnormal information with obvious clustering, which can be set according to the usage characteristics of the APP. These will not be listed individually in this embodiment.

[0122] Corresponding to the three preset strategies listed above, electronic devices can use the OLTP algorithm to identify user nodes connected to the same financial account node in the graph database as abnormal nodes, or identify user nodes connected to the same financial account node as abnormal nodes, or identify user nodes connected to the same device node as abnormal nodes.

[0123] In one alternative implementation, this step can be used as Figure 1 , Figure 4 and Figure 5 Parallel implementation schemes of the corresponding embodiments.

[0124] In another alternative implementation, this step can be performed after S102. Figure 1 , Figure 4 and Figure 5 The subsequent steps in the corresponding embodiment can more comprehensively identify abnormal nodes.

[0125] Using the above method, after identifying specified types of anomaly information based on a preset strategy, the electronic device can also identify anomaly nodes from the graph database using an OLTP algorithm. In this way, the electronic device can quickly identify anomaly nodes in the graph database, improving identification efficiency.

[0126] The information mining method provided in this application embodiment facilitates real-time monitoring of abnormal events and behaviors, enables data analysts to analyze the relationships between devices, users, and apps, promptly detects abnormal behaviors and potential intrusion activities, and provides timely alerts and responses, thereby improving app security.

[0127] Furthermore, for apps that need to acquire a large number of new users, this method can help assess the number of genuine and abnormal users among the current new users, thereby evaluating the effectiveness of app promotion. It also has significant advantages in combating fake account registrations, "coupon hunters," and identifying anomalies during various promotional activities.

[0128] Furthermore, the electronic device executing the embodiments of this application can be a server independent of the APP backend server, thus serving as a platform for sharing and cooperating on abnormal intelligence. It can integrate and mine data from multiple APPs (e.g., main APP and derivative APPs, or multiple APPs with related relationships), thereby discovering a wider range of abnormal networks and more comprehensive abnormal information.

[0129] Based on the same concept, embodiments of this disclosure provide an apparatus for information mining, such as... Figure 6 As shown, the device includes:

[0130] The acquisition module 601 is used to acquire basic user information and behavior-related information of each user from the log information;

[0131] The generation module 602 is used to generate a graph database based on user basic information and behavior-related information. The graph database includes multiple nodes and the connection relationships between nodes. The multiple nodes include nodes to be reviewed and abnormal nodes.

[0132] The partitioning module 603 is used to divide the nodes in the graph database into multiple communities based on the community detection algorithm;

[0133] The determination module 604 is used to determine that each node to be reviewed in the graph database is an abnormal node if the closeness between the node to be reviewed and the abnormal node in its community is greater than a preset threshold.

[0134] Optionally, module 604 is defined, specifically for:

[0135] For each node to be reviewed in the graph database, determine the number of connections between that node and the abnormal nodes in its community.

[0136] If the number of connections exceeds the preset connection threshold, the node to be reviewed is determined to be an abnormal node.

[0137] Optionally, module 604 is defined, specifically for:

[0138] For each node to be reviewed in the graph database, determine the first number of connections between the node to be reviewed and the abnormal nodes in its community, and determine the total number of connections of the node to be reviewed in its community.

[0139] If the ratio between the first number of connections and the total number of connections is greater than a preset weight threshold, then the node to be reviewed is determined to be an abnormal node.

[0140] Optionally, the device may also include a determination module:

[0141] The acquisition module 601 is also used to acquire, for each new user, the number of new users acquired by that user per day, the number and quality of videos uploaded by each user acquired by that user per day, and the number of videos played by each user acquired per day from the graph database.

[0142] The judgment module is used to determine whether a user who brought in new users is an abnormal user based on the number of new users brought in by the user each day, the number and quality of videos uploaded by each user brought in by the user each day, and the number of videos played by each user brought in each day.

[0143] Optional, a judgment module, specifically used for:

[0144] Determine whether the number of new users acquired by this user each day exceeds the first threshold;

[0145] Determine whether the number of videos uploaded per day by each user referred by the new user exceeds the second threshold;

[0146] Determine whether the number of videos played per day by each user referred by the newly acquired user exceeds the third threshold;

[0147] Determine whether the failure rate of the videos uploaded by the users recruited by the new user is greater than the preset failure rate threshold;

[0148] If all the judgment results are yes, then the newly acquired user is determined to be an abnormal user;

[0149] If any judgment result is negative, then the newly acquired user is determined not to be an abnormal user.

[0150] Optionally, the first threshold is the median number of new users acquired per day by each user who acquired new users;

[0151] The second threshold is the median number of videos uploaded per day by each user referred by each new user;

[0152] The third threshold is the median number of videos played per day by each user referred by each new user.

[0153] Optionally, the acquisition module 601 is also used to, after obtaining the user's basic information and behavior-related information of each user from the log information, obtain the number of devices, the number of data center IPs, the number of risky devices and the number of cloud data center IPs corresponding to each registered account;

[0154] The determination module 604 is also used to determine that the registered account is an abnormal account if the number of devices is greater than the device number threshold, the number of data center IPs is greater than the data center IP number threshold, the ratio of the number of risky devices to the number of devices is greater than the risky device proportion threshold, and the ratio of the number of cloud data center IPs to the number of data center IPs is greater than the cloud IP proportion threshold.

[0155] Optional, basic user information includes the registered account;

[0156] Module 602 is specifically used for:

[0157] Extract the account prefix of the registered account of the new user and establish a mapping relationship between the registered account and the account prefix;

[0158] A graph database is generated based on basic user information, user behavior-related information, and mapping relationships.

[0159] Optionally, the device may also include an identification module:

[0160] The identification module is used to identify nodes associated with the abnormal information from the graph database using the OLTP algorithm and the abnormal information of the specified type if the specified type of abnormal information has been identified using a preset strategy, and to identify the identified nodes as abnormal nodes.

[0161] Optionally, the user's basic information includes at least one of the following: user's ID, registered username, registered email, and whether the user is an abnormal user;

[0162] Behavioral information includes at least one of the following:

[0163] In the user acquisition process, determine the ID of the user who acquires the new user, the ID of the user who is acquired, and whether the user who acquires the new user and the user who is acquired are abnormal users.

[0164] The login process includes the device identifier of the device being logged in, the IP address of the data center being logged in, and whether the device being logged in is a risky device.

[0165] The registration process includes checking the data center IP address accessed during registration, the device identifier of the device used during registration, and whether the device used during registration is a risky device.

[0166] The uploaded file name, device identifier of the device logged in during upload, data center IP address, and whether the uploaded file is an invalid file are included in the file upload process.

[0167] The download file name, the device identifier of the device logged in when downloading the file, the IP address of the logged-in data center, and whether the downloaded file is an invalid file are all considered valid information in the download process.

[0168] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0169] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0170] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0171] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0172] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as information mining methods. For example, in some embodiments, the information mining method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the information mining method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform information mining methods by any other suitable means (e.g., by means of firmware).

[0173] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0175] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0177] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0178] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0179] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0180] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An information mining method, comprising: Obtain basic user information and behavior-related information for each user from log information, wherein the basic user information includes the registered account; Extract the account prefix of the registered account of the user who was invited to join, establish a mapping relationship between the registered account and the account prefix, and generate a graph database based on the user's basic information, user behavior information and the mapping relationship. The graph database includes multiple nodes and the connection relationship between the nodes. The multiple nodes include nodes under review and abnormal nodes. The graph database includes user nodes and prefix nodes. The connection relationship between user nodes indicates that there is an invitation relationship between the registered accounts corresponding to the user nodes. Based on the community detection algorithm, the nodes included in the graph database are divided into multiple communities; For each node to be reviewed in the graph database, if the density between the node to be reviewed and the abnormal node in its community is greater than a preset threshold, then the node to be reviewed is determined to be an abnormal node.

2. The method according to claim 1, wherein, For each node to be reviewed in the graph database, if the density of the node to be reviewed with abnormal nodes in its community is greater than a preset threshold, then the node to be reviewed is determined to be an abnormal node, including: For each node to be reviewed in the graph database, determine the number of connections between the node to be reviewed and the abnormal nodes in its community; If the number of connections is greater than a preset connection threshold, the node to be reviewed is determined to be an abnormal node.

3. The method according to claim 1, wherein, For each node to be reviewed in the graph database, if the density of the node to be reviewed with abnormal nodes in its community is greater than a preset threshold, then the node to be reviewed is determined to be an abnormal node, including: For each node to be reviewed in the graph database, determine the first number of connections between the node to be reviewed and the abnormal nodes in its community, and determine the total number of connections of the node to be reviewed in its community. If the ratio between the first number of connections and the total number of connections is greater than a preset weight threshold, then the node to be reviewed is determined to be an abnormal node.

4. The method according to claim 1, after generating the graph database based on the user's basic information, user behavior-related information, and the mapping relationship, the method further includes: For each new user acquired, the daily number of new users acquired by that user, the daily number and quality of videos uploaded by each user acquired by that user, and the daily number of videos played by each user acquired by that user are obtained from the graph database. Based on the number of new users a user brings in each day, the number and quality of videos uploaded by each user they bring in each day, and the number of videos played by each user they bring in each day, we can determine whether the user who brought in the new user is an abnormal user.

5. The method according to claim 4, wherein, The determination of whether a user is an abnormal user based on the number of new users they refer each day, the number and quality of videos uploaded by each user they refer each day, and the number of videos played by each user they refer each day includes: Determine whether the number of new users acquired by this user each day exceeds the first threshold; Determine whether the number of videos uploaded per day by each user referred by the new user exceeds the second threshold; Determine whether the number of videos played per day by each user referred by the newly acquired user exceeds the third threshold; Determine whether the failure rate of the videos uploaded by the users recruited by the new user is greater than the preset failure rate threshold; If all the judgment results are yes, then the newly acquired user is determined to be an abnormal user; If any judgment result is negative, then the newly acquired user is determined not to be an abnormal user.

6. The method according to claim 5, wherein, The first threshold is the median number of new users acquired per day by each user who acquired new users; The second threshold is the median number of videos uploaded per day by each user referred by each new user; The third threshold is the median number of videos played per day by each user who is referred by each new user.

7. The method according to claim 1, after obtaining the user's basic information and behavior-related information from the log information, the method further includes: For each registered account, obtain the number of devices, data center IPs, risky devices, and cloud data center IPs corresponding to that account. If the number of devices is greater than the device number threshold, the number of data center IPs is greater than the data center IP number threshold, the ratio of the number of risky devices to the number of devices is greater than the risky device proportion threshold, and the ratio of the number of cloud data center IPs to the number of data center IPs is greater than the cloud IP proportion threshold, then the registered account is determined to be an abnormal account.

8. The method according to any one of claims 1-7, after generating the graph database based on the user basic information, user behavior related information, and the mapping relationship, the method further includes: If a specified type of abnormal information has been identified using a preset strategy, then the Online Transaction Processing (OLTP) algorithm and the specified type of abnormal information are used to identify the nodes associated with the abnormal information from the graph database, and the identified nodes are taken as abnormal nodes.

9. The method according to any one of claims 1-7, wherein, The basic user information includes at least one of the following: user ID, registered username, registered email address, and whether the user is an abnormal user; The behavior-related information includes at least one of the following: In the user acquisition process, determine the ID of the user who acquires the new user, the ID of the user who is acquired, and whether the user who acquires the new user and the user who is acquired are abnormal users. The login process includes the device identifier of the device being logged in, the IP address of the data center being logged in, and whether the device being logged in is a risky device. The registration process includes checking the data center IP address accessed during registration, the device identifier of the device used during registration, and whether the device used during registration is a risky device. The uploaded file name, device identifier of the device logged in during upload, data center IP address, and whether the uploaded file is an invalid file are included in the file upload process. The download file name, the device identifier of the device logged in when downloading the file, the IP address of the logged-in data center, and whether the downloaded file is an invalid file are all considered valid information in the download process.

10. An apparatus for information mining, comprising: The acquisition module is used to acquire basic user information and behavior-related information of each user from log information, wherein the basic user information includes the registered account; The generation module is used to extract the account prefix of the registered account of the user being invited, establish a mapping relationship between the registered account and the account prefix, and generate a graph database based on the user's basic information, user behavior-related information and the mapping relationship. The graph database includes multiple nodes and the connection relationship between the nodes. The multiple nodes include nodes to be reviewed and abnormal nodes. The graph database includes user nodes and prefix nodes. The connection relationship between user nodes indicates that there is an invitation relationship between the registered accounts corresponding to the user nodes. The partitioning module is used to divide the nodes included in the graph database into multiple communities based on a community detection algorithm. The determination module is used to determine that each node to be reviewed in the graph database is an abnormal node if the closeness between the node to be reviewed and the abnormal node in its community is greater than a preset threshold.

11. The apparatus according to claim 10, wherein, The determining module is specifically used for: For each node to be reviewed in the graph database, determine the number of connections between the node to be reviewed and the abnormal nodes in its community; If the number of connections is greater than a preset connection threshold, the node to be reviewed is determined to be an abnormal node.

12. The apparatus according to claim 10, wherein, The determining module is specifically used for: For each node to be reviewed in the graph database, determine the first number of connections between the node to be reviewed and the abnormal nodes in its community, and determine the total number of connections of the node to be reviewed in its community. If the ratio between the first number of connections and the total number of connections is greater than a preset weight threshold, then the node to be reviewed is determined to be an abnormal node.

13. The apparatus according to claim 10, further comprising a determining module: The acquisition module is also used to acquire, for each new user, the number of new users acquired by that user per day, the number and quality of videos uploaded by each user acquired by that user per day, and the number of videos played by each user acquired per day from the graph database. The judgment module is used to determine whether a user is an abnormal user based on the number of new users the user attracts each day, the number and quality of videos uploaded by each user the user attracts each day, and the number of videos played by each user the user attracts each day.

14. The apparatus according to claim 13, wherein, The judgment module is specifically used for: Determine whether the number of new users acquired by this user each day exceeds the first threshold; Determine whether the number of videos uploaded per day by each user referred by the new user exceeds the second threshold; Determine whether the number of videos played per day by each user referred by the newly acquired user exceeds the third threshold; Determine whether the failure rate of the videos uploaded by the users recruited by the new user is greater than the preset failure rate threshold; If all the judgment results are yes, then the newly acquired user is determined to be an abnormal user; If any judgment result is negative, then the newly acquired user is determined not to be an abnormal user.

15. The apparatus according to claim 14, wherein, The first threshold is the median number of new users acquired per day by each user who acquired new users; The second threshold is the median number of videos uploaded per day by each user referred by each new user; The third threshold is the median number of videos played per day by each user who is referred by each new user.

16. The apparatus according to claim 10, The acquisition module is further configured to, after acquiring the user basic information and behavior-related information of each user from the log information, acquire, for each registered account, the number of devices, the number of data center IPs, the number of risky devices, and the number of cloud data center IPs corresponding to that registered account; The determining module is further configured to determine that the registered account is an abnormal account if the number of devices is greater than the device number threshold, the number of data center IPs is greater than the data center IP number threshold, the ratio of the number of risky devices to the number of devices is greater than the risky device proportion threshold, and the ratio of the number of cloud data center IPs to the number of data center IPs is greater than the cloud IP proportion threshold.

17. The apparatus according to any one of claims 10-16, further comprising an identification module: The identification module is used to identify, if a specified type of abnormal information has been identified using a preset strategy, the nodes associated with the abnormal information from the graph database using the Online Transaction Processing (OLTP) algorithm and the specified type of abnormal information, and to identify the identified nodes as abnormal nodes.

18. The apparatus according to any one of claims 10-16, wherein, The basic user information includes at least one of the following: user ID, registered username, registered email address, and whether the user is an abnormal user; The behavior-related information includes at least one of the following: In the user acquisition process, determine the ID of the user who acquires the new user, the ID of the user who is acquired, and whether the user who acquires the new user and the user who is acquired are abnormal users. The login process includes the device identifier of the device being logged in, the IP address of the data center being logged in, and whether the device being logged in is a risky device. The registration process includes checking the data center IP address accessed during registration, the device identifier of the device used during registration, and whether the device used during registration is a risky device. The uploaded file name, device identifier of the device logged in during upload, data center IP address, and whether the uploaded file is an invalid file are included in the file upload process. The download file name, the device identifier of the device logged in when downloading the file, the IP address of the logged-in data center, and whether the downloaded file is an invalid file are all considered valid information in the download process.

19. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

21. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-9.

Images