Iterative clue discovery method and system for social media

Abstract

The invention provides a social media-oriented iterative clue discovery method and system. Comprising a clue main body importance degree scoring method fusing texts, structures and interactive information. The method comprises the following steps: respectively modeling text content, a social network structure and interaction information of candidate accounts to respectively obtain importance scores of three dimensions of content, structure and interaction, and weighting the scores of multiple dimensions to obtain a final clue subject importance score. And selecting the highest score as a clue discovery result. According to the method and the system, the content, the structure and the interaction information in the social media platform can be comprehensively utilized to score the clue main body, and the accuracy of clue main body identification is improved.

Description

Technical Field

[0001] The present invention is applicable to the technical field of account retrieval in big data analysis, and in particular relates to a clue discovery method and system for social media. Background Art

[0002] The purpose of clue discovery is to collect relevant content and account information based on known clue information, such as keywords, social media accounts, etc., and to discover key clue subject information. Before the emergence of social media, the main goal of clue discovery was to mine keyword information related to clues for subsequent public opinion monitoring, clue tracking and other businesses. The main goal of social media clue discovery is to mine the key subjects involved in clue information (i.e. social media platform accounts) and monitor and track clue information from the source.

[0003] Existing clue analysis methods mainly use text content information as input, and model clue-related information into several topics through topic modeling. After obtaining the topic distribution, the frequency of user participation under each topic is counted, and the high-frequency users under the topic are obtained as clue subjects according to the frequency of participation. Some clue discovery methods also introduce network structure features. By analyzing the social network structure information and interaction information between different users in the clue information, users with high structural importance are found and used as key subjects.

[0004] Although existing clue discovery methods have been applied in different business scenarios, due to the wide variety of information and complex dissemination patterns on social media platforms, existing clue discovery methods still have some shortcomings and deficiencies.

[0005] First, existing clue discovery methods usually only use one dimension of data to analyze the main body of clues, such as the text content information or network structure information mentioned above. In fact, the data types in social media platforms are very rich, including basic information of users, content of users' posts, social network relationships between users, interactions between users (forwards, comments, likes, etc.), etc. These information are important indicative features of clue information. It is difficult to accurately locate and fully recall the main body of clues using single-dimensional data.

[0006] Secondly, the dissemination of different clue information is complicated, and it is difficult to use a universal model to uniformly discover the main clue information. Most existing methods adopt unsupervised analysis methods, which are easily affected by data noise. At the same time, it is difficult to guarantee the accuracy of the results, requiring a lot of repeated verification work by business personnel, and the actual use efficiency needs to be improved.

[0007] In summary, the existing clue discovery methods have two main problems: (1) They use single-dimensional data to characterize the clue subject, which results in low accuracy in locating the clue subject; (2) There is a lack of effective supervisory information guidance during the recognition process, resulting in poor clue discovery results and requiring a lot of manual review. Summary of the invention

[0008] The purpose of the present invention is to solve the problem that the existing clue analysis methods for social media have a single analysis dimension and the analysis process lacks supervision information and is easily affected by data noise. An iterative clue discovery method and system for social media is proposed. The method and system can comprehensively use the content, structure and interactive information in the social media platform to score the clue subject and improve the accuracy of clue subject identification; at the same time, it can further introduce business users to give feedback on the score of the clue subject, adjust the weight of the multi-dimensional score according to the feedback result, and enrich the initial clue information, thereby improving the accuracy and recall rate of clue subject identification.

[0009] Specifically, the present invention proposes an iterative clue discovery method for social media, which is characterized by comprising:

[0010] Step 1: Collect social media based on initial clue information to obtain social media information;

[0011] Step 2: Classify and sort the social media information according to the account subject to obtain the account information of each account subject;

[0012] Step 3: Score the account subject according to the content, structure, and interaction information of each account information to obtain the content dimension score, structure dimension score, and interaction dimension score of each account subject;

[0013] Step 4: Use the weighted average function to calculate the content dimension score, structure dimension score, and interaction dimension score of each account subject to obtain a comprehensive score for each account subject;

[0014] Step 5: The account subject with the highest comprehensive score is selected as a candidate account, and the weighted average function is adjusted and updated according to the marking result of whether the candidate account is the clue subject;

[0015] Step 6: Determine whether there is a newly added candidate account belonging to the clue subject among all the candidate accounts. If so, add the newly added candidate account belonging to the clue subject to the initial clue information and execute step 1 again; otherwise, all current candidate accounts belonging to the clue subject are used as clue discovery results.

[0016] The iterative clue discovery method for social media is characterized in that the initial clue information includes information dimension information and / or user dimension information;

[0017] The content information includes the text content information of the account subject's posts; the structural information includes the number of fans, the number of followers and the forwarding relationship of the account subject; the interactive information includes: the number of forwardings, comments and likes of each post of the account subject.

[0018] The iterative clue discovery method for social media is characterized in that step 5 includes determining whether the candidate account belongs to the clue subject based on the proportion of negative post content, social influence and activity level of the candidate account.

[0019] The iterative clue discovery method for social media is characterized in that the adjustment and update process of the weighted average function in step 5 includes: based on the degree of correlation between the content dimension score, structure dimension score and interaction dimension score of the candidate account and its corresponding annotation result, the weight of the weighted average function is redistributed and adjusted according to the exponential weight of the criticality to update the weighted average function.

[0020] The present invention also proposes an iterative clue discovery system for social media, which is characterized by comprising:

[0021] Module 1: Collecting social media based on initial clue information to obtain social media information;

[0022] Module 2: classify and sort the social media information according to the account subject to obtain the account information of each account subject;

[0023] Module 3: Score the account subject according to the content, structure, and interaction information of each account information, and obtain the content dimension score, structure dimension score, and interaction dimension score of each account subject;

[0024] Module 4: Use the weighted average function to calculate the content dimension score, structure dimension score and interaction dimension score of each account subject to obtain a comprehensive score for each account subject;

[0025] Module 5: taking the account subject with the highest comprehensive score as a candidate account, and adjusting and updating the weighted average function according to the marking result of whether the candidate account is the clue subject;

[0026] Module 6: Determine whether there is a newly added candidate account belonging to the clue subject among all the candidate accounts. If so, add the newly added candidate account belonging to the clue subject to the initial clue information and execute module 1 again; otherwise, all current candidate accounts belonging to the clue subject are used as clue discovery results.

[0027] The iterative clue discovery system for social media is characterized in that the initial clue information includes information dimension information and / or user dimension information;

[0028] The content information includes the text content information of the account subject's posts; the structural information includes the number of fans, the number of followers and the forwarding relationship of the account subject; the interactive information includes: the number of forwardings, comments and likes of each post of the account subject.

[0029] The iterative clue discovery system for social media is characterized in that module 5 includes determining whether a candidate account belongs to the clue subject based on the proportion of negative post content, social influence and activity level of the candidate account.

[0030] The iterative clue discovery system for social media is characterized in that the module 5 adjusts and updates the weighted average function, including: based on the degree of correlation between the content dimension score, structure dimension score and interaction dimension score of the candidate account and its corresponding annotation result, the weight of the weighted average function is redistributed and adjusted according to the exponential weight of the criticality to update the weighted average function.

[0031] The present invention also proposes a storage medium for storing a program for executing any one of the iterative clue discovery methods for social media.

[0032] The present invention also proposes a client for use in any iterative clue discovery system for social media.

[0033] It can be seen from the above scheme that the advantages of the present invention are:

[0034] 1. The present invention proposes a multi-dimensional scoring method for clue subjects that integrates content, structure and interaction information for social media. This method extracts and integrates important scores from the posting content, social network structure and social platform interaction of the clue subject, effectively avoiding the shortcomings of the existing method of using single-dimensional information and improving the accuracy and comprehensiveness of the clue subject scoring;

[0035] 2. The present invention proposes an iterative clue discovery framework, which effectively alleviates the problem of lack of supervision information faced by existing clue discovery methods by introducing feedback information from business personnel. By effectively utilizing feedback information from business personnel, the rationality of multi-dimensional scoring weights is increased, and the richness of initial clue information is enhanced, thereby improving the efficiency and effectiveness of social media clue discovery.

[0036] 3. The present invention constructs an iterative clue discovery system for social media, covering the full-link business process of clue information collection, single-dimensional score extraction (content, structure, interaction), multi-dimensional score fusion, business personnel feedback, multi-dimensional score weight adjustment and clue information expansion, which can comprehensively and efficiently discover clue entities associated with clue information from social media. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of the iterative clue discovery method for social media of the present invention;

[0038] Figure 2 This is a business logic diagram of the iterative clue discovery system for social media of the present invention. DETAILED DESCRIPTION

[0039] In order to solve the problem that single-dimensional data cannot adequately describe the subject of clues, this paper proposes a clue subject importance scoring method that integrates text, structure and interaction information. By modeling the text content, social network structure and interaction information of the candidate account, the importance scores of the three dimensions of content, structure and interaction are obtained respectively, and then the scores of multiple dimensions are weighted to obtain the final clue subject importance score.

[0040] In response to the problem of lack of effective supervision information in the clue discovery process, the present invention proposes an iterative clue discovery framework. Starting from the initial clue information, the framework collects relevant text, structure and interaction information, and uses a multi-dimensional scoring method to identify potential clue subjects and push them to business users. Business users perform binary annotations on the clue subject list pushed by the algorithm. Specifically, the main basis for business users to annotate is the proportion of negative post content of potential clue subjects, social influence, and their activeness in clue events, combined with business experience. According to the results of business users' annotations, the scoring weights under the current clue information are adjusted to obtain a more accurate clue subject scoring function; at the same time, for the clue subjects confirmed by the user, the keyword information of their content and the associated user information on the network structure are analyzed, and they are added to the clue information as expanded clues to start the next round of iterative analysis.

[0041] In order to achieve the above technical effects, the present invention includes the following key technical points:

[0042] Key point 1: To address the problem that single-dimensional data cannot adequately describe clue subjects, a clue subject importance scoring method that integrates text, structure, and interaction information is proposed.

[0043] Technical effect: This method can significantly improve the comprehensiveness and accuracy of clue subject scoring.

[0044] Key point 2: To address the problem that existing clue discovery methods lack supervised information and are susceptible to data noise, an iterative clue discovery framework is proposed.

[0045] Technical effect: The framework can introduce feedback information from business users in real time to improve the accuracy and recall rate of clue subject identification.

[0046] Key point 3: Build an iterative clue discovery system for social media.

[0047] Technical effect: The entire process of iterative clue discovery for social media is realized, including clue information collection, single-dimensional score extraction (content, structure, interaction), multi-dimensional score fusion, business personnel feedback, multi-dimensional score weight adjustment and clue information expansion, etc. It can comprehensively and efficiently discover clue entities associated with clue information from social media, facilitating the development of subsequent business.

[0048] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0049] An iterative clue discovery method for social media includes the following processing steps:

[0050] 1. Based on the given initial clue information, collect relevant social media information; clue information mainly includes two types of information, one is information dimension information, such as keywords and hashtags on the Twitter platform, and the other is user dimension information, such as social media accounts.

[0051] 2. Classify and sort the collected information according to the account subject information to obtain the information of each account subject;

[0052] 3. Score the account subject based on the content information, structure information, and interaction information of each account subject; content information: mainly the text content information of the account subject's posts; structure information: mainly the number of fans, number of followers, forwarding relationships, and mention relationships of the account subject; interaction information: mainly includes the forwarding volume, comment volume, and like volume data of each post of the account subject.

[0053] 4. Take a weighted average of the multi-dimensional scores of each account subject to obtain a comprehensive score of all account subjects and sort them from highest to lowest;

[0054] 5. Push the top K users with the highest comprehensive scores to business users, who will then label the leads;

[0055] 6. According to the annotation results fed back by business users, the multi-dimensional scoring function is adjusted to obtain a new function. This adjusted new function will be used to calculate the score of potential clues in the next round of iteration;

[0056] 7. Add the lead subject determined by the business user to the initial lead information and jump to step 1; if there is no new lead subject, the lead discovery process ends.

[0057] For specific process, please refer to the attached Figure 1 , described as:

[0058] 1. Data collection: Based on the initial clue keywords or clue account information, collect clue-related data from social media, including but not limited to: user basic information, user friend list, user follow list, user post information, and post reposts, comments, likes, etc.;

[0059] 2. Data collation: The collected data is collated by user to obtain the user’s post content information, social network structure information, and interaction information;

[0060] 3. Score each account entity in a single dimension:

[0061] 1) Content dimension: Conduct sentiment analysis on the account subject’s posts to obtain the positive or negative tendency of each post by the account subject, and count the proportion of posts with negative tendency in the account subject’s posts as the content dimension score of the account subject;

[0062] 2) Structural dimension: Count the number of followers of the account subject, take the natural logarithm of the number of followers of the account subject, and normalize the natural logarithm of the number of followers of all account subjects to the range of 0-100; at the same time, calculate the PageRank score of each account subject based on the social network between account subjects under the same clue and normalize it to the range of 0-100; take the average of the above two values ​​as the score of the account subject structure dimension;

[0063] 3) Interaction dimension: Count the number of reposts, comments and likes of all posts of the account subject and sum them up. Take the natural logarithm of the sum and normalize it to the range of 0-100 as the interaction dimension score of the account subject.

[0064] 4. Take a weighted average of the multi-dimensional scores of the account subject and sort them in descending order. The initial weights are set to 0.5 for content dimension, 0.3 for structure dimension, and 0.2 for interaction dimension.

[0065] 5. The top K account subjects with the highest ranking are handed over to business personnel for labeling. The business personnel give each account subject a binary label, 1 indicates that the subject is an account-related subject, and 0 indicates that the subject is a non-clue subject. The keyword information of the content of the account-related subject and the associated user information on the network structure are analyzed, and they are added to the clue information as extended clues;

[0066] 6. According to the annotations of business personnel, adjust the multi-dimensional weights of the account subject: calculate the correlation between the three dimensions and the annotation results of business users, and redistribute the three dimensions according to the index weights of the criticality;

[0067] 7. Check whether there are any new clue entities in the business user's annotation results. If so, add them to the initial clue list and repeat step 1; if not, the clue discovery process ends.

[0068] An iterative clue discovery system for social media. Its business logic is attached. Figure 2 , including at least the following modules:

[0069] Collection module: mainly responsible for data collection. According to the provided clue information, it downloads the original data from social media and stores it in the collected data, including but not limited to: user basic information, user friend list, user follow list, user post information, and post repost, comment and like information;

[0070] Scoring module: reads the content, structure and interaction information of each account subject from the collected data, and scores the account subject from the dimensions of content, structure, interaction, etc. according to the information, obtains the comprehensive score of the account subject according to the weighted average, and sorts the account subjects according to the comprehensive score. After receiving the annotation feedback from the business users, it is responsible for updating the weighted weights.

[0071] Annotation module: Business users annotate the first K account entities pushed to determine whether they are account entities related to clues. For identified clue entities, they are added to the clue library;

[0072] Line expansion and collection module: For clue subjects identified by business users, if they do not exist in the clue library, they will be added to the initial clue information and the iterative discovery will be restarted.

[0073] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0074] The present invention also proposes an iterative clue discovery system for social media, which is characterized by comprising:

[0075] Module 1: Collecting social media based on initial clue information to obtain social media information;

[0076] Module 2: classify and sort the social media information according to the account subject to obtain the account information of each account subject;

[0077] Module 3: Score the account subject according to the content, structure, and interaction information of each account information, and obtain the content dimension score, structure dimension score, and interaction dimension score of each account subject;

[0078] Module 4: Use the weighted average function to calculate the content dimension score, structure dimension score and interaction dimension score of each account subject to obtain a comprehensive score for each account subject;

[0079] Module 5: taking the account subject with the highest comprehensive score as a candidate account, and adjusting and updating the weighted average function according to the marking result of whether the candidate account is the clue subject;

[0080] Module 6: Determine whether there is a newly added candidate account belonging to the clue subject among all the candidate accounts. If so, add the newly added candidate account belonging to the clue subject to the initial clue information and execute module 1 again; otherwise, all current candidate accounts belonging to the clue subject are used as clue discovery results.

[0081] The iterative clue discovery system for social media is characterized in that the initial clue information includes information dimension information and / or user dimension information;

[0082] The content information includes the text content information of the account subject's posts; the structural information includes the number of fans, the number of followers and the forwarding relationship of the account subject; the interactive information includes: the number of forwardings, comments and likes of each post of the account subject.

[0083] The iterative clue discovery system for social media is characterized in that module 5 includes determining whether a candidate account belongs to the clue subject based on the proportion of negative post content, social influence and activity level of the candidate account.

[0084] The iterative clue discovery system for social media is characterized in that the module 5 adjusts and updates the weighted average function, including: based on the degree of correlation between the content dimension score, structure dimension score and interaction dimension score of the candidate account and its corresponding annotation result, the weight of the weighted average function is redistributed and adjusted according to the exponential weight of the criticality to update the weighted average function.

[0085] The present invention also proposes a storage medium for storing a program for executing any one of the iterative clue discovery methods for social media.

[0086] The present invention also proposes a client for use in any iterative clue discovery system for social media.

Claims

1. An iterative clue discovery method for social media, characterized in that: include: Step 1: Collect social media based on initial clue information to obtain social media information; Step 2: Classify and sort the social media information according to the account subject to obtain the account information of each account subject; Step 3: Score the account subject according to the content, structure, and interaction information of each account information to obtain the content dimension score, structure dimension score, and interaction dimension score of each account subject; Step 4: Use the weighted average function to calculate the content dimension score, structure dimension score, and interaction dimension score of each account subject to obtain a comprehensive score for each account subject; Step 5: The account subject with the highest comprehensive score is selected as a candidate account, and the weighted average function is adjusted and updated according to the marking result of whether the candidate account is the clue subject; Step 6: determine whether there is a newly added candidate account belonging to the clue subject among all the candidate accounts. If so, add the newly added candidate account belonging to the clue subject to the initial clue information, and execute step 1 again; Otherwise, all candidate accounts currently belonging to the clue subject will be taken as clue discovery results.

2. The iterative clue discovery method for social media as claimed in claim 1, characterized in that: The initial clue information includes information dimension information and / or user dimension information; The content information includes the posting text content information of the account subject; Structural information includes the number of followers, followings, and forwarding relationships of the account subject; Interaction information includes: the number of reposts, comments, and likes for each post of the account owner.

3. The iterative clue discovery method for social media according to claim 1, characterized in that: This step 5 includes determining whether the candidate account belongs to the clue subject based on the proportion of negative post content, social influence and activity level of the candidate account.

4. The iterative clue discovery method for social media as claimed in claim 1, characterized in that: The process of adjusting and updating the weighted average function in step 5 includes: redistributing and adjusting the weights of the weighted average function according to the exponential weights of the criticality based on the degree of correlation between the content dimension scores, structure dimension scores and interaction dimension scores of the candidate accounts and their corresponding annotation results, so as to update the weighted average function.

5. An iterative clue discovery system for social media, characterized in that: include: Module 1: Collecting social media based on initial clue information to obtain social media information; Module 2: classify and sort the social media information according to the account subject to obtain the account information of each account subject; Module 3: Score the account subject according to the content, structure, and interaction information of each account information, and obtain the content dimension score, structure dimension score, and interaction dimension score of each account subject; Module 4: Use the weighted average function to calculate the content dimension score, structure dimension score and interaction dimension score of each account subject to obtain a comprehensive score for each account subject; Module 5: taking the account subject with the highest comprehensive score as a candidate account, and adjusting and updating the weighted average function according to the marking result of whether the candidate account is the clue subject; Module 6: determine whether there is a newly added candidate account belonging to the clue subject among all the candidate accounts. If so, add the newly added candidate account belonging to the clue subject to the initial clue information, and execute module 1 again; Otherwise, all candidate accounts currently belonging to the clue subject will be taken as clue discovery results.

6. The iterative clue discovery system for social media as claimed in claim 1, characterized in that: The initial clue information includes information dimension information and / or user dimension information; The content information includes the posting text content information of the account subject; Structural information includes the number of followers, followings, and forwarding relationships of the account subject; Interaction information includes: the number of reposts, comments, and likes for each post of the account owner.

7. The iterative clue discovery system for social media as claimed in claim 1, characterized in that: This module 5 includes determining whether the candidate account belongs to the clue subject based on the proportion of negative post content, social influence and activity level of the candidate account.

8. The iterative clue discovery system for social media as claimed in claim 1, characterized in that: The process of adjusting and updating the weighted average function in module 5 includes: based on the degree of correlation between the content dimension score, structure dimension score and interaction dimension score of the candidate account and its corresponding annotation results, the weight of the weighted average function is redistributed and adjusted according to the exponential weight of the criticality to update the weighted average function.

9. A storage medium for storing a program for executing any one of the iterative clue discovery methods for social media as claimed in claims 1 to 4.

10. A client, used in any one of claims 5 to 8 for the iterative clue discovery system for social media.

Images