A brand public opinion monitoring method and system based on Internet events

Abstract

The present invention discloses a brand public opinion monitoring method and system based on Internet events. The system includes a data acquisition module, a sentiment analysis module, a social network relationship analysis module, a geographic location analysis module, a monitoring module and a public opinion orientation prediction module. The data acquisition module includes a data acquisition unit, a data cleaning unit and a data storage unit. The beneficial effects of the present invention are as follows: while performing sentiment analysis on text data, geographic location information and user relationship network are combined to achieve a comprehensive multi-dimensional and multi-angle analysis of public opinion, thereby improving the depth and breadth of public opinion analysis, collecting data from multiple social network platforms in real time, covering a wider range, and improving the comprehensiveness and objectivity of public opinion monitoring. By establishing a real-time monitoring system and a prediction model, public opinion data can be automatically collected, processed and analyzed, and the parameters of the prediction model can be dynamically adjusted according to the monitoring results, thereby improving the degree of automation and intelligence of public opinion monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet information analysis and processing, and in particular to a brand public opinion monitoring method and system based on Internet events. Background Art

[0002] In today's digital age, the rise of the Internet has made social networking platforms one of the main channels for people to obtain information, express opinions and interact. With the development of the Internet, brand public opinion monitoring has become increasingly important in corporate and organizational management. For brand owners, understanding and grasping the attitudes and opinions of public opinion on their brands can help them discover and resolve potential crises in a timely manner. At the same time, they can also use positive public opinion to promote and publicize their brands. Therefore, brand public opinion monitoring methods based on Internet events have become one of the research hotspots in the current technology field.

[0003] Traditional public opinion monitoring methods often only focus on the analysis of text data, lack the ability to comprehensively analyze multi-source data, and the depth and breadth of public opinion analysis are insufficient. Public opinion monitoring lacks real-time performance, accuracy, comprehensive analysis capabilities, intelligence, and comprehensiveness. Therefore, a brand public opinion monitoring method and system based on Internet events is proposed. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] The present invention has been proposed in view of the above-mentioned problems.

[0006] To solve the above technical problems, the present invention provides the following technical solution: a brand public opinion monitoring method based on Internet events, comprising the following steps:

[0007] Step 1: Use web crawlers and APIs to collect real-time text, images, videos, and geographic location information posted by users on major social networking platforms related to the target event, clean the data, and remove noise data;

[0008] Step 2: Use NLP algorithms to perform sentiment analysis on text data, classify the sentiment polarity into positive, negative, and neutral, and analyze facial expressions and voice intonation to assist in the text sentiment analysis results;

[0009] Step 3: Based on social network data, build a relationship graph between users, identify the connection relationships between each user, analyze the user relationship graph, and use graph algorithms to identify opinion leaders and key communication nodes;

[0010] Step 4: Integrate the user's geographic location information with the sentiment analysis results, mark the sentiment polarity of users at each location, and use geographic information system tools to conduct statistical analysis of positive and negative sentiment in different regions to identify the public opinion orientation in specific areas.

[0011] Step 5: Receive, process, and analyze new data in real time, dynamically track the changing trends of positive and negative emotions, and identify key moments of emotional change;

[0012] Step six: Combine the relationship diagram and geographic location data to analyze the transmission paths of positive and negative emotions, identify the key nodes and areas of emotional transmission, build a public opinion orientation prediction model, comprehensively consider the influence of positive and negative emotions, calculate the public opinion orientation index, generate a regional public opinion orientation heat map, predict public opinion orientation, train the model based on historical data and real-time monitoring data, and dynamically adjust the parameters of the prediction model based on the feedback of real-time monitoring results.

[0013] As a preferred solution of the brand public opinion monitoring method based on Internet events described in the present invention, the public opinion guidance model algorithm is:

[0014]

[0015] When OI(t)>1, it indicates that the influence of positive emotions is dominant and the overall public opinion is positive. When OI(t=1), it indicates that the influence of positive and negative emotions is equal and the public opinion is neutral. When OI(t)<1, it indicates that the influence of negative emotions is dominant and the overall public opinion is negative.

[0016] Among them, N and M represent the number of positive emotion users and negative emotion users respectively, E i (t) represents the emotional intensity of the i-th positive emotional user at time t, F i (t) represents the influence weight of the i-th positive sentiment user, L i (t) represents the geographical location weight of the i-th positive emotion user, W i (t) represents the interaction frequency of the i-th positive emotion user, G i (t) represents the total interactive influence index of the i-th positive emotion user at time t, H j (t) represents the emotional intensity of the i-th negative emotional user at time t, Φ j (t) represents the influence weight of the i-th negative sentiment user, K j(t) represents the geographical location weight of the i-th negative sentiment user, Γ j (t) represents the interaction frequency of the i-th negative sentiment user, is the sentiment diffusion function.

[0017] As a preferred solution of the brand public opinion monitoring method based on Internet events described in the present invention, the calculation algorithm of the total interactive influence index is:

[0018]

[0019] Among them, f k (E i ,F i ,L i ) represents the influence index calculation in the kth interaction mode, α k Represents the weight of the kth interaction mode in the total interaction influence coefficient.

[0020] As a preferred solution of the brand public opinion monitoring method based on Internet events described in the present invention, f k The calculation steps are:

[0021] S1, setting different interaction modes k, including direct interaction, indirect interaction and emotional communication;

[0022] S2, perform sentiment analysis on the content posted by users through the NIP model to obtain the sentiment intensity E i ;

[0023] S3, the influence weight F is calculated through the user relationship graph and graph algorithm i ;

[0024] S4, calculate the geographic location weight L through GIS tools i ;

[0025] S5, design corresponding weight functions according to different interaction modes k.

[0026] As a preferred solution of the brand public opinion monitoring method based on Internet events described in the present invention, S4 specifically includes the following steps:

[0027] S41, importing the geographical location information of the user when publishing the content into the GIS tool;

[0028] S42, dividing the geographic location data into different regions;

[0029] S43, based on the results of the previous sentiment analysis, the sentiment polarity of each user is associated with its geographic location information, and a new layer is created in the GIS tool to store the geographic location information marked with the sentiment polarity;

[0030] S44, the spatial analysis function of the practical GIS tool, counts the distribution of positive, negative and neutral emotions in each area;

[0031] S45, according to the statistical results, assign a weight L to each user's geographical location i To reflect the influence of public opinion in the region, L i The specific calculation method is:

[0032]

[0033] Among them, P pos (r i ), P neg (r i ) and P neu (r i ) represent the region r where user i is located i The proportion of users with positive, negative and neutral emotions.

[0034] As a preferred solution of the brand public opinion monitoring method based on Internet events described in the present invention, the direct interaction in S1 includes comments, replies and questions, the indirect interaction includes likes and reposts, and the emotional communication includes emotional expression, emotional guidance and emotional diffusion.

[0035] As a preferred solution of the brand public opinion monitoring method based on Internet events described in the present invention, the weight function in S5 includes a direct interaction mode function, an indirect interaction mode function and an emotion propagation mode function;

[0036] The direct interaction mode function is as follows:

[0037] f1(E i ,F i ,L i )=α1E i +β1F i +γ1L i

[0038] The indirect interaction mode function is as follows:

[0039]

[0040] The specific function of the emotional propagation model is:

[0041]

[0042] Among them, α, β and γ represent the emotional intensity E i , influence weight F i and geographic location weight L iThe weight coefficient in the calculation.

[0043] As a preferred solution of the brand public opinion monitoring method based on Internet events described in the present invention, the noise data specifically includes irrelevant content, repeated content and advertisements.

[0044] Based on the above-mentioned brand public opinion monitoring method based on Internet events, the present invention further provides a brand public opinion monitoring system based on Internet events, comprising:

[0045] Data collection module, used to collect and process data from social networks in real time;

[0046] Sentiment analysis module, used to perform sentiment analysis on the collected data;

[0047] Social network relationship analysis module, used to build user relationship graphs and identify opinion leaders and key communication nodes;

[0048] The geographic location analysis module is used to integrate user geographic location information with sentiment analysis results and conduct regional public opinion analysis;

[0049] The monitoring module is used to receive and process new data, dynamically track sentiment trends, and identify key time points;

[0050] The public opinion orientation prediction module is used to combine user relationship graphs and geographic location data to analyze the emotional transmission path, build and dynamically adjust the public opinion orientation prediction model.

[0051] As a preferred solution of the brand public opinion monitoring system based on Internet events described in the present invention, the data collection module includes:

[0052] A data collection unit, used to collect text, pictures, videos and geographic location information related to the target event from the social networking platform in real time;

[0053] Data cleaning unit, used to remove irrelevant content, duplicate content and advertisements from the data;

[0054] The data storage unit is used to store the cleaned data.

[0055] Beneficial effects of the present invention:

[0056] 1. The present invention combines geographic location information and user relationship networks while performing sentiment analysis on text data, thereby achieving a comprehensive multi-dimensional and multi-angle analysis of public opinion, and improving the depth and breadth of public opinion analysis.

[0057] 2. Real-time data collection from multiple social networking platforms provides wider coverage, more comprehensive and richer information, and improves the comprehensiveness and objectivity of public opinion monitoring.

[0058] 3. By establishing a real-time monitoring system and prediction model, it is possible to automatically collect, process and analyze public opinion data, and dynamically adjust the parameters of the prediction model based on the monitoring results, thereby improving the degree of automation and intelligence of public opinion monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0060] Figure 1 This is a system structure diagram of a brand public opinion monitoring method and system based on Internet events of the present invention.

[0061] Figure 2 This is a flow chart of a method for monitoring brand public opinion based on Internet events according to the present invention.

[0062] Figure 3 This is a regional public opinion orientation heat map of a brand public opinion monitoring system based on Internet events in the present invention. DETAILED DESCRIPTION

[0063] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0064] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0065] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0066] Furthermore, the present invention is described in detail with reference to schematic diagrams. For ease of illustration, when describing the embodiments of the present invention, cross-sectional views illustrating device structures may be partially enlarged and not to scale. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of protection of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0067] Example 1

[0068] A method for monitoring brand public opinion based on Internet events, comprising the following steps:

[0069] Step 1: Use web crawlers and APIs to collect real-time user-posted text, images, videos, and location information related to the target event from major social networking platforms. Clean the data and remove noise, which includes irrelevant content, duplicate content, and advertisements.

[0070] Step 2: Use NLP algorithms to perform sentiment analysis on text data, classify the sentiment polarity into positive, negative, and neutral, and analyze facial expressions and voice intonation to assist in the text sentiment analysis results;

[0071] Step 3: Based on social network data, build a relationship graph between users, identify the connection relationships between each user, analyze the user relationship graph, and use graph algorithms to identify opinion leaders and key communication nodes;

[0072] Step 4: Integrate the user's geographic location information with the sentiment analysis results, mark the sentiment polarity of users at each location, and use geographic information system tools to conduct statistical analysis of positive and negative sentiment in different regions to identify the public opinion orientation in specific areas.

[0073] Step 5: Receive, process, and analyze new data in real time, dynamically track the changing trends of positive and negative emotions, and identify key moments of emotional change;

[0074] Step six: Combine the relationship diagram and geographic location data to analyze the transmission paths of positive and negative emotions, identify the key nodes and areas of emotional transmission, build a public opinion orientation prediction model, comprehensively consider the influence of positive and negative emotions, calculate the public opinion orientation index, generate a regional public opinion orientation heat map, visualize the public opinion orientation results, predict public opinion orientation, train the model based on historical data and real-time monitoring data, and dynamically adjust the parameters of the prediction model based on the feedback of real-time monitoring results.

[0075] Specifically, the algorithm of the public opinion guidance model is:

[0076]

[0077] When OI(t)>1, it indicates that the influence of positive emotions is dominant and the overall public opinion is positive. When OI(t=1), it indicates that the influence of positive and negative emotions is equal and the public opinion is neutral. When OI(t)<1, it indicates that the influence of negative emotions is dominant and the overall public opinion is negative.

[0078] Among them, N and M represent the number of positive emotion users and negative emotion users respectively, Ei (t) represents the emotional intensity of the i-th positive emotional user at time t, F i (t) represents the influence weight of the i-th positive sentiment user, L i (t) represents the geographical location weight of the i-th positive emotion user, W i (t) represents the interaction frequency of the i-th positive emotion user, G i (t) represents the total interactive influence index of the i-th positive emotion user at time t, H j (t) represents the emotional intensity of the i-th negative emotional user at time t, Φ j (t) represents the influence weight of the i-th negative sentiment user, K j (t) represents the geographical location weight of the i-th negative sentiment user, Γ j (t) represents the interaction frequency of the i-th negative sentiment user, is the sentiment diffusion function.

[0079] Specifically, the calculation algorithm of the total interactive impact index is:

[0080]

[0081] Among them, f k (E i ,F i ,L i ) represents the influence index calculation in the kth interaction mode, α k Represents the weight of the kth interaction mode in the total interaction influence coefficient.

[0082] Specifically, k The calculation steps are:

[0083] S1, set different interaction modes k, including direct interaction, indirect interaction and emotional communication. Direct interaction includes comments, replies and questions, indirect interaction includes likes and reposts, and emotional communication includes emotional expression, emotional guidance and emotional diffusion;

[0084] S2, perform sentiment analysis on the content posted by users through the NIP model to obtain the sentiment intensity E i ;

[0085] S3, calculates the influence weight F through the user relationship graph and graph algorithms, such as PageRank i ;

[0086] S4, calculate the geographic location weight L through GIS tools i ;

[0087] S5, design corresponding weight functions according to different interaction modes k.

[0088] Among them, S4 specifically includes the following steps:

[0089] S41, importing the geographical location information of the user when publishing the content into the GIS tool;

[0090] S42, dividing the geographic location data into different regions;

[0091] S43, based on the results of the previous sentiment analysis, the sentiment polarity of each user is associated with its geographic location information, and a new layer is created in the GIS tool to store the geographic location information marked with the sentiment polarity;

[0092] S44, the spatial analysis function of the practical GIS tool, counts the distribution of positive, negative and neutral emotions in each area;

[0093] S45, according to the statistical results, assign a weight L to each user's geographical location i To reflect the influence of public opinion in the region, L i The specific calculation method is:

[0094]

[0095] Among them, P pos (r i ), P neg (r i ) and P neu (r i ) represent the region r where user i is located i The proportion of users with positive, negative and neutral emotions,

[0096] It should be noted that the weight functions in S5 include direct interaction mode function, indirect interaction mode function and emotion communication mode function;

[0097] The direct interaction mode function is as follows:

[0098] f1(E i ,F i ,L i )=α1E i +β1F i +γ1L i

[0099] The indirect interaction mode function is as follows:

[0100]

[0101] The specific function of the emotional propagation model is:

[0102]

[0103] Among them, α, β and γ represent the emotional intensity E i , influence weight F i and geographic location weight L i The weight coefficient in the calculation.

[0104] For example:

[0105] Assume there are three regions A, B, and C. The sentiment distribution and user relationships of each region are as follows:

[0106] Region A: 100 users with positive emotions, 50 users with negative emotions, 30 users with neutral emotions, 5 opinion leaders, and 8 key communication nodes;

[0107] Region B: 80 users with positive emotions, 40 users with negative emotions, 20 users with neutral emotions, 3 opinion leaders, and 6 key communication nodes;

[0108] Region C: 120 users with positive emotions, 60 users with negative emotions, 40 users with neutral emotions, 7 opinion leaders, and 9 key communication nodes;

[0109] The weight coefficients α = 0.5, β = 0.3, and γ = 0.2. The number of opinion leaders is used as the influence weight. A balanced calculation is used, and the weight of each region is 1.

[0110] For area A, the emotional intensity E can be calculated according to the formula A ≈0.278, influence weight F A =5, geographic location weight L A =1, and the public opinion orientation index OI can be obtained by substituting it into the model. A ≈1.839;

[0111] For region B, the emotional intensity E can be calculated according to the formula B ≈0.286, influence weight F B =3, geographic location weight L B =1, and the public opinion orientation index OI can be obtained by substituting it into the model. B ≈1.243;

[0112] For area A, the emotional intensity E can be calculated according to the formula C ≈0.273, influence weight F C =7, geographic location weight L C =1, and the public opinion orientation index OI can be obtained by substituting it into the model. C ≈2.437;

[0113] Through the above calculations, we can see that the public opinion orientation indexes of the three regions A, B, and C are all greater than 1, and the public opinion orientation of the three regions is all positive. The regional public opinion orientation heat map generated based on the public opinion orientation indexes of regions A, B, and C is as follows: Figure 3 .

[0114] Example 2

[0115] A brand public opinion monitoring system based on Internet events, comprising:

[0116] Data collection module, used to collect and process data from social networks in real time;

[0117] Sentiment analysis module, used to perform sentiment analysis on the collected data;

[0118] Social network relationship analysis module, used to build user relationship graphs and identify opinion leaders and key communication nodes;

[0119] The geographic location analysis module is used to integrate user geographic location information with sentiment analysis results and conduct regional public opinion analysis;

[0120] The monitoring module is used to receive and process new data, dynamically track sentiment trends, and identify key time points;

[0121] The public opinion orientation prediction module is used to combine user relationship graphs and geographic location data to analyze the emotional transmission path, build and dynamically adjust the public opinion orientation prediction model, and generate a public opinion orientation heat map.

[0122] The data acquisition module includes:

[0123] A data collection unit, used to collect text, pictures, videos and geographic location information related to the target event from the social networking platform in real time;

[0124] Data cleaning unit, used to remove irrelevant content, duplicate content and advertisements from the data;

[0125] The data storage unit is used to store the cleaned data.

[0126] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A brand public opinion monitoring method based on Internet events, characterized in that: The following steps are involved: Step 1: Use web crawlers and APIs to collect real-time text, images, videos, and geographic location information posted by users on major social networking platforms related to the target event, clean the data, and remove noise data; Step 2: Use NLP algorithms to perform sentiment analysis on text data, classify the sentiment polarity into positive, negative, and neutral, and analyze facial expressions and voice intonation to assist in the text sentiment analysis results; Step 3: Based on social network data, build a relationship graph between users, identify the connection relationships between each user, analyze the user relationship graph, and use graph algorithms to identify opinion leaders and key communication nodes; Step 4: Integrate the user's geographic location information with the sentiment analysis results, mark the sentiment polarity of users at each location, and use geographic information system tools to conduct statistical analysis of positive and negative sentiment in different regions to identify the public opinion orientation in specific areas. Step 5: Receive, process, and analyze new data in real time, dynamically track the changing trends of positive and negative emotions, and identify key moments of emotional change; Step 6: Combine the relationship graph and geographic location data to analyze the transmission paths of positive and negative emotions, identify key nodes and regions for emotional transmission, and construct a public opinion orientation prediction model. Taking into account the influence of positive and negative emotions, the public opinion orientation index is calculated, and a regional public opinion orientation heat map is generated to predict public opinion orientation. The model is trained based on historical data and real-time monitoring data, and the parameters of the prediction model are dynamically adjusted based on feedback from real-time monitoring results. The public opinion guidance model can be reflected by the following algorithm: when , indicating that the influence of positive emotions is dominant and the overall public opinion is positive. , indicating that positive and negative emotions have equal influence, and public opinion is neutral. , indicating that the influence of negative emotions is dominant and the overall public opinion is negative; in, and Represents the number of positive emotion users and negative emotion users, Indicates the The emotional intensity of a positive emotional user at time t, Indicates the The influence weight of positive emotional users, Indicates the The geographical location weight of positive emotion users, Indicates the The interaction frequency of positive emotional users, Indicates the The total interactive influence index of positive emotion users at time t, Indicates the The emotional intensity of a negative emotional user at time t, Indicates the The influence weight of negative emotion users, Indicates the The geographical location weight of negative sentiment users, Indicates the The interaction frequency of negative emotional users, is the sentiment diffusion function.

2. The method for monitoring brand public opinion based on Internet events according to claim 1, characterized in that: The calculation algorithm of the total interactive impact index is: in, Representative The impact index calculation in the interactive mode is Indicates the The weight of the interaction mode in the total interaction influence coefficient.

3. The method for monitoring brand public opinion based on Internet events according to claim 2, characterized in that: The calculation steps are: S1, set different interaction modes , including direct interaction, indirect interaction and emotional communication; S2, use the NIP model to perform sentiment analysis on the content posted by users and obtain the sentiment intensity ; S3, influence weight calculated through user relationship graph and graph algorithm ; S4, calculate geographic location weights using GIS tools ; S5, according to different interaction modes , design the corresponding weight function.

4. The method for monitoring brand public opinion based on Internet events according to claim 3, characterized in that: S4 specifically includes the following steps: S41, importing the geographical location information of the user when publishing the content into the GIS tool; S42, dividing the geographic location data into different regions; S43, based on the results of the previous sentiment analysis, the sentiment polarity of each user is associated with its geographic location information, and a new layer is created in the GIS tool to store the geographic location information marked with the sentiment polarity; S44, the spatial analysis function of the practical GIS tool, counts the distribution of positive, negative and neutral emotions in each area; S45, assign weights to each user's geographical location based on statistical results To reflect the influence of public opinion in the region, The specific calculation method is: in, 、 and Represents users Area The proportion of users with positive, negative and neutral emotions.

5. The method for monitoring brand public opinion based on Internet events according to claim 4, characterized in that: The direct interactions in S1 include comments, replies and questions, the indirect interactions include likes and reposts, and the emotional communication includes emotional expression, emotional guidance and emotional diffusion.

6. The method for monitoring brand public opinion based on Internet events according to claim 5, characterized in that: The weight functions in S5 include direct interaction mode function, indirect interaction mode function and emotion transmission mode function; The direct interaction mode function is as follows: The indirect interaction mode function is as follows: The specific function of the emotional propagation model is: in, 、 and Represents emotional intensity , influence weight and geographic location weight The weight coefficient in the calculation.

7. The method for monitoring brand public opinion based on Internet events according to claim 1, characterized in that: The noise data specifically includes irrelevant content, repeated content and advertisements.

8. A brand public opinion monitoring system based on Internet events, based on the brand public opinion monitoring method based on Internet events according to any one of claims 1 to 7, characterized in that: include: Data collection module, used to collect and process data from social networks in real time; Sentiment analysis module, used to perform sentiment analysis on the collected data; Social network relationship analysis module, used to build user relationship graphs and identify opinion leaders and key communication nodes; The geographic location analysis module is used to integrate user geographic location information with sentiment analysis results and conduct regional public opinion analysis; The monitoring module is used to receive and process new data, dynamically track sentiment trends, and identify key time points; The public opinion orientation prediction module is used to combine user relationship graphs and geographic location data to analyze the emotional transmission path, build and dynamically adjust the public opinion orientation prediction model, and generate a public opinion orientation heat map.

9. The brand public opinion monitoring system based on Internet events according to claim 8, characterized in that: The data acquisition module includes: A data collection unit, used to collect text, pictures, videos and geographic location information related to the target event from the social networking platform in real time; Data cleaning unit, used to remove irrelevant content, duplicate content and advertisements from the data; The data storage unit is used to store the cleaned data.

Images