Short video monitoring method based on big data

Abstract

The invention discloses a short video monitoring method based on big data. The method comprises the following steps: 1) establishing a public opinion dynamic platform; 2) extracting information in the short video; classifying the short video information; preliminarily judging whether the information can be matched with the information in the public opinion dynamic platform or not, and if so, performing a step 7); otherwise, step 3); 3) performing propagation foreground prediction on the short video information; if the predicted value is high, a step 7) is carried out, otherwise, a step 4) is carried out; 4) sorting according to the propagation foreground prediction, and sending to the step 5) according to the sequence; 5) performing the step 6) according to the classification of the short videos; 6) comparing the information in the short video with the information in the public opinion dynamic platform and the information in the sensitive information database; by means of the method, real-time monitoring of the short video can be carried out, the monitoring speed is high, the monitoring efficiency is high, development of public opinions can be controlled to a great extent, and large-area uncontrollable public opinion dynamic conditions are avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of short video monitoring, and in particular relates to a short video monitoring method based on big data. Background Art

[0002] Short video is a short video clip, which is usually less than five minutes long. Currently, short videos are widely circulated and have a large number of users. Monitoring short videos is crucial because once some bad remarks appear on a short video, the speed of their spread is unimaginable. They will be watched and spread by users in a short period of time. Therefore, good short video monitoring is of great significance to social stability.

[0003] The existing short video monitoring method is to extract multivariate data from the video and compare it with the values ​​in the sensitive information database. When the comparison is successful, a restriction label is attached to the short video, and the short video will be automatically removed from the shelves, thus losing the possibility of being viewed and spread, thereby effectively controlling public opinion.

[0004] For example, the patent application with application publication number CN118585688A discloses a big data processing method and system based on classification algorithm, including: collecting data from the target video platform to obtain video content and related metadata; extracting multimodal features of the video content; using the public opinion classification model to classify the multimodal features; based on the public opinion classification, predicting the public opinion trend under different categories, and displaying the public opinion analysis results. The present invention can obtain massive public opinion data of short video platforms in real time, discover valuable information from it, and provide a variety of data services such as public opinion monitoring, risk warning, hot spot discovery, and traceability analysis.

[0005] However, although this monitoring method can limit public opinion to a certain extent, it requires frame-by-frame analysis of data, which is slow and may even result in data analysis delays. Especially when there is a major public opinion, the above method cannot quickly distinguish public opinion information and has a slow response speed.

[0006] Therefore, there is an urgent need for a short video monitoring method that can quickly monitor short videos. Summary of the invention

[0007] The present invention aims to provide a short video monitoring method based on big data which is easy to use and has good use effect.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions: A short video monitoring method based on big data, the method comprising the following steps: 1) Establish a public opinion dynamic platform; 2) Extract the information in the short video; classify the short video information; and preliminarily determine whether it can be matched with the information in the public opinion dynamic platform. If so, proceed directly to step 6); otherwise, proceed to step 4); 3) Predict the dissemination prospects of short video information; if the predicted value is high, proceed directly to step 6), otherwise proceed to step 4); 4) Sort by propagation prospect prediction and send to step 5) in the order of sorting; 5) According to the classification of the short video, the short video information is sent to step 6) and step 7 in sequence); 6) Compare the information in the short video with the information in the public opinion dynamic platform; 7) Compare the information in the short video with the information in the sensitive information database; 8) After the comparison is successful, a warning message is generated and pushed to the warning platform; otherwise, the short video is marked with a pass label.

[0009] The information in the short video includes metadata in the image information and metadata in the sound information.

[0010] The information in the sensitive information database is stored in the following manner: the information in the sensitive information database is class information and keywords contained in the class information.

[0011] The categories of information include daily life, military, academic, entertainment, sports, and education.

[0012] The public opinion dynamics platform collects recent public opinion information and transmits the collected public opinion information to the public opinion dynamics platform in real time.

[0013] The method to predict the dissemination prospects of short videos is: calculate the dissemination volume per unit time S=△L / △t.

[0014] If the transmission volume is less than 200, it is a low-risk value; if the transmission volume is greater than or equal to 600, it is considered a high-risk value; When performing step 6), it is collected in real time to see whether there are short videos with a high-risk transmission volume. If so, it is directly sent to step 7).

[0015] Generate a specific user database, compare the user who sent the short video with the specific user database according to the generated specific user database, and if the match is successful, proceed directly to step 7); the method for generating the specific user database is: extract the specific user feature value, generate a clustering model based on the feature variables of each user; merge the clustering model with the preset specific user features to generate a specific user database.

[0016] The information in the public opinion dynamics platform is cleansed once a week, and the cleaned data is removed; the remaining data is transferred to the sensitive information database.

[0017] Through the above technical scheme, the technical effects of the present invention are as follows: 1. Through this method, short videos can be monitored in real time, with fast monitoring speed and high efficiency, which can control the development of public opinion to a great extent, purify the network environment, and avoid the occurrence of large-scale uncontrollable public opinion dynamics; 2. By generating a specific user database, user characteristics can be quickly compared with the specific user database to generate key monitoring objects, and short videos sent by specific users are paid special attention to improve monitoring efficiency and accuracy; 3. A public opinion dynamic monitoring platform and a sensitive information database are set up, and monitoring information is generated by comparing the metadata of short videos with the above two platforms and databases; 4. The process of use is summarized, the data on the public opinion dynamic monitoring platform will scroll, and the cleaned data will be automatically screened out after one week, and the remaining data will be pushed to the sensitive information database, thereby improving the accuracy of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings constituting the present invention are used to provide a further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute improper limitations on the present invention.

[0019] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0020] In order to make the technical means, objectives and effects of the present invention easy to understand, the embodiments of the present invention are described in detail below with reference to specific drawings.

[0021] Embodiment 1, a short video monitoring method based on big data, which is used for monitoring short videos, purifies the network environment and effectively controls the development of public opinion by monitoring short videos.

[0022] In this embodiment, "big data" is a technical term, and the "big" therein does not imply size.

[0023] Before implementing this method, it is necessary to divide the area and place a server in one area. The area division method is: classify by size. If there are no special circumstances, place one server in a community and one server in a school. Resources are shared between servers.

[0024] The placement of servers and the division of regions are mature existing technologies and will not be described in detail in this embodiment.

[0025] The short video monitoring method based on big data described in the present invention is as follows: Figure 1As shown, the method comprises the following steps in sequence: 1) Establish a public opinion dynamic platform; the data in the public opinion dynamic platform described in this embodiment is dynamic and flowing, and the current hot information is captured in real time, and the keywords in the hot information are extracted and sent to the public opinion dynamic platform. The public opinion dynamic platform here is shared, and each server can obtain data from the public opinion dynamic platform.

[0026] In order to improve the data response speed, the data in the public opinion dynamic platform is only within one week. Data older than one week will be cleaned. Through data cleaning, the normalized data will be cleaned out, and the cleaned data will be filtered out. The remaining data will be sent to the sensitive information database for comparison as normalized sensitive information. The method of data cleaning is to compare the data with the sensitive information label. If the comparison is successful, the data will be retained. If the comparison is not successful, the data will be cleaned.

[0027] Among them, sensitive information labels are regularly added to the sensitive information database, and big data are regularly pushed. Pushing sensitive information labels is a mature existing technology and will not be repeated in this embodiment.

[0028] 2) Extract information from short videos; The extracted short video information includes metadata in the image information and metadata in the sound information; the metadata extraction method in the image information is: analyze each frame of video, convert the information in each frame of video into data volume and save it as metadata; wherein, when analyzing each frame of video information, if the data volume of two frames of video information is repeated, the repeated data is automatically deleted.

[0029] The metadata extraction method in the sound information is: divide the sound information into sound segments, convert the information in each sound segment into data volume, and save it as metadata; when analyzing each sound segment, if the data volume of two sound segments is repeated, the repeated data is automatically deleted.

[0030] Next, the short video information is classified according to the extracted metadata; the classification method is: according to the amount of data appearing in the metadata, it is divided into categories of information; the types of category information include: daily life, military, entertainment, sports, and education.

[0031] For example, if sports words appear more frequently in the extracted metadata, they will be classified into the category information type of sports.

[0032] If the entertainment word appears more often, it is classified into the category of entertainment information type.

[0033] By classifying short video information, each short video is divided into different categories of information, and then compared separately, the accuracy of short video monitoring is improved, and the efficiency of short video monitoring is also improved. The above categories are just examples. In the actual operation process, specific categories can be divided according to specific circumstances, and the divided categories can be put into the sensitive information database.

[0034] The sensitive information database is stored as follows: Class information types: life (celebrity names, swear words, etc.), entertainment (names of celebrities with bad records, etc.), etc. Among them, life and entertainment belong to the class information type, and the information in brackets is specific class information. One class information type may contain multiple class information.

[0035] In this step, when extracting short video information, it is necessary to preliminarily match the extracted metadata with the information in the public opinion dynamic platform. If so, proceed directly to step 7); otherwise, proceed to step 3); through this step, the current public opinion can be quickly monitored. When the data in the short video matches the current public opinion, the short video will be sent to the monitoring point as soon as possible, instead of making subsequent judgments, which improves the speed of public opinion monitoring.

[0036] 3) Predict the dissemination prospects of the received short video; if the predicted value is high, proceed directly to step 7), otherwise proceed to step 5); Among them, the method for predicting the dissemination prospects of short videos is: calculate the dissemination volume per unit time S=△L / △t.

[0037] If the spread is less than 200, it is a low risk value; if the spread is greater than or equal to 600, it is judged as a high risk value; if a high risk value is encountered, go directly to step 6) and no longer sort, which improves the speed of public opinion monitoring. Among them, △L is the difference between the video spread in the next time period and the video spread in the previous time period, in units of number. △t is the unit time, in seconds.

[0038] 4) Sort by propagation prospect prediction and send to step 5) in the order of sorting.

[0039] 5) According to the classification of the short video, the short video information is sent to step 6) and step 7) in sequence.

[0040] In this step, the classification method of short videos has been described in step 2), and this embodiment will not repeat this part of the content.

[0041] 6) Compare the information in the short video with the information in the public opinion dynamic platform; The information in the received short video is compared again with the information in the public opinion dynamic platform to determine whether it belongs to the information in the public opinion dynamic platform. The method for establishing the public opinion dynamic platform described in this step has been explained in step 1) and will not be repeated here.

[0042] 7) Compare the information in the short video with the information in the sensitive information database.

[0043] The information in the sensitive information database is stored in the following manner: the information in the sensitive information database is class information and keywords contained in the class information.

[0044] The categories of information include daily life, military, academic, entertainment, sports, and education.

[0045] Here, the method of obtaining information in the sensitive information database is as follows: one part is artificially defined sensitive words; one part is captured from online platforms, such as: short video platforms, live broadcast platforms, WeChat public accounts, official apps; one part is words cleaned from public opinion dynamics platforms.

[0046] 8) If any of step 6) and step 7) is successfully matched, a warning message will be generated and pushed to the warning platform; otherwise, the short video will be marked as passed.

[0047] The present invention is more suitable for review in the early stage of video generation. In the early stage of video generation, the short video is monitored by the method described in the present invention. After the monitoring is completed, it is pushed to big data for netizens to browse and forward.

[0048] Embodiment 2: This embodiment differs from Embodiment 1 in that: in the above steps, step 1) also includes generating a specific user database. For the received short video information, its user information is collected, and the user who sent the short video is compared with the specific user database according to the generated specific user database. According to the comparison result, if the match is successful, step 6) is directly performed.

[0049] The method for generating a specific user database is as follows: extracting specific user feature values, generating a clustering model based on the feature variables of each user; merging the clustering model with the preset specific user features to generate a specific user database. By generating specific user information, while comparing short videos, the user information is also compared, and videos posted by users of a specific user class are submitted for monitoring as soon as possible.

[0050] In this embodiment, by comparing user characteristics with a specific user database, short videos sent by specific users are directly monitored, thereby effectively avoiding the transmission of short videos of specific users on the network.

[0051] As an alternative to this embodiment, a heat map can also be generated for an area. In the heat map, a specific user IP is marked according to the IP address, and when a short video sent by a specific user IP is encountered, it is directly monitored.

[0052] Embodiment 3, this embodiment is different from embodiment 2 in that: this embodiment will be explained in detail in combination with a specific monitoring method: First, based on the current hot information on the Internet, collect the hot public opinion information and transmit it to the public opinion dynamic platform; for example: the hot public opinion information includes: the news forwarding volume and attention on catering are relatively high, so the keyword: catering is added to the public opinion dynamic platform.

[0053] Artificially define sensitive words and store them in the sensitive information database; use "crawlers" to capture sensitive words on the Internet and add them to the sensitive information database; after the above information is completed, the public opinion dynamic platform and the sensitive information database will be used as indicators for subsequent comparison.

[0054] Next, short video information is received, still taking the short video of catering as an example; the received short video information is extracted, and the sound information and video information therein are extracted in the form of metadata respectively; during the extraction process, the extracted metadata is preliminarily compared with the public opinion dynamic platform, for example, the keywords such as "catering" are extracted; after the comparison is successful, the metadata of the short video is immediately compared with the public opinion dynamic platform and the sensitive information database, and no other calculations are performed. If the public opinion dynamic platform happens to contain the keywords such as "catering", then after the comparison, a warning information is generated for the short video of "catering", and it is pushed to the warning platform for management and control to prevent large-scale public opinion.

[0055] Example 4: This example differs from Example 3 in that after the metadata is extracted from the short video information, it is not compared with the information in the public opinion dynamic platform. Then, the spread of the short video is calculated. The short video was rebroadcasted 200 times in the first time period, and the rebroadcast reached 800 times in the second time period, so its spread rate reached 600 times. In this case, the short video will be marked as high-risk, and the short video will be directly sent to the public opinion dynamic platform and the sensitive information database for comparison. No other calculations will be performed. After the comparison, an early warning message is generated for the short video of "catering" and pushed to the early warning platform for management and control to prevent large-scale public opinion.

[0056] Example 5. The difference between this example and Example 4 is that after extracting the metadata of the short video information, there is no information comparison on the public opinion dynamic platform. When calculating the propagation rate of the short video, the propagation amount in the first time period is 100, and the propagation rate in the second time period is 150, then its propagation rate is 50. In this case, the video is marked as low risk; the short videos marked as low risk in the same time period are sorted according to their propagation rates, and are sent to the subsequent comparison step in order.

[0057] For example, there are 4 videos for which the spread rate calculation is performed, and these 4 short videos are all low-risk short videos. Their spread rates are: the first spread rate is 100, the second spread rate is 230, the third spread rate is 340, and the fourth spread rate is 50; then they are sorted, and these 4 short videos are sent to the public opinion dynamic platform and the sensitive information database in the order of the third, second, first, and fourth for comparison.

[0058] If the comparison is successful, the short video will generate a warning message and be pushed to the warning platform for management and control to prevent large-scale public opinion.

[0059] Example 6: This example differs from Example 5 in that: after the short video is sent to the public opinion dynamic platform and the sensitive information database for comparison, if it is found that the short video does not match the public opinion dynamic platform and the sensitive information database, the short video is marked as "passed". The short video can be shared and viewed on the Internet.

[0060] Embodiment 7, this embodiment is the application example 1 of the monitoring method applied to short video monitoring.

[0061] In September, at the beginning of the school term, food safety in schools in a certain city became a hot topic. User A and User B sent short videos to the short video platform themselves.

[0062] After receiving the short videos of user A and user B, the short video monitoring method first extracted the user features and compared the features with the information in the "specific user database". The result was that user A was successfully compared. The short video of user A was directly sent to the public opinion dynamic platform and the sensitive information database for comparison. The result was that the short video information of user A was about public opinion content on school food safety. The short video platform banned the short video information of user A.

[0063] After the metadata of user B's short video was extracted, it was matched with the keywords "catering" and "school" in the public opinion dynamics platform. The short video was directly pushed to the public opinion dynamics platform and the sensitive information database for comparison in turn.

[0064] After comparison, it was found that the short video sent by user B was a public opinion video about the hidden dangers of school catering, and the short video sent by user B was removed from the shelves.

[0065] In just a few tens of seconds, the short video processing problems of user A and user B were completed at the same time, with fast processing speed and high accuracy.

[0066] The short videos of user A and user B were monitored using the present monitoring method and the conventional method respectively. Table 1 shows the processing speed and accuracy of the short videos by different methods.

[0067] Table 1 The processing effects of different methods on short videos Embodiment 8, this embodiment is the application example 2 of the monitoring method applied to short video monitoring.

[0068] In this embodiment, the monitoring method of the present invention is applied to an area with 485 users. During the time period from 7:30 to 7:40 p.m., the users in the area sent a total of 3,124 short videos, of which 20 were automatically intercepted and the remaining 3,104 were sent to the network.

[0069] Taking these 3140 short videos as the objects, the conventional method, the method of the present invention and the manual judgment method were used to process the short videos in this area. The processing results are shown in Table 2: Table 2 The processing effects of different methods on short videos It can be seen from the above table that the method described in the present invention can greatly improve the processing speed of short videos. In addition, the method described in the present invention can also improve the accuracy of short video processing.

[0070] The present invention discloses a short video monitoring method based on big data. The method can be used to monitor short videos in real time with fast monitoring speed and high efficiency. It can control the development of public opinion to a great extent, purify the network environment, and avoid the occurrence of large-scale uncontrollable public opinion dynamics.

Claims

1. A short video monitoring method based on big data, characterized in that: The method comprises the following steps: 1) Establish a public opinion dynamic platform; 2) Extract the information in the short video; classify the short video information; and preliminarily determine whether it can be matched with the information in the public opinion dynamic platform. If so, proceed directly to step 6); otherwise, proceed to step 4); 3) Predict the dissemination prospects of short video information; if the predicted value is high, proceed directly to step 6), otherwise proceed to step 4); 4) Sort by propagation prospect prediction and send to step 5) in the order of sorting; 5) According to the classification of short videos, the short video information is sent to step 6) and step 7 in sequence); 6) Compare the information in the short video with the information in the public opinion dynamic platform; 7) Compare the information in the short video with the information in the sensitive information database; 8) After successful comparison, generate warning information and push it to the warning platform; Otherwise, give the short video a pass tag.

2. The short video monitoring method based on big data as claimed in claim 1, characterized in that: The information in the short video includes metadata in the image information and metadata in the sound information.

3. The short video monitoring method based on big data as claimed in claim 2, characterized in that: The information in the sensitive information database is stored in the following manner: the information in the sensitive information database is class information and keywords contained in the class information.

4. The short video monitoring method based on big data as claimed in claim 3, characterized in that: The categories of information include daily life, military, academic, entertainment, sports, and education.

5. The short video monitoring method based on big data as claimed in claim 4, characterized in that: The public opinion dynamics platform collects recent public opinion information and transmits the collected public opinion information to the public opinion dynamics platform in real time.

6. The short video monitoring method based on big data as claimed in claim 5, characterized in that: The method to predict the dissemination prospects of short videos is: calculate the dissemination volume per unit time S=△L / △t.

7. The short video monitoring method based on big data as claimed in claim 6, characterized in that: If the transmission volume is less than 200, it is a low risk value; If the transmission volume is greater than or equal to 600, it is judged as a high-risk value.

8. The short video monitoring method based on big data as claimed in claim 7, characterized in that: When performing step 6), it is collected in real time to see whether there are short videos with a high-risk transmission volume. If so, it is directly sent to step 7).

9. The short video monitoring method based on big data as claimed in claim 8, characterized in that: Generate a specific user database, compare the user who sent the short video with the specific user database according to the generated specific user database, and if the match is successful, proceed directly to step 7); the method for generating the specific user database is: extract the specific user feature value, generate a clustering model based on the feature variables of each user; merge the clustering model with the preset specific user features to generate a specific user database.

10. The short video monitoring method based on big data according to claim 9, characterized in that: The information in the public opinion dynamics platform is cleansed once a week, and the cleaned data is removed; the remaining data is transferred to the sensitive information database.

Images