Method and system for detecting illegal content in free online streaming media service

Abstract

The invention provides a method and a system for detecting illegal content in free online streaming media service, and relates to the technical field of network security and data analysis, the method comprises the following steps: retrieving and capturing a mainstream media platform through a keyword set to obtain a target mainstream media platform set; performing hierarchical processing and classification processing on the mainstream media platform set to realize hierarchical classification of the platforms; according to hierarchical classification of the platform, obtaining website platform content and calculating a difference degree to obtain a target content website platform; simulating video playing according to the obtained target content website platform, and processing a playing page to obtain a processing result; according to a processing result, extracting multi-modal content to obtain features of different modals; and performing illegal content detection on the features of different modes to obtain a detection result. According to the detection method, the two types of illegal contents can be effectively identified and prevented, and the legality of a mainstream media platform and the safety of a user are guaranteed.

Description

Technical Field

[0001] The invention belongs to the technical field of network security and data analysis, and in particular relates to a method and system for detecting illegal content in a free online streaming media service. Background Art

[0002] With the rapid development of the Internet, mainstream media platforms have become the main channels for global users to obtain entertainment, information and social interaction. However, the means of disseminating illegal content are also constantly upgrading, especially on mainstream media platforms, and the dissemination forms are concealed. For example, some links are packaged with normal video covers, and once users click to enter the video playback page, they actually present illegal content. Illegal content mainly includes two categories: the first category is pop-up pages containing illegal content such as pornography and gambling; the second category is illegal content contained in the video playback window embedded in the site. These illegal contents not only violate relevant laws and regulations and endanger the physical and mental health of young people, but also easily cause user privacy leaks, and users' digital security faces severe challenges.

[0003] On mobile devices, the problem of malicious pop-ups is particularly prominent. Since users operate more frequently on mobile devices and the triggering behavior is simple, it is more difficult to regulate. The randomness of illegal content makes it difficult for traditional filtering technology to cope with it, and users face great security threats. Therefore, how to effectively detect and identify these illegal contents in mainstream media platforms has become an urgent problem to be solved. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for detecting illegal content in a free online streaming media service, which can effectively identify and prevent these two types of illegal content detection methods, thereby ensuring the legitimacy of the mainstream media platform and the safety of users.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] In a first aspect, a method for detecting illegal content in a free online streaming service is provided, the method comprising:

[0007] Search and crawl mainstream media platforms through keyword sets to obtain the target mainstream media platform set;

[0008] Conduct hierarchical and categorized processing on the collection of mainstream media platforms to achieve hierarchical classification of platforms;

[0009] According to the hierarchical classification of the platform, the website platform content is obtained and the difference is calculated to obtain the target content website platform;

[0010] According to the target content website platform, simulate the video playback and process the playback page to obtain the processing result;

[0011] According to the processing results, multimodal content is extracted to obtain features of different modalities;

[0012] Illegal content detection is performed on features of different modalities to obtain detection results.

[0013] Furthermore, the mainstream media platforms are searched and crawled through the keyword set to obtain the target mainstream media platform set, including:

[0014] By collecting keywords to build a keyword set;

[0015] Based on the keyword set, search through multiple search engines to obtain search result pages;

[0016] Get the accessibility of the search results page to get the content of the accessible page;

[0017] Analyze the matching degree with the original search keywords based on the content of the accessible pages to obtain the keyword matching degree;

[0018] Mainstream media platforms are crawled based on accessibility and keyword matching to obtain a set of target mainstream media platforms.

[0019] Furthermore, the set of mainstream media platforms is layered and classified to achieve layered classification of platforms, including:

[0020] According to the mainstream media platform, to obtain the HTML document content;

[0021] All URLs in the traversed HTML document are converted into feature values ​​according to the suffix of the URL to obtain the feature value of each URL;

[0022] By analyzing the DOM tree, the level of the feature value of each URL in the DOM tree is determined to obtain the level of the URL;

[0023] According to the URL level, obtain the features of the URL at the same level, including HTML structure level features, string features, suffix features, and access features;

[0024] According to mainstream media platforms and URL characteristics, URLs are classified into website homepages, first-level pages, and second-level pages.

[0025] Furthermore, according to the hierarchical classification of the platform, the website platform content is obtained and the difference is calculated to obtain the target content website platform, including:

[0026] According to the hierarchical classification of platforms, HTML texts of desktop and mobile terminals are obtained;

[0027] According to the HTML text of the desktop and mobile terminals, the core elements of the page are extracted, and the extracted core elements are subjected to text vectorization processing to obtain vectorization processing results;

[0028] According to the vectorization processing results, Calculate the difference, where e A and e B Represent the vectors of HTML text on the desktop and mobile terminals respectively, Di represents the degree of difference;

[0029] Set a difference threshold, compare the calculated difference with the difference threshold, measure the information content, and obtain the content at the end with more information content to obtain the target content website platform.

[0030] Further, according to the target content website platform, the video playback is simulated, and the playback page is processed to obtain the processing result, including:

[0031] According to the target content website platform, determine the target URL of the content in the playback window and the secondary page of the collection to obtain the target URL;

[0032] According to the target URL, determine the time ratio of video playback, and obtain the playback strategy based on the time ratio of video playback.

[0033] According to the play window and play strategy, simulate the video play to achieve the simulated play of the video;

[0034] According to the video simulation playback process, monitor the changes in HTML structure and identify the pop-up ads or hidden pages that appear to obtain real-time playback data;

[0035] The playback data is processed to obtain a processing result.

[0036] Furthermore, based on the processing results, multimodal content is extracted to obtain features of different modalities, including:

[0037] According to the processing results, dynamically load pictures and pop-up windows to obtain simulated page content;

[0038] According to the simulated page content, extract the page information related to the illegal links and the packaged links to obtain the links and packaged content;

[0039] The link and the packaged content are stored, and features of different modalities are extracted from the stored data to obtain features of different modalities, wherein the features of different modalities include HTML structure features, URL features, text features, and image features.

[0040] Furthermore, illegal content detection is performed on features of different modalities to obtain detection results, including:

[0041] Fuse the features of different modalities to obtain fused features;

[0042] The fused feature vectors are classified and tested to obtain the test results.

[0043] In a second aspect, a system for detecting illegal content in a free online streaming service comprises:

[0044] The acquisition module is used to search and crawl mainstream media platforms through a keyword set to obtain a target mainstream media platform set; hierarchical processing and classification processing are performed on the mainstream media platform set to achieve hierarchical classification of the platforms; based on the hierarchical classification of the platforms, the website platform content is acquired and the difference degree is calculated to obtain the target content website platform;

[0045] The processing module is used to simulate video playback according to the target content website platform, and process the playback page to obtain the processing result; according to the processing result, extract multimodal content to obtain the characteristics of different modes; and perform illegal content detection on the characteristics of different modes to obtain the detection result.

[0046] According to a third aspect, a computing device includes:

[0047] one or more processors;

[0048] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described.

[0049] In a fourth aspect, a computer-readable storage medium stores a program, and when the program is executed by a processor, the method described is implemented.

[0050] The above solution of the present invention includes at least the following beneficial effects:

[0051] By comprehensively using a variety of technical means such as crawler technology, HTML text analysis, image recognition and text recognition, it is possible to more effectively identify and classify illegal content, especially those pages and video content that are highly disguised and difficult to find, thereby significantly improving the detection accuracy; by accurately detecting and preventing the spread of illegal content, it can provide users with a safer streaming media browsing experience, prevent users from being exposed to bad information, and protect users from online fraud and malware; through automated detection and classification, it can greatly save the time and cost of manual review and improve regulatory efficiency. By removing illegal and bad content, it helps to improve the overall quality and image of the streaming media platform, attract more legal and high-quality content providers to join, and can cover illegal content identification on both desktop and mobile terminals at the same time, ensuring that users can have a safe browsing experience on different devices. It can also prevent and combat cybercrime activities to a certain extent and maintain the security and stability of cyberspace. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 The present invention is a flowchart of a method for detecting illegal content in a free online streaming media service provided by an embodiment of the present invention.

[0053] Figure 2 The invention is a schematic diagram of an illegal content detection system in a free online streaming media service provided by an embodiment of the invention. DETAILED DESCRIPTION

[0054] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in a form and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0055] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting illegal content in a free online streaming service, the method comprising the following steps:

[0056] Step 11, searching and crawling mainstream media platforms through a keyword set to obtain a target mainstream media platform set;

[0057] Step 12, hierarchical processing and classification processing are performed on the set of mainstream media platforms to achieve hierarchical classification of the platforms;

[0058] Step 13, according to the hierarchical classification of the platform, obtain the website platform content and calculate the difference degree to obtain the target content website platform;

[0059] Step 14, simulating video playback according to the target content website platform, and processing the playback page to obtain a processing result;

[0060] Step 15: extracting multimodal content according to the processing result to obtain features of different modalities;

[0061] Step 16: Perform illegal content detection on features of different modes to obtain detection results.

[0062] In the embodiment of the present invention, by searching through a keyword set, the mainstream media platform related to a specific topic or content can be quickly located, thereby improving the efficiency and accuracy of information acquisition; hierarchical processing and classification processing of the mainstream media platform set is helpful to structure the complex data. Through hierarchical classification, the platform can be more accurately classified according to its characteristics and content type, and the difference of the website platform content can be calculated, which helps to identify platforms with richer information content and more frequent updates, thereby optimizing resource allocation and improving detection efficiency. Through difference analysis, the target content website platform can be more accurately located, reducing unnecessary data processing; through simulated video playback, the scene of user accessing and browsing streaming media content can be more realistically restored, thereby improving the accuracy and reliability of detection. Processing the playback page can capture various changes in the video playback process, including pop-up ads and hidden pages, thereby achieving comprehensive content monitoring. Extracting multimodal content features can more comprehensively reflect the characteristics of streaming media content, including text, images, links and other aspects, thereby improving the comprehensiveness and accuracy of detection. By detecting illegal content based on the features of different modes, streaming media content containing illegal information can be accurately identified. The detection results can be used as a basis for timely warnings to help relevant departments take prompt measures to prevent the spread and dissemination of illegal content.

[0063] In a preferred embodiment of the present invention, the above step 11 may include:

[0064] By collecting keywords to build a keyword set;

[0065] Based on the keyword set, search through multiple search engines to obtain search result pages;

[0066] Get the accessibility of the search results page to get the content of the accessible page;

[0067] Analyze the matching degree with the original search keywords based on the content of the accessible pages to obtain the keyword matching degree;

[0068] Mainstream media platforms are crawled based on accessibility and keyword matching to obtain a set of target mainstream media platforms.

[0069] In the embodiment of the present invention, by collecting keywords to build a set, it is possible to more accurately locate content related to a specific topic or query, thereby improving the pertinence and accuracy of the search; a complete keyword set can cover various situations that a user may search for, thereby ensuring comprehensive acquisition of information, and searching through multiple search engines based on the keyword set to obtain a search result page; searching using multiple search engines can obtain results from different information sources, thereby increasing the diversity and breadth of information. Different search engines have different algorithms and index libraries, and integrating the results of multiple search engines can improve the reliability and integrity of information. By checking the accessibility of the page, invalid or expired links can be filtered out, and further processing of inaccessible pages can be avoided, thereby saving system resources and time. According to the content of the accessible page, the matching degree with the original search keyword is analyzed to obtain the keyword matching degree; the matching degree analysis can evaluate the degree of association between the page content and the user query, help identify high-quality, highly relevant information, and sort the search results according to the matching degree, so as to ensure that the user first sees the content that best meets the needs. Combining accessibility and keyword matching for crawling can effectively screen out mainstream media platforms that meet specific conditions. By accurately crawling mainstream media platforms, unnecessary data processing can be reduced and overall processing efficiency can be improved.

[0070] In a specific embodiment of the present invention, GET requests are sent to multiple target streaming media platforms to obtain HTML documents and parse page contents, extract text content from the pages, truncate words with a length of less than 10 based on space delimiters, and remove duplicates and invalid symbols; count the number of times a keyword appears on different platforms, calculate the word frequency, select keywords with high word frequency to represent the total sample volume of the target streaming media platform and add them to the keyword thesaurus, and divide the keywords into three categories: general keywords, dynamic hot keywords and target keywords, among which general keywords are keywords that appear more frequently on streaming media platforms, dynamic hot keywords are keywords in recent hot network events, and target keywords are randomly combined keywords of different categories for retrieval, with keywords with higher word frequency being selected first.

[0071] In another specific embodiment of the present invention, by sending GET requests to multiple search engines to verify accessibility, invalid links or expired pages are excluded, the captured page content is cleaned, HTML tags and advertisement-irrelevant information are removed, and plain text content is extracted; Calculate the similarity between the keyword vector and the text vector. The similarity value corresponds to the keyword matching score. According to the keyword matching, confirm whether it is the target streaming platform. If the conditions are met, add it to the target streaming platform set.

[0072] In a preferred embodiment of the present invention, the above step 12 may include:

[0073] According to the mainstream media platform, to obtain the HTML document content;

[0074] All URLs in the traversed HTML document are converted into feature values ​​according to the suffix of the URL to obtain the feature value of each URL;

[0075] By analyzing the DOM tree, the level of the feature value of each URL in the DOM tree is determined to obtain the level of the URL;

[0076] According to the URL level, obtain the features of the URL at the same level, including HTML structure level features, string features, suffix features, and access features;

[0077] According to mainstream media platforms and URL characteristics, URLs are classified into website homepages, first-level pages, and second-level pages.

[0078] In the embodiment of the present invention, directly obtaining the HTML document content of the mainstream media platform can ensure that the most original and comprehensive data information of the platform is obtained; by traversing all URLs in the HTML document and converting their suffixes into feature values, the characteristics of each URL can be more carefully portrayed. The URL suffix is ​​often closely related to the type and function of the web page. Using these feature values ​​can improve the accuracy of identifying the type and purpose of the URL. The DOM tree is a structured representation of the HTML document. By analyzing the DOM tree, the position and role of each URL in the web page structure can be more clearly understood. Determining the level of the URL in the DOM tree helps to understand the organizational structure and information flow of the web page. Combining multiple features, such as HTML structure, string, suffix, access features; analyzing the URLs of the same level can more comprehensively describe the properties and behaviors of these URLs. Comprehensively using multiple features to classify URLs can improve the accuracy and reliability of classification, and help accurately identify different types of web pages. Classifying URLs helps to organize massive amounts of web page information in an orderly manner. Different types of web pages differ in content importance, update frequency, etc. Classification can optimize resource allocation and improve information processing efficiency.

[0079] In a specific embodiment of the present invention, the desktop is simulated to open the streaming media platform website, the automatic collection tool is configured, the waiting time is set, all elements of the page are ensured to be loaded, the HTML document content is obtained, the URL is extracted, and the URL feature conversion is performed. The conversion strategy is: ① According to the suffix type of the URL, if the suffix type is .html, the feature value is 0, if it is .js, the feature value is 1, and if it is other types of suffixes, the feature value is 2. ② Through the analysis of the HTML text structure and based on the hierarchical characteristics of the DOM tree, the level where the html tag is located is the 0th level, and the level where each tag is located is the level of the external nested tag plus 1, and the feature value is the hierarchical tree where the URL is located. ③ Calculate the number of other URLs at the level where the URL is located. ④ Observe whether there is a title attribute after the URL. If so, the feature value is 1, otherwise it is 0. ⑤ Analyze whether there is a target attribute after the URL. If so, the feature value is 1, otherwise it is 0. ⑥ Analyze whether the URL contains the play string. If it is included, the feature value is 1, otherwise it is 0. ⑦ Analyze whether the URL is accessible. If it is accessible, that is, the HTTP status code returned after the request to the URL is 200, the feature value is 1, otherwise it is 0. ⑧ Analyze whether the URL contains a substring of a number-number pattern. If it does, the feature value is 1, otherwise it is 0. By using the trained CART algorithm, the URL is divided into the homepage, first-level page, and second-level page of the website, and the pages to be detected are placed in the detection target URL library.

[0080] In a preferred embodiment of the present invention, the above step 13 may include:

[0081] According to the hierarchical classification of platforms, HTML texts of desktop and mobile terminals are obtained;

[0082] According to the HTML text of the desktop and mobile terminals, the core elements of the page are extracted, and the extracted core elements are subjected to text vectorization processing to obtain vectorization processing results;

[0083] According to the vectorization processing results, Calculate the difference, where e A and e B Represent the vectors of HTML text on the desktop and mobile terminals respectively, Di represents the degree of difference;

[0084] Set a difference threshold, compare the calculated difference with the difference threshold, measure the information content, and obtain the content at the end with more information content to obtain the target content website platform.

[0085] In an embodiment of the present invention, the HTML text of the desktop and mobile terminals is obtained by hierarchical classification, which can ensure the comprehensiveness and diversity of the data. Analyzing the HTML text of different platforms (desktop and mobile terminals) helps to understand the differences between platforms and improve the adaptability and flexibility of information processing. Extracting the core elements of the page can remove redundant information, focus on key content, improve the efficiency and accuracy of data processing, and vectorize the core elements to convert them into a mathematically operable format. By calculating the difference, the degree of difference between the HTML text of the desktop and mobile terminals can be objectively quantified. Using a vectorized method to calculate the difference can achieve a more accurate and detailed comparison, which helps to discover subtle but important differences. By setting a difference threshold, the requirements for information content can be flexibly controlled and the screening criteria can be adjusted as needed. Selecting the end with more information content for content acquisition, through this method, the target content website platform can be more accurately located, improving the pertinence and efficiency of information acquisition.

[0086] In a specific embodiment of the present invention, an automated acquisition tool is used to simulate a desktop browser accessing the target streaming media platform, and the window size is set to 1920×1080, and the simulated mobile device model can be set to iPhone 12. Use regular expressions to preprocess HTML documents, remove irrelevant tags and elements, and retain key elements including: ① identifiers in the div tag where the pop-up window is placed; ② href links pointing to illegal content to extract href attributes in all tags on the page and identify potential illegal links; ③ introduce and package the image img tag of illegal content, extract the src attribute in the img tag, and obtain image links that may be used for advertising or illegal content; ④ text content.

[0087] In another specific embodiment of the present invention, some keywords commonly used in HTML text composition are eliminated, the pre-processed desktop and mobile HTML texts are vectorized, and the difference is calculated; the difference threshold is set to 2%, and when the calculated difference is higher than the threshold, it is determined that there is a significant difference between the desktop and mobile terminals, and the text lengths are compared to measure the information content, and the end with more information content is selected for content acquisition.

[0088] In a preferred embodiment of the present invention, the above step 14 may include:

[0089] According to the target content website platform, determine the target URL of the content in the playback window and the secondary page of the collection to obtain the target URL;

[0090] According to the target URL, determine the time ratio of video playback, and obtain the playback strategy based on the time ratio of video playback.

[0091] According to the play window and play strategy, simulate the video play to achieve the simulated play of the video;

[0092] According to the video simulation playback process, monitor the changes in HTML structure and identify the pop-up ads or hidden pages that appear to obtain real-time playback data;

[0093] The playback data is processed to obtain a processing result.

[0094] In the embodiment of the present invention, by determining the playback window and the target URL, the video content that needs to be played and collected can be accurately located, and the content target URL in the secondary page can be directly located, which can reduce unnecessary page jumps and loading time, and improve the efficiency of content collection. By determining the playback strategy according to the time ratio of video playback, a more intelligent and flexible playback control can be achieved to adapt to the playback requirements of different video contents. A reasonable playback strategy can reduce unnecessary video loading and playback resource consumption, and improve the operating efficiency of the overall system. By simulating video playback, the user's experience when browsing and watching videos can be restored more realistically, and the simulation playback of videos can automatically process a large amount of video content, reduce the burden of manual operation, and improve processing efficiency. Monitoring the changes in HTML structure can capture dynamic content changes in the video playback process in real time, including the appearance of pop-up ads and hidden pages. By acquiring playback data in real time, the timeliness and accuracy of the data can be ensured, and processing the playback data can remove redundant information and extract key data. The processing results can be displayed and output in a more intuitive and easier to understand form.

[0095] In a specific embodiment of the present invention, the playback window positioning is determined by parsing the HTML structure of the page to check whether it contains (div) tags related to the video player, and the content acquisition target URL of the secondary page is determined to simulate video playback; intensive acquisition is performed within the time ratio intervals of 0.0 to 0.1 and 0.9 to 1.0 of video playback; sparse acquisition is performed during other time periods of video playback, and during video playback, the HTML structure is regularly checked for changes to determine whether there are new pop-up ads or hidden pages.

[0096] In a preferred embodiment of the present invention, the above step 15 may include:

[0097] According to the processing results, dynamically load pictures and pop-up windows to obtain simulated page content;

[0098] According to the simulated page content, extract the page information related to the illegal links and the packaged links to obtain the links and packaged content;

[0099] The link and the packaged content are stored, and features of different modalities are extracted from the stored data to obtain features of different modalities, wherein the features of different modalities include HTML structure features, URL features, text features, and image features.

[0100] In an embodiment of the present invention, by dynamically loading pictures and pop-up windows, all the contents of the simulated page can be obtained more comprehensively, and dynamic loading ensures that the page content is captured and obtained in real time during the simulated playback process, reflecting the latest page status. By extracting information related to illegal links and packaged links, illegal or fraudulent activities on the network can be more effectively identified and tracked. Timely discovery and processing of illegal links helps to improve network security and protect users from malicious software. Storing links and packaged content can ensure the persistence and security of data, and extracting features of different modalities, such as HTML structure, URL, text, and image, can provide a multi-dimensional data analysis perspective and enhance the comprehensiveness and accuracy of the analysis. Comprehensively utilizing multiple features of HTML structure, URL, text, and image can build a richer feature set.

[0101] In a specific embodiment of the present invention, the page is scrolled gradually to ensure that all contents in the page, especially dynamically loaded contents such as images and pop-up windows, can be fully presented. The relevant elements in HTML include: (a) the href attribute of the tag, which is used to extract possible illegal jump links; (div) the id or class attribute of the tag, which is used to locate the container containing the pop-up window; (img) the src attribute of the tag, which is used to obtain the image link related to the illegal content and identify the packaging method of the illegal content. By combining with the base URL of the page, a complete image path is generated. The link in the pop-up window is usually nested in HTML together with the related images or advertising text. Through the nested relationship in the HTML structure, the image or text packaging content is associated with the corresponding link and stored in the detection resource library.

[0102] In another specific embodiment of the present invention, features of different modes are extracted from the stored data to obtain features of different modes, and the features of different modes include:

[0103] Extraction of HTML structural features: HTML structure mainly includes the following four features: ① Element hierarchical position: Through the analysis of HTML text structure and based on the hierarchical characteristics of the DOM tree, the level of the (html) tag is the 0th level; the level of each tag is the level of the tag nested outside it plus 1, and the feature value is the hierarchical tree where the URL is located. ② Tag features: Pop-up pages usually appear in a specific tag combination, with a feature value of 1 if included and 0 if not. ③ Element attributes: The attributes of the element can provide key information for pop-up detection, with a feature value of 1 if included and 0 if not. ④ The relative position of the element in the platform page: Get the total height and width of the page and calculate the relative position of the pop-up. Convert the vertical and horizontal positions of the pop-up window into proportional values ​​to determine whether the pop-up window is in a specific area such as the top, bottom, left and right sides, center or lower right corner of the page, and correspond these positions to codes, with the top being 1 and the bottom being 2.

[0104] URL feature extraction: By using standard URL parsing tools or libraries, separate the protocol, subdomain, main domain and path parts, and focus on extracting the main domain part.

[0105] Text feature extraction: Use optical character recognition technology to extract embedded advertisements or slogans from images and merge them with other page texts, and encode the text content in the web page.

[0106] Image feature extraction: Use convolutional neural networks to perform deep feature extraction on images on web pages.

[0107] In a preferred embodiment of the present invention, the above step 16 may include:

[0108] Fuse the features of different modalities to obtain fused features;

[0109] The fused feature vectors are classified and tested to obtain the test results.

[0110] In an embodiment of the present invention, the fusion of features of different modalities can achieve complementarity between information, making the fused features more representative and comprehensive, thereby enhancing the expressive power of the features. The fusion of multimodal features helps to reduce the noise and uncertainty that may exist in single modal features and improve the robustness and stability of the features. Using the fused feature vector for classification detection can make full use of the advantages of multimodal information, achieve more accurate classification and recognition, and improve the accuracy of the detection results. By processing the fused feature vector at one time instead of processing the features of each modality separately, the detection process can be simplified and the detection efficiency can be improved.

[0111] In a specific embodiment of the present invention, after each feature is extracted independently, it is converted into a feature vector to represent the integration of four features (HTML structure features, URL features, text features, and image features), different weights are assigned according to the importance of the features, and a multimodal fusion algorithm is used to fuse features of different modalities. The fused feature vector is used for detection to determine whether there is illegal content, and the illegal content is classified to obtain a detection result.

[0112] like Figure 2 As shown, an embodiment of the present invention further provides an illegal content detection system 20 in a free online streaming media service, comprising:

[0113] The acquisition module 21 is used to search and crawl mainstream media platforms through a keyword set to obtain a target mainstream media platform set; perform hierarchical processing and classification processing on the mainstream media platform set to achieve hierarchical classification of the platforms; obtain website platform content and calculate the difference degree according to the hierarchical classification of the platforms to obtain the target content website platform;

[0114] The processing module 22 is used to simulate video playback according to the target content website platform, and process the playback page to obtain a processing result; based on the processing result, extract multimodal content to obtain features of different modalities; and perform illegal content detection on the features of different modalities to obtain a detection result.

[0115] It should be noted that the system is a system corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0116] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting illegal content in a free online streaming service, characterized in that: The method comprises: Search and crawl mainstream media platforms through keyword sets to obtain the target mainstream media platform set; Conduct hierarchical and categorized processing on the collection of mainstream media platforms to achieve hierarchical classification of platforms; According to the hierarchical classification of the platform, the website platform content is obtained and the difference is calculated to obtain the target content website platform; According to the target content website platform, simulate the video playback and process the playback page to obtain the processing result; According to the processing results, multimodal content is extracted to obtain features of different modalities; Illegal content detection is performed on features of different modalities to obtain detection results.

2. The method for detecting illegal content in a free online streaming service according to claim 1, characterized in that: Search and crawl mainstream media platforms through keyword sets to obtain the target mainstream media platform set, including: By collecting keywords to build a keyword set; Based on the keyword set, search through multiple search engines to obtain search result pages; Get the accessibility of the search results page to get the content of the accessible page; Analyze the matching degree with the original search keywords based on the content of the accessible pages to obtain the keyword matching degree; Mainstream media platforms are crawled based on accessibility and keyword matching to obtain a set of target mainstream media platforms.

3. The method for detecting illegal content in a free online streaming service according to claim 2, characterized in that: The mainstream media platform collection is layered and classified to achieve layered classification of the platforms, including: According to the mainstream media platform, to obtain the HTML document content; All URLs in the traversed HTML document are converted into feature values ​​according to the suffix of the URL to obtain the feature value of each URL; By analyzing the DOM tree, the level of the feature value of each URL in the DOM tree is determined to obtain the level of the URL; According to the URL level, obtain the features of the URL at the same level, including HTML structure level features, string features, suffix features, and access features; According to mainstream media platforms and URL characteristics, URLs are classified into website homepages, first-level pages, and second-level pages.

4. The method for detecting illegal content in a free online streaming service according to claim 3, characterized in that: According to the hierarchical classification of the platform, the website platform content is obtained and the difference is calculated to obtain the target content website platform, including: According to the hierarchical classification of platforms, HTML texts of desktop and mobile terminals are obtained; Extract the core elements of the page based on the HTML text of the desktop and mobile terminals, and perform text vectorization processing on the extracted core elements to obtain vectorization processing results; According to the vectorization processing results, Calculate the difference, where e A and e B Represent the vectors of HTML text on the desktop and mobile terminals respectively, Di represents the degree of difference; Set a difference threshold, compare the calculated difference with the difference threshold, measure the information content, and obtain the content at the end with more information content to obtain the target content website platform.

5. The method for detecting illegal content in a free online streaming service according to claim 4, characterized in that: According to the target content website platform, simulate the video playback and process the playback page to obtain the processing results, including: According to the target content website platform, determine the target URL of the content in the playback window and the collection secondary page to obtain the target URL; According to the target URL, determine the time ratio of video playback, and obtain the playback strategy based on the time ratio of video playback. According to the play window and play strategy, simulate the video play to achieve the simulated play of the video; According to the video simulation playback process, monitor the changes in HTML structure and identify the pop-up ads or hidden pages that appear to obtain real-time playback data; The playback data is processed to obtain a processing result.

6. The method for detecting illegal content in a free online streaming service according to claim 5, characterized in that: Based on the processing results, multimodal content is extracted to obtain features of different modalities, including: According to the processing results, dynamically load pictures and pop-up windows to obtain simulated page content; According to the simulated page content, extract the page information related to the illegal links and the packaged links to obtain the links and packaged content; The link and the packaged content are stored, and features of different modalities are extracted from the stored data to obtain features of different modalities, wherein the features of different modalities include HTML structure features, URL features, text features, and image features.

7. The method for detecting illegal content in a free online streaming service according to claim 6, characterized in that: Perform illegal content detection on features of different modalities to obtain detection results, including: Fuse the features of different modalities to obtain fused features; The fused feature vectors are classified and tested to obtain the test results.

8. A system for detecting illegal content in a free online streaming service, characterized in that: include: An acquisition module is used to search and crawl mainstream media platforms through a keyword set to obtain a target mainstream media platform set; Perform hierarchical processing and classification on the set of mainstream media platforms to achieve hierarchical classification of the platforms; obtain the website platform content and calculate the difference according to the hierarchical classification of the platforms to obtain the target content website platform; The processing module is used to simulate video playback according to the target content website platform, and process the playback page to obtain a processing result; according to the processing result, extract multimodal content to obtain features of different modes; Illegal content detection is performed on features of different modalities to obtain detection results.

9. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images