Illegal advertisement identification method, device, electronic device and storage medium
By acquiring and processing users' DNS access records and webpage source code, and combining them with a dataset of advertising violation keywords to identify illegal advertisements, this approach solves the problems of high computational resource consumption and low identification accuracy in existing technologies, and achieves efficient identification of illegal advertisements.
Patent Information
- Application Number
- CN202110661330.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-15
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-06-15
AI Technical Summary
Existing methods for identifying illegal internet advertisements consume significant computational resources, take long processing times, and have low accuracy when dealing with massive amounts of data.
By obtaining users' DNS access records, deduplication and determination of valid web pages, crawling source code, extracting advertising URLs and text, using a pre-set dataset of advertising violation words to determine whether it contains illegal text, and outputting the URL of the illegal advertisement.
It improves the accuracy of identifying illegal advertisements and the efficiency of computing resource utilization, while saving computing time.
Smart Images

Figure CN113505317B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and specifically to a method, device, electronic device and storage medium for identifying illegal advertisements. Background Art
[0002] With the development of the Internet, the number of websites in my country is increasing. According to statistics, as of December 2019, the total number of domain names in my country was 50.94 million. The Internet advertising business revenue was 470 billion yuan. Internet advertising is an important form of advertising. Compared with traditional radio, television, newspaper and print advertising, which often costs tens of thousands or hundreds of thousands of yuan, Internet advertising is cheap, has a fast transmission speed and a wide range of dissemination, reflecting a super high "cost performance".
[0003] However, driven by profit, Internet advertising has become a hotbed for false and illegal advertising. When faced with massive amounts of big data, existing Internet illegal advertising identification methods consume large amounts of computing resources, take a long time to calculate, and have low recognition accuracy. Summary of the invention
[0004] Since the existing methods have the above-mentioned problems, the embodiments of the present application provide a method, device, electronic device and storage medium for identifying illegal advertisements.
[0005] The specific embodiments of the present application provide the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides a method for identifying illegal advertisements, including:
[0007] Obtain the user's domain name system DNS access record, determine whether the web page corresponding to the DNS access record is valid, and if valid, crawl the source code of the web page;
[0008] According to the source code, determining an advertisement uniform resource locator URL contained in the source code;
[0009] Determine, according to the advertisement URL, the advertisement text corresponding to the advertisement URL;
[0010] According to a preset advertisement illegal word data set, it is determined whether the advertisement text contains illegal words. If so, the current illegal words and the illegal advertisement URL corresponding to the current illegal words are output.
[0011] Optionally, before determining whether the webpage corresponding to the DNS access record is valid, the method further includes:
[0012] The DNS access records are deduplicated to remove redundant DNS access records and inaccessible domain names.
[0013] Optionally, determining, according to the source code, an advertisement URL contained in the source code includes:
[0014] Extracting all web page URLs and web page URLs contained in inline frame elements in the source code;
[0015] The advertisement URL included in the source code is determined according to the webpage URL.
[0016] Optionally, determining the advertisement URL contained in the source code according to the webpage URL includes:
[0017] According to the length of the web page URL and / or the domain name combination form of the web page URL, it is determined whether the web page URL is a normal jumpable link. If so, it is determined that the current web page URL is a non-advertisement URL, otherwise the current web page URL is an advertisement URL.
[0018] Optionally, determining, according to the advertisement URL, an advertisement text corresponding to the advertisement URL includes:
[0019] The advertisement text corresponding to the advertisement URL is determined according to the text of the a tag in the webpage source code corresponding to the advertisement URL and the text appearing in the remark attribute.
[0020] Optionally, the preset advertising illegal word data set is calculated based on the easily influenced characteristic of the current webpage; the easily influenced characteristic of the current webpage is the degree to which the subject of the current webpage is easily influenced by the subject content of other webpages.
[0021] In a second aspect, an embodiment of the present application provides a device for identifying illegal advertisements, including:
[0022] The first processing module is used to obtain the user's domain name system DNS access record, determine whether the web page corresponding to the DNS access record is valid, and if valid, crawl the source code of the web page;
[0023] A second processing module, configured to determine, based on the source code, an advertisement uniform resource locator URL contained in the source code;
[0024] A third processing module, used to determine the advertisement text corresponding to the advertisement URL according to the advertisement URL;
[0025] The fourth processing module is used to determine whether the advertisement text contains illegal words according to a preset advertisement illegal word data set, and if so, output the current illegal words and the illegal advertisement URL corresponding to the current illegal words.
[0026] Optionally, the second processing module is specifically used to:
[0027] Extracting all web page URLs and web page URLs contained in inline frame elements in the source code;
[0028] The advertisement URL included in the source code is determined according to the webpage URL.
[0029] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for identifying illegal advertisements as described in the first aspect is implemented.
[0030] In a fourth aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for identifying illegal advertisements as described in the first aspect.
[0031] It can be seen from the above technical solution that the embodiment of the present application first obtains the user's domain name system DNS access record, determines whether the web page corresponding to the DNS access record is valid, and if valid, crawls the source code of the web page, otherwise discards the current DNS access record; then, according to the source code, determines the advertisement uniform resource locator URL contained in the source code, and according to the advertisement URL, determines the advertisement text corresponding to the advertisement URL; finally, according to the preset advertisement violation word data set, determines whether the advertisement text contains violation text, and if so, outputs the current violation text and the violation advertisement URL corresponding to the current violation text. It can be seen that the embodiment of the present application provides a method for identifying violation advertisements for massive streaming data. On the one hand, by eliminating inaccessible web pages in the user access record, it saves the time for calculating advertisement links in the web page. On the other hand, the advertisement links and advertisement text are extracted through the web page source code structure to identify the violation advertisements in the web page, which has high accuracy and strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0033] Figure 1 This is one of the step flow charts of the illegal advertisement identification method provided in the embodiment of the present application;
[0034] Figure 2 This is the second step flow chart of the illegal advertisement identification method provided in the embodiment of the present application;
[0035] Figure 3It is a structural diagram of an illegal advertisement identification device provided in an embodiment of the present application;
[0036] Figure 4 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] like Figure 1 As shown, a method for identifying illegal advertisements provided in an embodiment of the present application includes:
[0039] Step 101: Obtain the user's domain name system DNS access record, determine whether the web page corresponding to the DNS access record is valid, and if valid, crawl the source code of the web page;
[0040] In this step, since the amount of web page access data from different users is large every day and new access data is generated every day, the daily user's Domain Name System DNS (Domain Name System) access records can be regarded as streaming data, specifically, new data is received every day, and the amount of user access records in a single day is huge, which can be called massive data. Therefore, the continuous daily input data is classified as massive streaming data.
[0041] Among them, streaming data refers to data that is continuously generated by thousands of data sources, usually sent in the form of data records at the same time, and is small in size (about a few kilobytes). Streaming data includes a variety of data, such as log files generated by global wide area network web applications, online shopping data, in-game player activities, social networking site information, financial trading halls or geospatial services, and telemetry data from connected devices or instruments in data centers. Such data needs to be processed incrementally by record or in sequence according to a sliding time window, and can be used for a variety of analyses, including association, aggregation, filtering and sampling. With the information obtained from such analysis, companies can gain in-depth understanding of all aspects of their business and customer activities, such as service usage (for metering / billing), server activity, website clicks, and the geographic location of devices, people and physical objects, so as to respond quickly to new situations. For example, companies can continuously analyze social media streams to track changes in public perception of their brands and products and respond in a timely manner when necessary.
[0042] Among them, the domain name system is a service provided to people on the Internet. DNS is actually a special type of distributed database that can map domain names and IP addresses to each other. Its value lies in the convenience and speed it brings. Users can easily perform network queries without knowing the principles behind it. Domain name resolution is to point a domain name to its corresponding website space IP address, so that network users can achieve their desire to access specific web pages only through the registered domain name.
[0043] In this step, for the massive streaming Uniform Resource Locator URL (Uniform Resource Locator) access data, first obtain the DNS access records of a large number of users, and perform domain name deduplication operations on them to eliminate redundant DNS access records. It is understandable that since the websites that users may visit are repeated, and multiple judgments on the same website greatly consume computing resources and waste computing time, deduplication operations are very necessary. In addition, optionally, inaccessible web pages in the user's DNS access records can also be eliminated, for example, domain names containing "Content Delivery Network (CDN)" in the domain name, which are inaccessible. The above method not only ensures that the DNS access record data is not redundant, but also simplifies the operation of subsequent identification steps.
[0044] In this step, for the DNS access records after deduplication, it is also possible to determine whether the web page corresponding to each DNS access record can be opened, and the web pages that can be accessed will be retained, and the rest will be discarded. This is because users may want to visit some randomly generated websites or mistakenly enter the URL of a website, but such websites themselves cannot access specific content, so there is no benefit in analyzing such websites. Therefore, this method further saves computing time, so that computing resources can be used more reasonably.
[0045] Step 102: Determine the advertisement uniform resource locator URL contained in the source code according to the source code;
[0046] In this step, after obtaining a non-duplicate and valid web page, a crawler program is used to crawl the source code of the web page, and then the web page source code is analyzed. It is believed that the web page URL contained in the web page source code may be an advertising link. Therefore, all web page URLs are extracted from the web page source code. Since only URL jumps on the web page can be extracted, floating window advertisements cannot be extracted, the URLs in the web page source code are extracted. <iframe>The frame also adds the URL of the web page contained in it to the list of URLs to be detected.
[0047] In this step, for each URL in the list of URLs to be detected, two steps of screening can be performed. First, the length of the URL is determined, and URLs with a length of less than 150 are regarded as normal in-site jump links and can be removed from the list. Then, the remaining URLs are judged for the second time, and URLs containing the combination of "top-level domain name + first-level domain name" of the original website are regarded as normal in-site jump links. These are removed, and the remaining ones are advertising URLs.
[0048] Step 103: Determine the advertisement text corresponding to the advertisement URL according to the advertisement URL;
[0049] In this step, since the text on the advertisement URL appears in two locations in the webpage source code, one is the text of the a tag and the other is the remark attribute of the a tag, and the contents of the two are often repeated. Therefore, the union of the text and the remark attribute of the a tag corresponding to each URL link that jumps to the external advertisement is taken out, which is the advertisement text with the URL link.
[0050] Step 104: Based on a preset advertisement illegal word data set, determine whether the advertisement text contains illegal words. If so, output the current illegal words and the illegal advertisement URL corresponding to the current illegal words.
[0051] In this step, for each original web page advertisement text corresponding to the URL link that jumps to the external advertisement, it is determined whether it contains words in the pre-set advertisement violation word data set. If it does, the text is considered to be a violation text, the URL link it jumps to is a violation advertisement link, and the violation text and the violation advertisement URL corresponding to the current violation text are output. Among them, the preset violation advertisement word data set is calculated based on the easily influenced characteristics of the current web page, and the easily influenced characteristics of the current web page are the degree to which the subject content of the current web page is easily influenced by the subject content of other web pages.
[0052] For example, the subject content of the current webpage is "learning and education", and after analysis, it is found that the advertising text existing in the current webpage is "the most beautiful cake shop", and the preset advertising violation word data set that matches the subject content of "learning and education" of the current webpage includes "the most beautiful cake shop", then "the most beautiful cake shop" is the illegal advertising text in the current webpage, and clicking on the five words "the most beautiful cake shop" can jump to the link on the promotional webpage of a certain cake shop, which is the illegal advertising URL.
[0053] As can be seen from the above technical solution, the embodiment of the present application first obtains the user's domain name system DNS access record, determines whether the web page corresponding to the DNS access record is valid, and if valid, crawls the source code of the web page, otherwise discards the current DNS access record; and then determines the advertisement uniform resource locator URL contained in the source code according to the source code, and determines the advertisement text corresponding to the advertisement URL according to the advertisement URL; finally, according to the preset advertisement violation word data set, determines whether the advertisement text contains violation text, and if so, outputs the current violation text and the violation advertisement URL corresponding to the current violation text. It can be seen that the embodiment of the present application provides a method for identifying violation advertisements for massive streaming data, which saves the time of calculating advertisement links in web pages by eliminating inaccessible web pages in user access records on the one hand. On the other hand, the advertisement links and advertisement text are extracted through the web page source code structure to identify the violation advertisements in the web page, which has high accuracy and strong practicality.
[0054] Based on the content of the above embodiment, in this embodiment, before judging whether the webpage corresponding to the DNS access record is valid, it also includes:
[0055] The DNS access records are deduplicated to remove redundant DNS access records and inaccessible domain names.
[0056] Based on the content of the above embodiment, in this embodiment, determining the advertisement URL contained in the source code according to the source code includes:
[0057] Extracting all web page URLs and web page URLs contained in inline frame elements in the source code;
[0058] Based on the web page URL, determine the advertisement URL contained in the source code.
[0059] Based on the content of the above embodiment, in this embodiment, determining the advertisement URL contained in the source code according to the webpage URL includes:
[0060] Based on the length of the web page URL and / or the domain name combination form of the web page URL, determine whether the web page URL is a normal jumpable link. If so, determine that the current web page URL is a non-advertising URL; otherwise, the current web page URL is an advertising URL.
[0061] Based on the content of the above embodiment, in this embodiment, determining the advertisement text corresponding to the advertisement URL according to the advertisement URL includes:
[0062] The advertisement text corresponding to the advertisement URL is determined according to the text of the a tag in the webpage source code corresponding to the advertisement URL and the text appearing in the remark attribute.
[0063] Based on the content of the above embodiment, in this embodiment, the preset advertising violation word data set is calculated based on the easily influenced characteristics of the current web page; the easily influenced characteristics of the current web page are the degree to which the theme of the current web page is easily influenced by the theme content of other web pages.
[0064] The present application will be described in detail below through specific embodiments.
[0065] First Embodiment:
[0066] In this embodiment, as shown in Figure 2, the method for identifying illegal advertisements provided by the embodiments of the present application includes a cleaning module for massive streaming data and an illegal advertisement detection module for web pages. The method specifically includes:
[0067] S1: Input data: DNS access records of a large number of users daily;
[0068] S2: Remove duplicate access records and CDN records;
[0069] S3: Determine whether the web page corresponding to each DNS access record can be accessed. If it can be accessed, send it to the illegal advertisement detection module; otherwise, discard the current access record;
[0070] S4: The crawler program extracts the web page source code;
[0071] S5: Extract all URLs in the source code and URLs in the <iframe> frame;
[0072] S6: Determine the length and domain name combination form of the extracted URLs, retain the URLs that conform to the advertisement link URLs, and discard the rest;
[0073] S7: Extract the text on the link, and take the union of the text and the note attribute of the a tag;
[0074] S8: Search for the preset set of illegal advertising words and determine whether the text on the link belongs to the illegal advertising words;
[0075] S9: If the text on the link belongs to the illegal advertising words, output the illegal advertisement link and the illegal advertisement text of the URL.
[0076] Based on the same inventive concept, another embodiment of the present invention provides an illegal advertisement recognition device. As shown in Figure 3, an illegal advertisement recognition device provided by an embodiment of the present application includes:
[0077] A first processing module 1, configured to obtain the Domain Name System (DNS) access record of a user, determine whether the web page corresponding to the DNS access record is valid, and if valid, crawl the source code of the web page;
[0078] A second processing module 2, configured to determine the advertisement Uniform Resource Locator (URL) included in the source code according to the source code;
[0079] A third processing module 3, configured to determine the advertisement text corresponding to the advertisement URL according to the advertisement URL;
[0080] A fourth processing module, configured to determine whether the advertisement text contains illegal text according to a preset advertisement illegal word data set. If so, output the current illegal text and the illegal advertisement URL corresponding to the current illegal text.
[0081] In this embodiment, since the web page access data of different users is large every day and new access data is generated every day, the Domain Name System (DNS) access record of daily users can be regarded as streaming data, specifically referring to receiving new data every day, and the amount of user access records within a single day is huge, which can be called massive data. Therefore, the continuously incoming daily input data is classified as massive streaming data.
[0082] Among them, streaming data refers to data continuously generated by thousands of data sources, usually also sent in the form of data records, with a relatively small scale (about several kilobytes). Streaming data includes various types of data, such as log files generated by web applications, online shopping data, in-game player activities, social networking site information, financial trading hall or geospatial services, and telemetry data from connected devices or instruments within a data center. Such data needs to be processed incrementally in sequence according to records or according to a sliding time window, and can be used for various analyses, including association, aggregation, filtering, and sampling.The information derived from such analysis provides companies with insights into all aspects of their business and customer activity, such as service usage (for metering / billing), server activity, website hits, and the geographic location of devices, people, and objects, allowing them to respond quickly to new situations. For example, a company can continuously analyze social media streams to track changes in public perception of its brand and products and respond promptly when necessary.
[0083] Among them, the domain name system is a service provided to people on the Internet. DNS is actually a special type of distributed database that can map domain names and IP addresses to each other, and its value lies in the convenience and speed it brings. Users can easily perform network queries without knowing the principles behind it. Domain name resolution is to point a domain name to its corresponding website space IP address, so that network users can achieve their desire to access specific web pages only through the registered domain name.
[0084] In this embodiment, for massive streaming Uniform Resource Locator URL (Uniform Resource Locator) access data, first obtain the DNS access records of massive users, and perform domain name deduplication operation on them to eliminate redundant DNS access records. It is understandable that since the websites that users may visit are repeated, and multiple judgments on the same website greatly consume computing resources and waste computing time, so the deduplication operation is very necessary. In addition, optionally, inaccessible web pages in the user's DNS access record can also be eliminated, for example, domain names containing "CDN" in the domain name, which are inaccessible. The above method not only ensures that the DNS access record data is not redundant, but also simplifies the operation of the subsequent identification step.
[0085] In this embodiment, for the DNS access records after deduplication, it is also possible to determine whether the web page corresponding to each DNS access record can be opened, and the web pages that can be accessed will be retained as the output results, and the rest will be discarded. This is because the user may want to access some randomly generated websites or mistype the URL of the website, and such websites themselves cannot access the specific content, so there is no benefit in analyzing such websites. Therefore, the use of this method further saves computing time, so that computing resources can be used more reasonably.
[0086] In this embodiment, after obtaining a non-repetitive and valid web page, a crawler program is used to crawl the source code of the web page, and then the web page source code is analyzed. It is believed that the web page URL contained in the web page source code may be an advertising link. Therefore, all web page URLs in the web page source code are extracted. Since only URL jumps on the web page can be extracted, floating window advertisements cannot be extracted. Therefore, the web page URLs in the web page source code are extracted.<iframe> The frame also adds the URL of the web page contained in it to the list of URLs to be detected.
[0087] In this embodiment, for each URL in the list of URLs to be detected, two steps of screening can be performed. First, the length of the URL is determined, and URLs with a length of less than 150 are regarded as normal in-site jump links and can be removed from the list. Then, the remaining URLs are judged for the second time, and URLs containing the combination of "top-level domain name + first-level domain name" of the original website are regarded as normal in-site jump links. These are removed, and the remaining ones are advertising URLs.
[0088] In this embodiment, since the text on the advertisement URL appears in two locations in the webpage source code, one is the text of the a tag and the other is the remark attribute of the a tag, and the contents of the two are often repeated. Therefore, the union of the text and remark attribute of the a tag corresponding to each URL link that jumps to the external advertisement is taken out, which is the advertisement text with the URL link.
[0089] In this embodiment, for each original web page advertisement text corresponding to the URL link that jumps to the external advertisement, it is determined whether it contains words in the pre-set advertisement violation word data set. If it does, the text is considered to be a violation text, the URL link it jumps to is a violation advertisement link, and the violation text and the violation advertisement URL corresponding to the current violation text are output. Among them, the preset advertisement violation word data set is calculated based on the easily influenced characteristics of the current web page, and the easily influenced characteristics of the current web page are the degree to which the subject content of the current web page is easily influenced by the subject content of other web pages.
[0090] For example, the subject content of the current webpage is "learning and education", and after analysis, it is found that the advertising text existing in the current webpage is "the most beautiful cake shop", and the preset advertising violation word data set that matches the subject content of "learning and education" of the current webpage includes "the most beautiful cake shop", then "the most beautiful cake shop" is the illegal advertising text in the current webpage, and clicking on the five words "the most beautiful cake shop" can jump to the link on the promotional webpage of a certain cake shop, which is the illegal advertising URL.
[0091] As can be seen from the above technical solution, the embodiment of the present application first obtains the user's domain name system DNS access record, determines whether the web page corresponding to the DNS access record is valid, and if so, crawls the source code of the web page, otherwise discards the current DNS access record; then, according to the source code, determines the advertisement uniform resource locator URL contained in the source code, and according to the advertisement URL, determines the advertisement text corresponding to the advertisement URL; finally, according to the preset advertisement violation word data set, determines whether the advertisement text contains violation text, and if so, outputs the current violation text and the violation advertisement URL corresponding to the current violation text. It can be seen that, on the one hand, the embodiment of the present application saves the time of calculating the advertisement link in the web page by eliminating the inaccessible web pages in the user access record. On the other hand, the advertisement link and advertisement text are extracted through the web page source code structure to identify the violation advertisement in the web page, which has high accuracy and strong practicality.
[0092] Based on the content of the above embodiment, in this embodiment, the second processing module is specifically used for:
[0093] Extracting all web page URLs and web page URLs contained in inline frame elements from the source code;
[0094] Based on the web page URL, determine the advertisement URL contained in the source code.
[0095] The illegal advertisement identification device described in this embodiment can be used to execute the above method embodiment. Its principles and technical effects are similar and will not be repeated here.
[0096] Based on the same inventive concept, another embodiment of the present invention provides an electronic device. Referring to the schematic structural diagram of the electronic device shown in Figure 4, it specifically includes the following components: a processor 401, a memory 402, a communication interface 403, and a communication bus 404;
[0097] Among them, the processor 401, the memory 402, and the communication interface 403 complete mutual communication through the communication bus 404; the communication interface 403 is used to implement information transmission between devices;
[0098] The processor 401 is used to call the computer program in the memory 402. When the processor executes the computer program, it implements all the steps of the above-mentioned method for identifying illegal advertisements. For example, it obtains the Domain Name System (DNS) access record of the user, determines whether the web page corresponding to the DNS access record is valid. If it is valid, it crawls the source code of the web page. Otherwise, it discards the current DNS access record. Based on the source code, it determines the advertisement Uniform Resource Locator (URL) included in the source code; according to the advertisement URL, it determines the advertisement text corresponding to the advertisement URL; according to the preset advertisement violation word data set, it determines whether the advertisement text contains violation words. If so, it outputs the current violation word and the violation advertisement URL corresponding to the current violation word.
[0099] Based on the same inventive concept, another embodiment of the present invention provides a non-transitory computer-readable storage medium. A computer program is stored on this computer-readable storage medium. When the computer program is executed by a processor, it implements all the steps of the above-mentioned method for identifying illegal advertisements. For example, it obtains the Domain Name System (DNS) access record of the user, determines whether the web page corresponding to the DNS access record is valid. If it is valid, it crawls the source code of the web page. Otherwise, it discards the current DNS access record. Based on the source code, it determines the advertisement Uniform Resource Locator (URL) included in the source code; according to the advertisement URL, it determines the advertisement text corresponding to the advertisement URL; according to the preset advertisement violation word data set, it determines whether the advertisement text contains violation words. If so, it outputs the current violation word and the violation advertisement URL corresponding to the current violation word.
[0100] In addition, when the logical instructions in the above-mentioned memory can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium.Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., various media that can store program codes.
[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. Those of ordinary skill in the art may understand and implement them without creative work.
[0102] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the illegal advertisement identification method described in each embodiment or some parts of the embodiment.
[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.< / iframe>
Claims
1. A method for identifying illegal advertisements, characterized in that: include: Obtaining the DNS access records of a large number of users, determining whether the webpage corresponding to the DNS access record is valid, and if so, crawling the source code of the webpage; Determining, according to the source code, an advertisement URL contained in the source code; Determine, according to the advertisement URL, the advertisement text corresponding to the advertisement URL; According to a preset advertisement illegal word data set, determining whether the advertisement text contains illegal words, and if so, outputting the current illegal words and the illegal advertisement URL corresponding to the current illegal words, wherein the preset advertisement illegal word data set is calculated based on the easily affected characteristics of the current web page; Determining, according to the source code, an advertisement URL contained in the source code, including: Extracting all web page URLs and web page URLs contained in inline frame elements in the source code; Determining the advertisement URL contained in the source code according to the webpage URL; Wherein, determining the advertisement URL contained in the source code according to the webpage URL includes: According to the length of the web page URL and / or the domain name combination form of the web page URL, it is determined whether the web page URL is a normal jumpable link. If so, it is determined that the current web page URL is a non-advertisement URL, otherwise the current web page URL is an advertisement URL.
2. The method for identifying illegal advertisements according to claim 1, characterized in that: Before determining whether the webpage corresponding to the DNS access record is valid, the method further includes: The DNS access records are deduplicated to remove redundant DNS access records and inaccessible domain names.
3. The method for identifying illegal advertisements according to claim 1, characterized in that: Determining, according to the advertisement URL, the advertisement text corresponding to the advertisement URL, including: The advertisement text corresponding to the advertisement URL is determined according to the text of the a tag in the webpage source code corresponding to the advertisement URL and the text appearing in the remark attribute.
4. The method for identifying illegal advertisements according to claim 1, characterized in that: The preset advertisement violation word data set is calculated based on the easily influenced characteristic of the current webpage; the easily influenced characteristic of the current webpage is the degree to which the subject of the current webpage is easily influenced by the subject content of other webpages.
5. A device for identifying illegal advertisements, characterized in that: include: The first processing module is used to obtain the domain name system DNS access records of a large number of users, determine whether the web page corresponding to the DNS access record is valid, and if valid, crawl the source code of the web page; A second processing module, configured to determine an advertisement URL contained in the source code according to the source code; A third processing module, used to determine the advertisement text corresponding to the advertisement URL according to the advertisement URL; a fourth processing module, configured to determine whether the advertisement text contains illegal words according to a preset advertisement illegal words data set, and if so, output the current illegal words and the illegal advertisement URL corresponding to the current illegal words, wherein the preset advertisement illegal words data set is calculated according to the easily affected characteristics of the current web page; The second processing module is specifically used for: Extracting all web page URLs and web page URLs contained in inline frame elements in the source code; Determining the advertisement URL contained in the source code according to the webpage URL; Wherein, determining the advertisement URL contained in the source code according to the webpage URL includes: According to the length of the web page URL and / or the domain name combination form of the web page URL, it is determined whether the web page URL is a normal jumpable link. If so, it is determined that the current web page URL is a non-advertisement URL, otherwise the current web page URL is an advertisement URL.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the illegal advertisement identification method as described in any one of claims 1-4 are implemented.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the illegal advertisement identification method according to any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Violation propaganda method and device and computer readable storage medium
CN108829680A
Page monitoring method and device and computer storage medium
CN109740094A
Illegal content identification method and device, electronic equipment and storage medium
CN111859234A
