Internet-oriented illegal advertising identification method, device and system

Abstract

The present invention relates to the field of text processing technology, and in particular to an Internet-oriented illegal advertisement identification method, device and system. The method specifically analyzes the text information data carried by the advertisement, compares and analyzes the high distribution consistency of the part of speech and word frequency of each word in the local text before and after the special symbol processing, as well as the similarity of the entire text, and combines the connection and association of the words before and after the special symbol to obtain the necessary processing degree of the special symbol; through the processed text data combined with the abnormal possibility of the image and the release time, the violation score of the advertisement is evaluated for identification, and a more accurate evaluation result is obtained. The present invention processes the interfering special symbols carried in the advertisement text by comparing the semantic changes represented by the coherent text content before and after the special symbol processing, thereby improving the accuracy of identifying illegal advertisement information and making the identification of illegal advertisements more comprehensive and reliable.

Description

Technical Field

[0001] The present invention relates to the field of text processing technology, and in particular to a method, device, and system for identifying illegal advertisements on the Internet. Background Art

[0002] With the rapid development of the internet ecosystem, businesses and enterprises are investing increasing resources online, leading to a continuous increase in the number of advertisements on the internet. Some unscrupulous businesses and enterprises are taking advantage of this opportunity to flood the internet with illegal advertisements, mixing them with legitimate advertising, engaging in false propaganda and vulgar content, and seriously undermining consumer trust. These illegal advertisements require automated identification, blocking, and interception to protect consumer rights.

[0003] In the application of illegal advertising identification technology on the Internet, some bad advertising businesses, in order to pass illegal identification, will use methods such as text distortion, character obstruction or splitting sensitive words to bypass keyword detection when placing advertisements on the Internet, resulting in the failure of illegal advertising identification, and thus flowing into the Internet, causing adverse effects on Internet users. Summary of the Invention

[0004] The present application provides an Internet-oriented illegal advertisement identification method, device and system. The method eliminates the obstructing special symbols that may be carried in the advertisement text, thereby solving the problem that advertisers deliberately insert special symbols in advertisements to bypass keyword detection and interfere with the identification of illegal advertisements, thereby making the identification of illegal information in advertisements more accurate.

[0005] The first aspect of the present application provides a method for identifying illegal advertisements on the Internet, comprising:

[0006] Collect advertisements and perform pre-processing to obtain image data and text data of each advertisement; determine each special character in the text data;

[0007] Segment the local text data before and after each special character is symbolized, and obtain the key degree of each segmented word in the local text data before and after the symbolization based on the part of speech and word frequency of each segmented word; obtain the consistency of the processed text for each special character based on the high distribution approximation of the segmented key degrees in the local text data before and after the symbolization of each special character;

[0008] The connection relevance of each special character is obtained based on the connection correlation degree and consistent distribution degree between each special character and the preceding and following word segments; the similarity between the local text data before and after the symbol processing of each special character is analyzed, and the necessity of processing each special character is obtained by combining the consistency of the processed text and the connection relevance; the special characters in the text data are eliminated based on the degree of necessity of processing to obtain the processed text data;

[0009] The processed text data and image data of each advertisement are analyzed for the degree of violation, and combined with the degree of abnormality of the advertisement delivery time, a violation score for each advertisement is obtained; violations are identified based on the advertisement violation score.

[0010] Furthermore, the method for obtaining the criticality includes:

[0011] For any special character, the local text data before and after symbol processing of the character are respectively segmented to obtain segmented words; the key weight of each segmented word is determined according to the part of speech of each segmented word;

[0012] For any word segment, the frequency of occurrence of the word segment in the text data of the advertisement is used as the word frequency of the word segment; the product of the value of the negative correlation mapping of the word segment and the key weight is used as the key degree of the word segment.

[0013] Furthermore, the method for obtaining the consistency of the processed text includes:

[0014] For any special character, all the segmented words in the local text data before the character is symbolized are arranged from large to small according to the key degree to obtain a key sequence; the first preset number of segmented words in the key sequence are used as the front key segmented words before the character is symbolized; similarly, the back key segmented words after the character is symbolized are obtained based on the local text data after the character is symbolized;

[0015] Count the data of the same segmentation in the preceding key segmentation and the following key segmentation of the character and perform normalization processing to obtain the segmentation distribution approximation of the character;

[0016] After calculating the key difference between the first key word and the second key word with the same sequence number in the key sequence, the sum of all differences is calculated and negative correlation mapping is performed to obtain the key similarity of the word of the character;

[0017] The processed text consistency of the character is obtained by combining the word segmentation distribution approximation and word segmentation high key similarity of the character.

[0018] Furthermore, the method for obtaining the connection correlation degree includes:

[0019] For any special character, the previous segmentation word adjacent to the character in the local text data is used as the front segmentation word of the character, and the next segmentation word adjacent to the character in the local text data is used as the back segmentation word of the character;

[0020] Get the sentiment scores of the front participle and the back participle respectively, and negatively map the difference in sentiment scores between the front participle and the back participle to obtain the connected sentiment correlation of the character;

[0021] After counting the frequency of occurrence of the front participle and the character in the text data of the advertisement, and the frequency of occurrence of the character and the back participle in the text data of the advertisement, the sum of the two frequency values ​​is used as the phrase consistency of the character;

[0022] The connection relevance of the character is obtained by combining the connection sentiment relevance and phrase consistency of the character.

[0023] Furthermore, the method for obtaining the processing necessity includes:

[0024] For any special character, calculate the similarity between the local text data of the character before symbol processing and the local text data after symbol processing to obtain the local text similarity index of the character;

[0025] The product of the value of negative correlation mapping of the connection relevance of the character, the local text similarity index, and the consistency of the processed text is used as the necessity of processing the character.

[0026] Furthermore, the method for obtaining processed text data includes:

[0027] In the text data of the advertisement, special characters whose processing necessity is greater than a preset processing threshold are eliminated, and otherwise, symbol processing is performed to obtain processed text data of the advertisement.

[0028] Furthermore, the method for obtaining the advertising violation score includes:

[0029] For any advertisement, the image data of the advertisement is input into the trained image violation scoring model, and the image violation score of the advertisement is output; based on the preset violation word library, the proportion of violation words in the processed text data of the advertisement is used as the text violation score of the advertisement;

[0030] Based on the degree of proximity between the ad delivery time and the preset abnormal time, the delivery period abnormality of the ad is obtained; the delivery frequency of the same type of ads in the preset local time period of each delivery is used as the type frequency abnormality of the ad; combining the delivery period abnormality and type frequency abnormality of the ad, the delivery time abnormality index of the ad is obtained;

[0031] The ad violation score is obtained by taking a weighted sum of the ad's image violation score, text violation score, and delivery time anomaly index.

[0032] Furthermore, the violation identification based on the violation score of the advertisement includes:

[0033] When the ad violation score is greater than the preset violation threshold, the corresponding ad will be marked as a violation ad;

[0034] When the ad violation score is lower than the preset non-violation threshold, the corresponding ad will be recorded as a non-violation ad;

[0035] Otherwise, the advertisement will be manually reviewed for violation identification; the preset non-violation threshold is less than the preset violation threshold.

[0036] In a second aspect, the present application provides an Internet-based illegal advertisement identification system, the system comprising:

[0037] Advertisement collection unit, used to collect advertisements and perform pre-processing to obtain image data and text data of each advertisement; determine each special character in the text data;

[0038] The advertising text processing unit is used to segment each special character in the local text data before and after the symbol processing, and obtain the key degree of each segmented word in the local text data before and after the symbol processing based on the part of speech and word frequency of each segmented word; and obtain the processed text consistency of each special character based on the high distribution approximation of the segmented key degree in the local text data before and after the symbol processing of each special character;

[0039] The connection relevance of each special character is obtained based on the connection correlation degree and consistent distribution degree between each special character and the preceding and following word segments; the similarity between the local text data before and after the symbol processing of each special character is analyzed, and the necessity of processing each special character is obtained by combining the consistency of the processed text and the connection relevance; the special characters in the text data are eliminated based on the degree of necessity of processing, and the processed text data is obtained;

[0040] The advertising violation identification unit is used to analyze the degree of violation of the processed text data and image data of each advertisement, and obtain a violation score for each advertisement based on the abnormality of the advertisement delivery time; and perform violation identification based on the advertisement violation score.

[0041] In the third aspect, the present application provides an Internet-oriented illegal advertising identification device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method of the first aspect or any embodiment of the first aspect of the present application is implemented.

[0042] In a fourth aspect, the present application provides a computer program product, comprising a computer program code. When the computer program code is executed, the method of the first aspect or any embodiment of the first aspect of the present application is performed.

[0043] In a fifth aspect, the present application provides a computer-readable storage medium, which stores computer program code. When the computer program code is executed, it performs the method of the first aspect of the present application or any embodiment of the first aspect.

[0044] The present invention has the following beneficial effects:

[0045] The present invention specifically analyzes the text information data carried by the advertisement, compares and analyzes the high distribution consistency of the part of speech and word frequency of each word in the local text before and after the special symbol processing, as well as the similarity of the overall text, combined with the connection association of the words before and after the special symbol, and by comparing the consistency of the key subjects represented by the text before and after the special symbol processing, analyzes the degree of interference caused by the special symbol, and the degree to which the special symbol destroys the semantics of the connection between the word segments on both sides, and obtains the necessary processing degree of the special symbol, providing more accurate text information for the subsequent identification of illegal information. By combining the processed text data with the possibility of abnormality of the image and the release time, the violation score of the advertisement is evaluated for identification, and a more accurate evaluation result is obtained. The present invention processes the interfering special symbols carried in the advertisement text by comparing the semantic changes represented by the coherent text content before and after the special symbol processing, improves the accuracy of identifying illegal information of advertisements, and makes the identification of illegal advertisements more comprehensive and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of a method for identifying illegal advertisements on the Internet provided by one embodiment of the present invention;

[0048] Figure 2 This is a structural diagram of an Internet-based illegal advertisement identification system provided by one embodiment of the present invention;

[0049] Figure 3 A schematic diagram of the structure of an Internet-based illegal advertisement identification device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation methods, structures, features and effects of an Internet-based illegal advertising identification method, device and system proposed by the present invention. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0051] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0052] The following describes in detail a specific solution of an Internet-based illegal advertisement identification method, device, and system provided by the present invention with reference to the accompanying drawings.

[0053] This application embodiment provides a method for identifying illegal advertisements on the Internet. Figure 1 , which shows a flow chart of a method for identifying illegal advertisements on the Internet provided by an embodiment of the present invention, the method comprising the following steps:

[0054] S1: Collect advertisements and perform pre-processing to obtain image data and text data of each advertisement; determine each special character in the text data.

[0055] Ad violation identification primarily relies on analyzing and identifying specific advertisements on the internet to identify violations. Since internet ads are distributed across static web pages, dynamically rendered pages, and in-app ads, ad violation identification cannot directly apply to all ad data. Therefore, proactively collecting ad content from multiple channels is necessary to identify violations.

[0056] In an embodiment of the present invention, advertising information from multiple channels is captured by using a crawling tool, and relevant illegal advertising information is obtained by marking violations by users. The crawling tool can use tools such as Scrapy and Selenium, and multi-channel advertising information includes advertising content from mainstream websites, forums or social media. It can also capture advertisements within APPs through Android emulators, parse APK files to extract advertising SDK interfaces to obtain advertisements, etc. It should be noted that when using crawler tools to capture advertising information in multiple channels, it should be ensured that the law and the Robots protocol are complied with, and personal privacy data should be avoided. The process does not violate relevant laws and regulations, and does not violate public order and good morals. The specific advertising collection method is a technical means well known to those skilled in the art and will not be elaborated here.

[0057] In the process of identifying advertising violations, sentiment dictionaries and the like are usually used to analyze the semantic sentiment contained in the text. However, in the process of identifying illegal advertisements, there are special symbols in the text, and these special symbols are not included in the Chinese sentiment dictionary. The special symbols include meaningful symbols and meaningless symbols. For example, "*" or "-" are direct interference words, and the interference generated is like "low * vulgar", and at this time, it can be directly eliminated for sensitive information analysis.

[0058] However, there are also symbols that contain distorted interference information. For example, " " Such special characters have a replacement meaning. After replacement processing, it is "snacks and private message to receive benefits". At this time, if the characters are directly eliminated, it will become "point and private receive benefits", which will cause a large fuzzy impact on sensitive information and violate the analysis. Therefore, the characters that can be replaced need to be further analyzed for their impact on semantic understanding, and the characters with information impact are retained.

[0059] To make the processing of special characters more accurate, in the embodiment of the present invention, first, the advertisement is split multimodally, that is, the image data and text data of each advertisement are separated, and the advertisement text data is preprocessed. The preprocessing includes simplified and traditional processing and translation processing to unify the text data into simplified Chinese content for the accuracy of subsequent analysis. Then, the special characters in the preprocessed text data are determined to facilitate the subsequent analysis of the impact of special characters in the text. It should be noted that preprocessing and special character determination are well-known technical means in the art. For example, special character determination can be performed by matching the symbol dictionary through regular expressions, etc., and will not be elaborated and restricted here.

[0060] S2: Perform word segmentation on the local text data before and after symbol processing for each special character, and obtain the key degree of each word segment in the local text data before and after symbol processing according to the word nature and word frequency degree of each word segment; according to the high distribution approximation of the word segment key degrees in the local text data before and after symbol processing for each special character, obtain the processing text consistency of each special character.

[0061] First, based on the symbol classification dictionary, different symbol processing methods are defined. For example, characters without literal meaning such as " " can be directly deleted, but for those with literal replacement meaning such as " ", the special characters can be symbolically replaced through the dictionary to replace them with literal meanings. For example, " " is replaced by "heart", " [[ID=XX]]" is replaced by "star", " " is replaced by "8", etc. Therefore, the symbol processing of each special character is based on the symbol classification dictionary.

[0062] By analyzing the changes in the semantic representation of the text before and after symbol processing, it is determined whether special characters affect the expression of text data. Considering that the meaning expression in text data is conveyed by words of various词性, word segmentation analysis is performed on the local text of special text. The key degree of each word segmentation expression information is reflected by the word segmentation information in the local text data segment where each special sentence is located. To ensure the analyzability of local text data, in the embodiments of the present invention, the local text data of special characters refers to the data with coherent text before and after special characters, that is, the text data composed of the sentence where the special character is located, the previous sentence and the subsequent sentence of the sentence where the special character is located is used as the local text data of each special character to represent a local literal meaning.

[0063] Preferably, in the embodiments of the present invention, the method for obtaining the key degree includes:

[0064] For any special character, the local text data before symbol processing and the local text data after symbol processing of the character are respectively subjected to word segmentation processing to obtain word segments. The key weight of each word segment is determined according to the词性 of each word segment. In the embodiments of the present invention, the BERT model can be used to perform word segmentation and词性 annotation on text data. The word segmentation and词性 analysis method is a well-known technical means for those skilled in the art and will not be elaborated here. By assigning different key weights to different词性, it can be understood that after word segmentation of the local text data before and after symbol processing, the word segmentation results may be the same or different.

[0065] In the embodiments of the present invention, since key information generally appears as object > predicate > subject > others, noun > verb > preposition > others, the corresponding key weight values are set as: 0.8 > 0.5 > 0.3 > 0.1, 0.8 > 0.5 > 0.3 > 0.1. For example, "Add WeChat to get free courses" highlights WeChat and courses, and "Scan the code and immediately grab a 50% discount" highlights scanning the code and a 50% discount. The specific implementation of assigning key weights can be adjusted by the implementer according to the specific implementation scenario and will not be limited here.

[0066] For any word segment, the frequency of occurrence of the word segment in the text data of the advertisement where the word segment is located is used as the word frequency of the word segment, which reflects the high-frequency degree of the corresponding word segment in the advertisement. The higher the frequency of the word segment, the less key information it may carry. For example, for high-frequency words that often appear in the text, such as "get", "of", etc., these words generally do not serve as the subject and predicate of the noun, so they carry less key information and are given a lower key degree. On the contrary, for low-frequency words such as "WeChat" and "courses", they generally serve as the subject and predicate of the noun, so they carry more key information related to the advertisement and are given a higher key degree.

[0067] Therefore, the product of the negative correlation mapping value of the word frequency and the key weight is used as the key level of the word. The lower the word frequency and the higher the key weight, the more critical the information contained in the advertisement text. It should be noted that negative correlation mapping is a technical method well known to those skilled in the art, such as using an inverse proportional or negative exponential form, and is not limited or elaborated on here.

[0068] Through the criticality, we can analyze the changes in higher criticality word segments before and after symbol processing to reflect whether the special symbol has a key impact on the expression of the text segment. The higher the criticality of the word segment, the more consistent the expression subject before and after symbol processing, which means that the special symbol has little influence.

[0069] Preferably, in an embodiment of the present invention, by analyzing the high distribution of word segmentation keyness before and after processing, the consistency of the processed text is obtained, and the method for obtaining the consistency of the processed text includes:

[0070] For any special character, all the segmented words in the local text data before the character is symbolized are arranged from large to small according to the criticality to obtain a key sequence. The first preset number of segmented words in the key sequence are used as the front key segmented words before the character is symbolized. Several segmented words with higher criticality are determined by sorting for subsequent approximate analysis. In an embodiment of the present invention, the preset number is 3, that is, the first 3 segmented words from the key sequence are taken as the front key segmented words.

[0071] Similarly, based on the local text data after the character is symbolized, the post-key word segmentation of the character after symbolization is obtained, that is, all the word segments in the local text data after the character is symbolized are arranged from large to small according to the key degree to obtain a key sequence, and the first preset number of word segments in the key sequence are used as the post-key word segmentation of the character after symbolization to ensure the consistency of the preset number selection, and the same number of word segments are taken for approximate analysis.

[0072] Furthermore, the data of the same segmentation in the preceding key segmentation and the following key segmentation of the character are counted and normalized to obtain the segmentation distribution approximation of the character. The more similar the high-key segmentation content before and after the symbol processing is, that is, the more identical segmentations there are in the preceding key segmentation and the following key segmentation, the smaller the impact of the special symbol processing.

[0073] It should be noted that normalization is a technical means well known to those skilled in the art. The normalization options may be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0074] Furthermore, after calculating the difference in the key degree between the previous key segmentation and the subsequent key segmentation with the same sequence number in the key sequence, the sum of all differences is calculated and negative correlation mapping is performed to obtain the high key similarity of the segmentation of the character. According to the order in the key sequence, the key degree differences between the segmentations with the same sequence number in the key sequence are calculated in sequence. In an embodiment of the present invention, the key degree difference between the first segmentation, the key degree difference between the second segmentation and the key word difference between the third segmentation are calculated respectively between the two key sequences, and then the sum of the three differences is calculated to reflect the deviation between the key degrees of the high distribution. The smaller the deviation, the more consistent the distribution. Therefore, the high key similarity of the segmentation is obtained through negative correlation mapping.

[0075] Finally, the processed text consistency of the character is obtained by combining the word segmentation distribution approximation and the word segmentation high-key similarity of the character. In an embodiment of the present invention, the product of the word segmentation distribution approximation and the word segmentation high-key similarity is used as the processed text consistency of the character. When the word segmentation distribution approximation and the word segmentation high-key similarity are higher, it means that the expression of the key content of the text before and after processing is more consistent, and the interference effect of the special character is not high.

[0076] S3: Obtain the connection correlation of each special character based on the connection correlation degree and consistent distribution degree between each special character and the preceding and following word segmentations; analyze the similarity between the local text data before and after the symbol processing of each special character, and obtain the necessity of processing each special character by combining the consistency of the processed text and the connection correlation; perform elimination processing based on the size of the necessity of processing special characters in the text data to obtain the processed text data.

[0077] When the text before and after processing is highly consistent, some special symbols may act as modifiers to the words they connect to. Even if these special symbols have minimal influence on the text, removing them can still affect sentence coherence analysis, such as when some numbers represent symbols. Therefore, we further analyze the relationship between each special symbol and the surrounding words, improving the analysis of the special symbol's impact through the use of sentimental semantics.

[0078] Preferably, in an embodiment of the present invention, the connection relevance is obtained by determining the connection correlation and the consistent distribution between the special character and the preceding and following participles. The method for obtaining the connection relevance includes:

[0079] For any special character, the previous segmentation adjacent to the character in the local text data is used as the front segmentation of the character, and the next segmentation adjacent to the character in the local text data is used as the back segmentation of the character. The segmentations before and after each special character are marked separately, and the connection situation is analyzed.

[0080] The sentiment scores of the front participle and the back participle are obtained respectively, and the difference in sentiment scores between the front participle and the back participle is negatively correlated to obtain the connection sentiment correlation of the character. The connection of the word sentiment reflects the degree of sentiment semantic correlation of the text. For example, the sentiment semantics of "free" is positive, with a sentiment score of +0.8, while the sentiment semantics of "fraud" is negative, with a sentiment score of -0.9. The greater the difference in sentiment scores, the less relevant the semantic coherence. Therefore, the negative correlation of the difference in situation scores is used to reflect the correlation of the connection sentiment. In this embodiment, the sentiment score is obtained using the textblob library, and can also be obtained through a pre-trained neural network model. The acquisition method is not repeated here and is not limited.

[0081] Furthermore, after counting the frequency of occurrence of the front participle and the character in the text data of the advertisement, as well as the frequency of occurrence of the character and the back participle in the text data of the advertisement, the sum of the two frequencies is used as the phrase consistency of the character. When a special symbol frequently appears with a certain continuous participle, it reflects that the correlation between the special symbol and the participle is high, and removing it may have an impact. Therefore, the synchronization frequency of the special character and the front participle, and the synchronization frequency of the special character and the back participle are counted separately, and the possible phrase consistency of the special character is obtained by summing up the frequencies.

[0082] Finally, the connection correlation of the character is obtained by combining the connection sentiment correlation and phrase consistency of the character. In an embodiment of the present invention, the product of the connection sentiment correlation and phrase consistency of the character is used as the connection correlation of the character. The greater the connection sentiment correlation and phrase consistency, the higher the relevance of the special character to the previous and next words, and the greater the possibility of elimination of the impact.

[0083] Therefore, from a global analysis, the necessity of processing each special character is obtained by combining the overall semantic similarity of the text before and after symbol processing and the consistency and connection correlation of the processed text. In an embodiment of the present invention, the method for obtaining the necessity of processing includes:

[0084] For any special character, the similarity between the local text data before and after symbol processing is calculated to obtain a local text similarity index for the character. Text vector conversion is performed on the local text data before and after symbol processing, respectively. After obtaining two text vectors, the cosine similarity between the text vectors is calculated to obtain the local text similarity index. It should be noted that obtaining text vectors and calculating cosine similarity are both well-known techniques to those skilled in the art, such as using a sampled word embedding model for vector conversion, and are not further elaborated or limited herein.

[0085] Then, the value of the negative correlation mapping of the connection correlation of the character is multiplied by the local text similarity index and the processed text consistency as the necessity of processing the character. When the local text similarity index of the local text data of the special character and the processed text consistency are higher, it means that the influence of the text data before and after the special character processing from the global and local analysis is not high, and the possibility of elimination processing is higher. When the connection correlation is smaller, it means that the connection correlation influence of the special character and the previous and next word segmentations is not high, and the influence after elimination is smaller, so the necessity of processing is greater.

[0086] Therefore, elimination processing can be performed based on the size of the processing necessity of special characters in the text data. In an embodiment of the present invention, a method for obtaining processed text data includes: in the text data of the advertisement, special characters whose processing necessity is normalized and whose value is greater than a preset processing threshold are eliminated, otherwise symbol processing is performed to obtain the processed text data of the advertisement, wherein the preset processing threshold can be set to 0.9. When it is greater than the processing threshold, it means that the elimination result has little impact, and elimination processing is directly performed. The rest can be processed according to the processing method of the symbol classification dictionary, that is, the symbols are retained after being replaced with text to ensure semantic integrity, which is used to improve the accuracy of subsequent violation identification.

[0087] S4: Analyze the degree of violation of the processed text data and image data of each advertisement, and obtain a violation score for each advertisement based on the abnormality of the advertisement delivery time; and identify violations based on the violation score of the advertisement.

[0088] After processing the text data of the advertisement, the violations of the text data and image data can be identified separately. At the same time, considering that illegal advertisements are usually released in the early morning, a more comprehensive final violation score evaluation can be conducted in combination with the abnormality of the advertisement release. Preferably, in this embodiment of the present invention, the method for obtaining the advertisement violation score includes:

[0089] For any advertisement, the image data is fed into a trained image violation scoring model, which then outputs an image violation score. The model then identifies the offending elements in the image and outputs a score. Furthermore, based on a pre-set violation vocabulary, the percentage of offending words in the processed text data of the advertisement is used as the text violation score. The processed text data is then compared with the pre-set violation vocabulary to identify the number of offending words. The ratio of the number of offending words to the total number of words in the processed text data is then used as the text violation score.

[0090] Further analysis of the time anomaly is performed, and based on the degree of proximity between the ad delivery time and the preset abnormal time, the degree of abnormality of the ad delivery period is obtained. In an embodiment of the present invention, the time difference between the ad delivery time and the preset abnormal time is calculated and negatively correlated with each other, and this is used as the period abnormality. The smaller the time difference, the closer the ad delivery time is to the abnormal time, and the higher the likelihood of the ad violation. The preset abnormal time can be set to midnight every day, that is, the closer the delivery time is to the early morning, the higher the likelihood of the ad violation. The implementation of the preset abnormal time can adjust it at will.

[0091] The frequency of similar ads delivered during the preset local time period of each ad delivery is then used as the ad's type frequency anomaly. The more frequent the delivery of similar ads, the more significant the violation. In this embodiment of the present invention, the number of ads of the same type delivered during the hour centered around each delivery of the ad is counted as the type frequency anomaly. The preset local time period is the hour centered around each delivery, and the specific value can be adjusted by the implementer.

[0092] The advertisement delivery time anomaly index is obtained by combining the delivery period anomaly and type frequency anomaly of the advertisement. In an embodiment of the present invention, the product of the delivery period anomaly and type frequency anomaly is used as the advertisement delivery time anomaly index, reflecting the possible degree of abnormality in the delivery time of the advertisement.

[0093] Finally, the ad's image violation score, text violation score, and delivery time anomaly index are weighted and summed to obtain the ad's ad violation score. In this embodiment of the present invention, the weights for the image violation score, text violation score, and delivery time anomaly index are set to 0.3, 0.6, and 0.1, respectively. The specific weights can be adjusted by the implementer.

[0094] As an example, the expression for ad violation score is: Where, Indicates the advertising violation score. Denotes the image violation score, Indicates the text violation score, This is an indicator of abnormal delivery time. A comprehensive assessment of ad violations is conducted across multiple aspects. A higher ad violation score indicates a greater likelihood of ad violations and a higher likelihood of ad violations.

[0095] Based on the violation score of the advertisement, the final violation identification is further performed. In an embodiment of the present invention, when the violation score of the advertisement is greater than the preset violation threshold, the advertisement is considered to be a violation advertisement, and the corresponding advertisement is recorded as a violation advertisement. When the violation score of the advertisement is less than the preset non-violation threshold, the advertisement is considered not to be a violation advertisement, and the corresponding advertisement is recorded as a non-violation advertisement. Otherwise, that is, when the violation score of the advertisement is less than or equal to the preset violation threshold and greater than or equal to the preset non-violation threshold, it is impossible to directly determine whether the advertisement is a violation, and a judgment is required. Therefore, the advertisement is manually reviewed for violation identification. In an embodiment of the present invention, the higher the violation score of the advertisement, the more likely the advertisement is to be a violation advertisement. Therefore, the preset non-violation threshold is less than the preset violation threshold. The preset non-violation threshold is set to 0.4, and the preset violation threshold is set to 0.6. The specific values ​​can be adjusted by the implementer.

[0096] In summary, the present invention specifically analyzes the text information data carried by the advertisement, compares and analyzes the high distribution consistency of the part of speech and word frequency of each word in the local text before and after the special symbol processing, as well as the similarity of the overall text, combined with the connection association of the words before and after the special symbol, and by comparing the consistency of the key subjects represented by the text before and after the special symbol processing, analyzes the degree of interference caused by the special symbol, and the degree to which the special symbol destroys the semantic connection of the word segments on both sides, and obtains the necessary processing degree of the special symbol, providing more accurate text information for the subsequent identification of illegal information. By combining the processed text data with the possibility of abnormality of the image and the release time, the violation score of the advertisement is evaluated for identification, and a more accurate evaluation result is obtained. The present invention processes the interfering special symbols carried in the advertisement text by comparing the semantic changes represented by the coherent text content before and after the special symbol processing, improves the accuracy of identifying illegal information of advertisements, and makes the identification of illegal advertisements more comprehensive and reliable.

[0097] This application also provides an Internet-based illegal advertising identification system, please refer to Figure 2 , which shows a structural diagram of an Internet-oriented illegal advertisement identification system provided by an embodiment of the present invention. The system includes: an advertisement collection unit 201, an advertisement text processing unit 202 and an advertisement illegal identification unit 203.

[0098] Advertisement collection unit 201, for collecting advertisements and performing pre-processing to obtain image data and text data of each advertisement; determining each special character in the text data;

[0099] Advertisement text processing unit 202 is configured to segment each special character in the local text data before and after symbol processing, obtain the key degree of each segmented word in the local text data before and after symbol processing based on the part of speech and word frequency of each segmented word, and obtain the processed text consistency of each special character based on the high distribution approximation of the segmented key degrees in the local text data before and after symbol processing.

[0100] The connection relevance of each special character is obtained based on the connection correlation degree and consistent distribution degree between each special character and the preceding and following word segments; the similarity between the local text data before and after the symbol processing of each special character is analyzed, and the necessity of processing each special character is obtained by combining the consistency of the processed text and the connection relevance; the special characters in the text data are eliminated based on the degree of necessity of processing, and the processed text data is obtained;

[0101] The advertisement violation identification unit 203 is used to analyze the violation degree of the processed text data and image data of each advertisement, and obtain a violation score for each advertisement in combination with the abnormality degree of the advertisement delivery time; and perform violation identification based on the advertisement violation score.

[0102] It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the above embodiment provides an Internet-oriented illegal advertising identification system and an Internet-oriented illegal advertising identification method embodiment, which are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0103] The present application also provides an Internet-based illegal advertisement identification device. Figure 3 , which shows a schematic structural diagram of an Internet-oriented illegal advertisement identification device provided by an embodiment of the present invention. The computer device includes a memory 301, a processor 302, and a computer program 303 stored in the memory 301 and running on the processor 302. When the processor 302 executes the computer program 303, the computer device can execute any one of the Internet-oriented illegal advertisement identification methods introduced above.

[0104] An embodiment of the present application also provides a computer program product. When the computer program product is run on a computer device, the computer device can execute any one of the above-mentioned methods for identifying illegal advertisements on the Internet.

[0105] An embodiment of the present application also provides a computer-readable storage medium, in which computer program code is stored. When the computer program code runs on a computer device, the computer device can execute any of the above-mentioned methods for identifying illegal advertisements on the Internet.

[0106] In the embodiments provided in the present application, it should be understood that the provided computer devices, computer program products and computer-readable storage media are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the methods provided above and will not be repeated here.

[0107] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0108] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A method for identifying illegal advertisements on the Internet, characterized in that: The method comprises: Collect advertisements and perform pre-processing to obtain image data and text data of each advertisement; determine each special character in the text data; Segment the local text data before and after each special character is symbolized, and obtain the key degree of each segmented word in the local text data before and after the symbolization based on the part of speech and word frequency of each segmented word; obtain the consistency of the processed text for each special character based on the high distribution approximation of the segmented key degrees in the local text data before and after the symbolization of each special character; The connection relevance of each special character is obtained based on the connection correlation degree and consistent distribution degree between each special character and the preceding and following word segments; the similarity between the local text data before and after the symbol processing of each special character is analyzed, and the necessity of processing each special character is obtained by combining the consistency of the processed text and the connection relevance; the special characters in the text data are eliminated based on the degree of necessity of processing to obtain the processed text data; The processed text data and image data of each advertisement are analyzed for the degree of violation, and combined with the degree of abnormality of the advertisement delivery time, a violation score for each advertisement is obtained; violations are identified based on the advertisement violation score.

2. The method for identifying illegal advertisements on the Internet according to claim 1, characterized in that: The method for obtaining the criticality includes: For any special character, the local text data before and after symbol processing of the character are respectively segmented to obtain segmented words; the key weight of each segmented word is determined according to the part of speech of each segmented word; For any word segment, the frequency of occurrence of the word segment in the text data of the advertisement is used as the word frequency of the word segment; the product of the value of the negative correlation mapping of the word segment and the key weight is used as the key degree of the word segment.

3. The method for identifying illegal advertisements on the Internet according to claim 1, characterized in that: The method for obtaining the consistency of the processed text includes: For any special character, all the segmented words in the local text data before the character is symbolized are arranged from large to small according to the key degree to obtain a key sequence; the first preset number of segmented words in the key sequence are used as the front key segmented words before the character is symbolized; similarly, the back key segmented words after the character is symbolized are obtained based on the local text data after the character is symbolized; Count the data of the same segmentation in the preceding key segmentation and the following key segmentation of the character and perform normalization processing to obtain the segmentation distribution approximation of the character; After calculating the key difference between the first key word and the second key word with the same sequence number in the key sequence, the sum of all differences is calculated and negative correlation mapping is performed to obtain the key similarity of the word of the character; The processed text consistency of the character is obtained by combining the word segmentation distribution approximation and word segmentation high key similarity of the character.

4. The method for identifying illegal advertisements on the Internet according to claim 1, characterized in that: The method for obtaining the connection correlation degree includes: For any special character, the previous segmentation word adjacent to the character in the local text data is used as the front segmentation word of the character, and the next segmentation word adjacent to the character in the local text data is used as the back segmentation word of the character; Get the sentiment scores of the front participle and the back participle respectively, and negatively map the difference in sentiment scores between the front participle and the back participle to obtain the connected sentiment correlation of the character; After counting the frequency of occurrence of the front participle and the character in the text data of the advertisement, and the frequency of occurrence of the character and the back participle in the text data of the advertisement, the sum of the two frequency values ​​is used as the phrase consistency of the character; The connection relevance of the character is obtained by combining the connection sentiment relevance and phrase consistency of the character.

5. The method for identifying illegal advertisements on the Internet according to claim 1, characterized in that: The method for obtaining the processing necessity includes: For any special character, calculate the similarity between the local text data of the character before symbol processing and the local text data after symbol processing to obtain the local text similarity index of the character; The product of the value of negative correlation mapping of the connection relevance of the character, the local text similarity index, and the consistency of the processed text is used as the necessity of processing the character.

6. The method for identifying illegal advertisements on the Internet according to claim 1, characterized in that: The method for obtaining processed text data includes: In the text data of the advertisement, special characters whose processing necessity is greater than a preset processing threshold are eliminated, and otherwise, symbol processing is performed to obtain processed text data of the advertisement.

7. The method for identifying illegal advertisements on the Internet according to claim 1, characterized in that: The method for obtaining the advertising violation score includes: For any advertisement, the image data of the advertisement is input into the trained image violation scoring model, and the image violation score of the advertisement is output; based on the preset violation word library, the proportion of violation words in the processed text data of the advertisement is used as the text violation score of the advertisement; Based on the degree of proximity between the ad delivery time and the preset abnormal time, the delivery period abnormality of the ad is obtained; the delivery frequency of the same type of ads in the preset local time period of each delivery is used as the type frequency abnormality of the ad; combining the delivery period abnormality and type frequency abnormality of the ad, the delivery time abnormality index of the ad is obtained; The ad violation score is obtained by taking a weighted sum of the ad's image violation score, text violation score, and delivery time anomaly index.

8. The method for identifying illegal advertisements on the Internet according to claim 1, characterized in that: The violation identification based on the violation score of the advertisement includes: When the ad violation score is greater than the preset violation threshold, the corresponding ad will be marked as a violation ad; When the ad violation score is lower than the preset non-violation threshold, the corresponding ad will be recorded as a non-violation ad; Otherwise, the advertisement will be manually reviewed for violation identification; the preset non-violation threshold is less than the preset violation threshold.

9. An Internet-based illegal advertising identification system, characterized in that: The system comprises: Advertisement collection unit, used to collect advertisements and perform pre-processing to obtain image data and text data of each advertisement; determine each special character in the text data; The advertising text processing unit is used to segment each special character in the local text data before and after the symbol processing, and obtain the key degree of each segmented word in the local text data before and after the symbol processing based on the part of speech and word frequency of each segmented word; and obtain the processed text consistency of each special character based on the high distribution approximation of the segmented key degree in the local text data before and after the symbol processing of each special character; The connection relevance of each special character is obtained based on the connection correlation degree and consistent distribution degree between each special character and the preceding and following word segments; the similarity between the local text data before and after the symbol processing of each special character is analyzed, and the necessity of processing each special character is obtained by combining the consistency of the processed text and the connection relevance; the special characters in the text data are eliminated based on the degree of necessity of processing, and the processed text data is obtained; The advertising violation identification unit is used to analyze the degree of violation of the processed text data and image data of each advertisement, and obtain a violation score for each advertisement based on the abnormality of the advertisement delivery time; and perform violation identification based on the advertisement violation score.

10. An Internet-based illegal advertisement identification device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements an Internet-oriented illegal advertisement identification method as described in any one of claims 1 to 8.

Images