A material auditing and content security filtering method

Abstract

The application discloses a material auditing and content security filtering method, relates to the technical field of material auditing, and aims to solve the problems of reduced auditing accuracy and increased consumption of server computing resources. The method is used for monitoring a game platform user display interface, calling an upload button attribute to determine whether it is game content when a user clicks to upload an image, setting a text interval and an image interval to obtain non-game content images, determining interval attributes through a natural semantic understanding model and an image feature capture model, focusing on auditing and marking the images in the text interval and the image interval, substituting the images into a deep learning model to divide a to-be-tested area, calling an auditing standard library, substituting the to-be-tested area into multiple to-be-tested areas to obtain a risk word combination number and a risk pixel density, and comprehensively determining whether to call artificial auditing intervention according to the text pixel coverage and the risk pixel density, so that the accuracy and processing efficiency of image content auditing are improved.

Description

Technical Field

[0001] This invention relates to the field of material review technology, and more specifically, to a method for material review and content security filtering. Background Technology

[0002] With the rapid development of online games, uploading and sharing images on game platforms has become an important part of daily operations. Due to the complexity of the types of images uploaded by users, which may include game screenshots, personal photos, text information, sensitive images or inappropriate content, in order to prevent the spread of illegal or inappropriate content, it is usually necessary to manually or automatically review the images uploaded by users in the current technology.

[0003] The existing technology has the following shortcomings:

[0004] Currently, existing technologies perform uniform review and processing on all images without distinguishing between image content types. They cannot adopt differentiated review strategies for text-based or image-based content, making it difficult to detect sensitive text or illegal images in composite images. This leads to reduced review accuracy and increased consumption of server computing resources. Therefore, a material review and content security filtering method is proposed.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for material review and content security filtering, which solves the problems mentioned in the background art by employing multimodal content recognition, deep learning image feature extraction, text semantic analysis, and multidimensional risk assessment mechanisms.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for material review and content security filtering, comprising the following steps:

[0008] Step S1: Monitor the user display interface of the game platform. When the user clicks to upload an image, call the upload button properties. Analyze whether the uploaded image is game content based on the upload button properties, and conduct a preliminary review of non-game content images.

[0009] Step S2: Set the text interval and image interval, acquire non-game content images, and determine whether the non-game content images are in the text interval and image interval respectively through the natural language understanding model and the image feature capture model;

[0010] Step S3: Call non-game content images that are simultaneously in the text and image regions for key review and mark them as key images. Substitute the key images into the deep learning model and divide them into multiple test regions.

[0011] Step S4: Call the audit standard library, substitute multiple areas to be tested to obtain the number of risky word combinations and risky pixel density, and determine whether to call manual audit intervention based on the comprehensive text pixel coverage and risky pixel density.

[0012] In a preferred embodiment, in step S1, an event listening plugin is set to monitor the user display interface of the game platform in real time. The user display interface has an upload button, a delete button, and an interaction button.

[0013] When a user clicks to upload an image, the upload button's properties are invoked. The upload button's properties include upload buttons for game content and upload buttons for non-game content.

[0014] In a preferred embodiment, in step S1, the uploaded image is analyzed based on the upload button attributes. The specific analysis process is as follows:

[0015] If the upload button is designated as a game content upload button, the uploaded image will be marked as game content, and the review process will be skipped.

[0016] If the upload button is designated as a non-game content upload button, the uploaded image will be marked as a non-game content image, and a preliminary review will be conducted on the non-game content image.

[0017] In a preferred embodiment, in step S2, a text range and an image range are preset, and non-game content images are processed by a natural language understanding model and an image feature capture model, respectively.

[0018] Non-game content images are input into a natural language understanding model to collect the percentage of text pixels and the color monotonicity of the remaining pixels in the non-game content images.

[0019] The text detection branch of the natural semantic understanding model is used to recognize characters in the image and extract the mask of the text region. The number of effective pixels in the mask and the total number of pixels in the whole image are counted and the difference is calculated to obtain the proportion of text pixels.

[0020] By counting the total number of remaining pixels, the color vectors of the remaining pixel set are clustered according to the color clustering algorithm. The number of color clusters is obtained and the ratio is calculated with the total number of remaining pixels to obtain the ratio result. The difference between the value 1 and the ratio result is calculated to obtain the color monotonicity of the remaining pixels.

[0021] In a preferred embodiment, in step S2, the proportion of text pixels and the monotonicity of the remaining pixel colors are standardized and weighted summation is performed to obtain the text content coefficient.

[0022] The text content coefficient is compared and analyzed with the preset text threshold.

[0023] If the text content coefficient is greater than or equal to the text threshold, then non-game content images are classified into the text interval;

[0024] If the text content coefficient is lower than the text threshold, then non-game content images will not be classified into the text interval;

[0025] Continue to input non-game content images into the image feature capture model to collect the number of suspected graphics in the non-game content images.

[0026] In a preferred embodiment, in step S2, the image is grayscaled and Gaussian smoothed, the pixel gradient magnitude matrix is ​​calculated, the edge distribution is calculated using an edge detection operator, and the closed contour regions formed by continuous edges are identified based on the connected component labeling algorithm. The number of detected closed contour regions is used as the number of suspected images.

[0027] The number of suspected images is compared and analyzed with a preset image threshold;

[0028] If the number of suspected images is greater than or equal to the image threshold, then non-game content images are classified into the image interval;

[0029] If the number of suspected images is lower than the image threshold, then non-game content images will not be classified into the image range;

[0030] Four results are obtained by determining whether non-game content images are in the text and image regions: images divided into text regions, images divided into image regions, images in both text and image regions, and images not divided.

[0031] Non-game content images that are simultaneously greater than or equal to both the text threshold and the image threshold are considered to be images that are simultaneously in the text interval and the image interval.

[0032] Meanwhile, non-game content images that are below both the text threshold and the image threshold are considered unclassified images.

[0033] In a preferred embodiment, in step S3, non-game content images that are simultaneously located in the text area and the image area are called for key review and marked as key images;

[0034] The key images are fed into a deep learning model, and the image is divided into multiple test regions by the feature extraction layer and region proposal network of the deep learning model.

[0035] Multi-layer feature maps of key images are extracted using a convolutional neural network. Gradient response intensity is calculated on each feature map. Candidate region sets are generated based on the gradient response intensity. The confidence level of each candidate region is calculated using a region proposal network.

[0036] In a preferred embodiment, in step S3, the test confidence of each candidate region is compared and analyzed with a preset test threshold.

[0037] If the confidence level of a candidate region is greater than or equal to the threshold, the corresponding candidate region is marked as the region to be tested.

[0038] If the confidence level of a candidate region is lower than the threshold, the corresponding candidate region will be filtered out.

[0039] The candidate regions marked as the test regions are statistically analyzed to obtain multiple test regions.

[0040] In a preferred embodiment, in step S4, the audit standard library is invoked to obtain a set of risk word combinations and risk pixel standards;

[0041] The number of risk word combinations and risk pixel density are calculated by substituting multiple regions to be tested into the audit standard library;

[0042] The text content in the test area is extracted by the text recognition algorithm. The recognized text is combined into multiple word expressions and compared with the risk word combination set. If there is a matching word combination, it is counted as a match. The number of matches is counted as the number of risk word combinations.

[0043] Extract the features of all pixels in the test area, compare them with the risk pixel standard, and record the pixels with a similarity greater than or equal to the set threshold as risk pixels. Calculate the risk pixel density by the ratio of the number of risk pixels to the total number of pixels in the test area.

[0044] In a preferred embodiment, in step S4, the number of risk word combinations and the risk pixel density of all regions to be tested are summed to obtain the total number of risk word combinations and the total risk pixel density.

[0045] The total number of risk word combinations and the total density of risk pixels are standardized and then substituted into a multinomial regression formula to calculate the audit intervention coefficient.

[0046] The review intervention coefficient is compared and analyzed with the preset review threshold.

[0047] If the review intervention coefficient is greater than or equal to the review threshold, then manual review intervention will be initiated.

[0048] If the review intervention coefficient is lower than the review threshold, manual review intervention will not be invoked.

[0049] The technical effects and advantages of this invention are as follows:

[0050] This invention monitors the user interface of a game platform. When a user clicks to upload an image, the upload button's attributes are invoked. Based on these attributes, the uploaded image is analyzed to determine if it contains game content. Text and image regions are set, and non-game content images are acquired. Natural language understanding and image feature capture models are used to determine whether the non-game content image falls within either the text or image region. Non-game content images that fall within both regions are prioritized for review and marked as key images. These key images are then fed into a deep learning model and divided into multiple test regions. A review standard library is invoked, and the number of risky word combinations and risky pixel density are obtained by substituting these into the test regions. The combined text pixel coverage and risky pixel density determine whether manual review should be initiated. This achieves intelligent allocation and pre-filtering of review resources, improving the accuracy and efficiency of image content review. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the implementation of a material review and content security filtering method according to the present invention.

[0052] Figure 2 This is a schematic diagram illustrating the steps of a material review and content security filtering method according to the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] This invention monitors the user interface of a game platform. When a user clicks to upload an image, the upload button's attributes are invoked. Based on these attributes, the uploaded image is analyzed to determine if it contains game content. Text and image regions are set, and non-game content images are acquired. Natural language understanding and image feature capture models are used to determine whether the non-game content image falls within either the text or image region. Non-game content images that simultaneously fall within both regions are prioritized for review and marked as key images. These key images are then fed into a deep learning model and divided into multiple test regions. A review standard library is invoked, and the number of risky word combinations and risky pixel density are obtained by substituting these into the test regions. The combined text pixel coverage and risky pixel density determine whether manual review intervention is necessary. This achieves intelligent allocation and pre-filtering of review resources, reduces the frequency of manual intervention, and improves the platform's overall response speed and processing efficiency.

[0055] Example 1

[0056] Please see Figures 1 to 2 A method for material review and content security filtering, the specific operation process is as follows:

[0057] Step S1: Monitor the user display interface of the game platform. When the user clicks to upload an image, call the upload button properties. Analyze whether the uploaded image is game content based on the upload button properties, and conduct a preliminary review of non-game content images.

[0058] Step S2: Set the text interval and image interval, acquire non-game content images, and determine whether the non-game content images are in the text interval and image interval respectively through the natural language understanding model and the image feature capture model;

[0059] Step S3: Call non-game content images that are simultaneously in the text and image regions for key review and mark them as key images. Substitute the key images into the deep learning model and divide them into multiple test regions.

[0060] Step S4: Call the audit standard library, substitute multiple areas to be tested to obtain the number of risky word combinations and risky pixel density, and determine whether to call manual audit intervention based on the comprehensive text pixel coverage and risky pixel density.

[0061] The specific implementation is as follows:

[0062] In step S1, an event listener plugin is set up to monitor the user interface of the game platform in real time. The user interface has upload, delete and interaction buttons.

[0063] Among them, the event listening plugin continuously captures user operation event data, including click events, hover events and interactive feedback events, by loading a listening script in the user display interface, and records the captured operation events in chronological order to the operation behavior cache area;

[0064] Furthermore, the user display interface of the game platform is a user-personalized display interface built into the game, used to display the user's game character information and user-defined content, such as character screenshots, daily displays, achievement images, etc., and to establish in-game interpersonal communication methods and content sharing channels.

[0065] Furthermore, the upload button is an interactive control used to trigger users to upload image resources, and the delete button is an interactive control used to perform the deletion operation of uploaded image resources. The interactive buttons include the like button and the comment button. The like button is a trigger control used to provide positive feedback to the image resources uploaded by users, and the comment button is a text interactive control used to enter comment text and associate it with the image resource.

[0066] When a user clicks to upload an image, the upload button's properties are invoked. The upload button's properties include upload buttons for game content and upload buttons for non-game content.

[0067] Among them, the game content upload button is an upload button that is automatically generated and displayed when the user performs a screenshot operation within the game running page. It is used to directly upload the current screen screenshot in the game running environment. The non-game content upload button is the upload button displayed on the user interface.

[0068] It is understandable that the upload button on the user interface is for users to actively select local image resources for upload. It selects and uploads image resources by calling the user's local file selection interface, rather than being an upload button generated when taking screenshots in the game. Its triggering logic is independent of the game's running process and is used to upload image content not generated by the game.

[0069] The uploaded image is analyzed based on the upload button's attributes. The specific analysis process is as follows:

[0070] If the upload button is designated as a game content upload button, the uploaded image will be marked as game content, and the review process will be skipped.

[0071] If the upload button is designated as a non-game content upload button, the uploaded image will be marked as a non-game content image, and the non-game content image will undergo preliminary review.

[0072] The option to skip the review process means to skip the subsequent review and display uploaded images marked as game content on the user's display interface.

[0073] In step S2, text and image ranges are pre-set to distinguish the types of non-game content images;

[0074] In this experiment, the researchers set up text intervals and image intervals based on the text pixel coverage distribution and edge complexity of the training images. The text interval is used to reflect the probability interval of the presence of identifiable text content in the image, while the image interval is used to reflect the structural and contextual features of the image content.

[0075] Non-game content images are processed using both a natural language understanding model and an image feature capture model.

[0076] Non-game content images are input into a natural language understanding model to collect the percentage of text pixels and the color monotonicity of the remaining pixels in the non-game content images.

[0077] The text pixel ratio of non-game content images refers to the ratio of the number of pixels in the text region to the total number of pixels in the entire image. The logic for obtaining it is to perform character recognition on the image and extract the mask of the text region through the text detection branch in the natural semantic understanding model, count the number of effective pixels in the mask and count the total number of pixels in the entire image, and calculate the difference to obtain the text pixel ratio.

[0078] It should be noted that the text detection branch in the natural semantic understanding model is a text detection structure built on the fusion mechanism of convolutional neural network and attention feature, which is used to identify and locate text regions in images. Character recognition is achieved based on feature map projection and serial connection algorithm. That is, by projecting the convolutional feature map in the horizontal and vertical directions, the continuous features of the text boundary are extracted, and the continuous feature sequence is input into the bidirectional long short-term memory network to complete the character sequence recognition and obtain the character position information.

[0079] Furthermore, the mask for the text region is a binary pixel matrix generated based on the character confidence vector and the feature boundary coordinates. The text region can be separated from the background region through the mask matrix of the text region.

[0080] The color monotonicity of the remaining pixels in non-game content images is the ratio between the number of different color clusters in the remaining pixels of the image and the total number of pixels after removing the text region. The logic for obtaining it is to count the total number of remaining pixels, cluster the color vectors of the remaining pixel set according to the color clustering algorithm, obtain the number of color clusters, and calculate the ratio with the total number of remaining pixels. The difference between the value 1 and the ratio result is used to calculate the color monotonicity of the remaining pixels.

[0081] It should be noted that although the basic colors of an image are derived from the additive mixing of the three primary colors of red, green, and blue, the color space distribution can be regarded as a continuous distribution because the quantization precision of each channel can reach 8 bits or more. In this experiment, the researchers performed cluster analysis on the color space to achieve an effective estimation of color monotonicity without affecting the accuracy of the results.

[0082] Furthermore, the color clustering algorithm is based on the K-means clustering principle. The K-means clustering algorithm clusters the color vectors of the remaining pixels in the color space using Euclidean distance as a metric to obtain the number of color clusters. The specific implementation process is common knowledge to those skilled in the art and will not be elaborated here.

[0083] Furthermore, since the total number of remaining pixels is greater than or equal to the number of color clusters, the range of the color monotonicity of the remaining pixels is [0,1). In actual calculations, to ensure numerical stability, the researchers set up a fault tolerance process for the case where the color monotonicity of the remaining pixels is negative. If the color monotonicity of the remaining pixels is negative, it is reset to zero to ensure the validity and consistency of the color monotonicity results.

[0084] The text pixel ratio and the color monotonicity of the remaining pixels are standardized so that the text pixel ratio and the color monotonicity of the remaining pixels are on the same dimension and the numerical expression is between 0 and 1.

[0085] It should be noted that the standardization methods include, but are not limited to, standard linear transformation based on interval scaling, statistical Z-Score standardization method, or normalization method based on nonlinear mapping function. The application methods of standardization will not be elaborated here.

[0086] The text content coefficient is obtained by weighted summation of the standardized text pixel ratio and the color monotonicity of the remaining pixels.

[0087] The specific calculation formula is as follows:

[0088] ;

[0089] In the formula, The coefficient represents the text content. This represents the pixel ratio of the text after standardization. To reduce the monotonicity of the remaining pixel colors after standardization, and These are weight parameters;

[0090] It should be noted that when the proportion of text pixels and the monotony of the remaining pixel colors are greater, the proportion of text areas in non-game content images is higher and the background color is more uniform. This indicates that the image as a whole mainly expresses text information, the image texture complexity is low, and the image form tends to be textual. Therefore, the larger the text content coefficient, the more necessary it is to divide non-game content images into text areas.

[0091] The text content coefficient is compared and analyzed with the preset text threshold.

[0092] If the text content coefficient is greater than or equal to the text threshold, then non-game content images are classified into the text interval;

[0093] If the text content coefficient is lower than the text threshold, then non-game content images will not be classified into the text interval;

[0094] It should be noted that the text threshold was set by the researchers based on the proportion of text pixels in historical image samples and the recognition accuracy of the natural language understanding model in the classification experiment, and will not be elaborated here.

[0095] Continue to input non-game content images into the image feature capture model to collect the number of suspected graphics in the non-game content images;

[0096] It should be noted that the order of using the natural language understanding model and the image feature capture model is not limited (i.e., all non-game content images need to be divided into intervals using both the natural language understanding model and the image feature capture model). In this example, the operation process is carried out by first using the natural language understanding model and then using the image feature capture model. In practical applications, the order of using the natural language understanding model and the image feature capture model can be set according to the number of non-game content images and the computational resource consumption of the model, which will not be elaborated here.

[0097] The number of suspected graphics refers to the number of continuous closed regions identified in non-game content images after edge detection and target contour extraction. The acquisition logic is to perform grayscale and Gaussian smoothing on the image, calculate the pixel gradient magnitude matrix, use the edge detection operator to calculate the edge distribution, and identify the closed contour regions formed by continuous edges based on the connected component labeling algorithm. The number of detected closed contour regions is used as the number of suspected graphics.

[0098] Specifically, the image feature capture model is a model used to perform feature gradient analysis and target contour recognition on input images. Its main function is to capture the shape, edge and spatial structure features in non-text images to identify regions with graphic semantics and to distinguish between image content and text content.

[0099] It should be noted that the methods of performing grayscale conversion and Gaussian smoothing on the image, calculating the pixel gradient magnitude matrix, using edge detection operators to calculate the edge distribution, and identifying the closed contour region formed by continuous edges based on the connected component labeling algorithm were all implemented by our researchers using an image processing software toolkit. The specific edge detection and contour extraction processing of the image are common knowledge in this field and will not be elaborated here.

[0100] The number of suspected images is compared and analyzed with a preset image threshold;

[0101] If the number of suspected images is greater than or equal to the image threshold, then non-game content images are classified into the image interval;

[0102] If the number of suspected images is lower than the image threshold, then non-game content images will not be classified into the image range;

[0103] It should be noted that the image threshold was set by the researchers based on the number of suspected graphics in historical image samples and the recognition accuracy of the image feature capture model in the classification experiment, and will not be elaborated here.

[0104] Four results are obtained by determining whether non-game content images are in the text and image regions: images divided into text regions, images divided into image regions, images in both text and image regions, and images not divided.

[0105] Among them, non-game content images that are simultaneously greater than or equal to both the text threshold and the image threshold are images that are simultaneously in the text interval and the image interval.

[0106] At the same time, non-game content images that are below both the text threshold and the image threshold are considered undivided images;

[0107] Furthermore, for non-game content images classified into text regions, content security filtering is performed through subsequent text semantic understanding and sensitive word matching operations; for non-game content images classified into image regions, image review is performed through subsequent image feature analysis and risk pixel density determination operations; for images that are not classified, they are blurred and returned to the user's terminal with the message "Image invalid".

[0108] In step S3, non-game content images that are simultaneously located in the text area and the image area are selected for key review and marked as key images;

[0109] Among them, those that are simultaneously located in the text area and the image area have a high proportion of text and many graphic features, and belong to composite images that contain both text information and complex image structures.

[0110] The key images are fed into a deep learning model, and the image is divided into multiple test regions by the feature extraction layer and region proposal network of the deep learning model.

[0111] Among them, the region to be tested refers to the region in the key image that needs to be reviewed in detail by identifying the saliency distribution and gradient changes of the feature map;

[0112] The specific implementation process for this division is as follows:

[0113] Multi-layer feature maps of key images are extracted using a convolutional neural network. Gradient response intensity is calculated on each feature map. Candidate region set is generated based on gradient response intensity. The confidence level of each candidate region is calculated using a region proposal network.

[0114] It should be noted that a convolutional neural network (CNN) is a deep learning structure used to extract multi-level feature information from images. Through continuous convolution, activation, and downsampling operations, the spatial information of a key image is converted into multi-layer feature maps to reflect the edge, texture, and semantic structure features of the image. The multi-layer feature maps are the feature representation results output from different layers of the CNN. The gradient response intensity is used to measure the degree of pixel change in the feature map. The larger the gradient response intensity, the more obvious the feature change at that location, which is used to determine the region to be tested. The region proposal network is used to generate candidate regions in the feature map and calculate the confidence level of each candidate region in containing potential targets to filter out the regions to be tested. The above calculation processes of convolutional feature extraction, gradient calculation, candidate region generation, and confidence level determination are common knowledge to those skilled in the art and will not be elaborated here.

[0115] The test confidence level of each candidate region is compared and analyzed with the preset test threshold.

[0116] If the confidence level of a candidate region is greater than or equal to the threshold, the corresponding candidate region is marked as the region to be tested.

[0117] If the confidence level of a candidate region is lower than the threshold, the corresponding candidate region will be filtered out.

[0118] It should be noted that the preset test threshold was set by the researchers based on the confidence convergence interval of the deep learning model during the training phase and the misjudgment rate of the actual review samples, which will not be elaborated here.

[0119] The candidate regions marked as the test regions are statistically analyzed to obtain multiple test regions.

[0120] In step S4, the audit standard library is invoked to obtain a set of risk term combinations and risk pixel standards;

[0121] Specifically, the audit standard library is a multi-dimensional risk feature database established by our researchers through summarizing and refining historical violation image samples, violation text samples, and content security review rules, which is used to provide a reference for auditing.

[0122] Among them, the risk word combination set is a set of word sequences with fixed contextual relationships obtained by extracting semantic fragments and analyzing word associations from violation text samples, and the risk pixel standard is a risk pixel standard extracted by performing color space clustering and texture pattern analysis on the pixel features of violation image samples.

[0123] The number of risk word combinations and risk pixel density are calculated by substituting multiple regions to be tested into the audit standard library;

[0124] The number of risk vocabulary combinations refers to the number of matches obtained by matching the character sequences extracted by the text recognition algorithm in the area to be measured with each combination in the risk vocabulary combination set. Its acquisition logic is to extract all the text content in the area to be measured through the text recognition algorithm, combine the recognized characters into multiple vocabulary expressions and compare them with the risk vocabulary combination set. If there is a consistent vocabulary combination, it is recorded as one match, and the number of matches is counted as the number of risk vocabulary combinations;

[0125] It should be noted that the text recognition algorithm is an algorithm for extracting text information from the area to be measured and restoring its semantic combination relationship. In this embodiment, through the detection, segmentation and character classification of the text pixel features in the area to be measured, character-level recognition output is achieved; after the recognized single-word information is analyzed by the language model for context association, multiple possible vocabulary combination expressions are generated. For example, when the two characters "bee" and "honey" are recognized, vocabulary expressions such as "honey" and "bee" can be combined. All the generated vocabulary combinations are matched and compared with the risk vocabulary combination set. If there is a consistent or highly similar vocabulary combination, it is marked as one match;

[0126] The acquisition of the risk pixel density is to extract the features of all pixel points in the area to be measured, perform similarity comparison with the risk pixel standard, record the pixels with similarity greater than or equal to the set threshold as risk pixels and count them, and calculate the ratio of the number of risk pixels to the total number of pixels in the area to be measured to obtain the risk pixel density;

[0127] It should be noted that the similarity comparison is a single-point similarity evaluation based on the color vector information of pixel points. The set threshold is set by the experimenter according to the gray distribution characteristics of the risk pixel standard and the average texture difference of the risk area, which will not be elaborated here;

[0128] Accumulate the number of risk vocabulary combinations and the risk pixel density of all areas to be measured to obtain the total number of risk vocabulary combinations and the total risk pixel density;

[0129] Perform standardization processing on the total number of risk vocabulary combinations and the total risk pixel density, so that the total number of risk vocabulary combinations and the total risk pixel density are in the same dimension, and the numerical expression range is between 0 and 1;

[0130] Furthermore, the standardization processing has been described in the above embodiment and will not be elaborated here;

[0131] Substitute the total number of risk vocabulary combinations and the total risk pixel density after standardization processing into the polynomial regression formula to calculate the review intervention coefficient. The specific formula expression is as follows:

[0132] ;

[0133] In the formula, For the audit intervention coefficient, This represents the total number of standardized risk term combinations. The total density of risky pixels after standardization. To adjust the parameters, as well as The weighting coefficients corresponding to the total number of risk word combinations and the total density of risk pixels after standardization.

[0134] It should be noted that the order and form of the polynomial regression function in this invention can be selected according to the actual application requirements. The specific order and function form are not limited here. Those skilled in the art can determine them based on the actual number of regions to be tested, which will not be elaborated here.

[0135] It should be explained that when the total number of risky word combinations and the total density of risky pixels are greater, it means that the area under test contains more sensitive semantic information and high-risk image features, the content needs more security filtering, the review intervention coefficient is greater, and the need to call in manual review intervention for fine-grained judgment is greater, so as to ensure the accuracy of the review results.

[0136] The review intervention coefficient is compared and analyzed with the preset review threshold.

[0137] If the review intervention coefficient is greater than or equal to the review threshold, then manual review intervention will be initiated.

[0138] If the review intervention coefficient is lower than the review threshold, manual review will not be invoked.

[0139] Specifically, if manual review is called in, key images will be subject to manual review.

[0140] Furthermore, if manual review is not invoked, key images will be reviewed by performing text semantic understanding and sensitive word matching operations for content security filtering, and image feature analysis and risk pixel density determination operations for image review.

[0141] It should be noted that the review threshold was set by the researchers based on the review intervention coefficient values ​​of historical review samples and the manual review pass rate, which will not be elaborated here.

[0142] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0143] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0144] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0145] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0146] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for material review and content security filtering, the method comprising: Comprise the following steps: ​ Step S1: Monitor the user interface of the game platform, call the upload button attribute when the user clicks to upload the picture, analyze whether the uploaded picture is game content according to the upload button attribute, and preliminarily review the non-game content image; Step S2: Set the text interval and image interval, obtain the non-game content image, and judge whether the non-game content image is in the text interval and the image interval through the natural semantic understanding model and the image feature capture model respectively; In step S2, the text interval and the image interval are set in advance. The text interval is used to reflect the probability interval of the existence of identifiable text content in the image, and the image interval is used to reflect the structural and scene features of the image content. The non-game content image is respectively input into the natural semantic understanding model and the image feature capture model; The text pixel ratio of the non-game content image and the remaining pixel color monotony are collected by inputting the non-game content image into the natural semantic understanding model; The character recognition of the image is carried out by the text detection branch in the natural semantic understanding model, and the mask of the text area is extracted. The number of effective pixels in the mask and the total number of image pixels are counted and the difference is calculated to obtain the text pixel ratio; By counting the total number of remaining pixels, the color vector of the remaining pixel set is clustered according to the color clustering algorithm, the number of color clusters is obtained, and the ratio is calculated by the total number of remaining pixels to obtain the ratio result. The difference between the value 1 and the ratio result is calculated to obtain the remaining pixel color monotony; Step S3: Call the non-game content image which is in the text interval and the image interval at the same time for key review and mark as key image. The key image is substituted into the deep learning model and divided into multiple test areas; Step S4: Call the review standard library, substitute the multiple test areas to obtain the number of risk word combinations and the risk pixel density, and determine whether to call artificial review intervention according to the text pixel coverage and the risk pixel density.

2. The material review and content security filtering method according to claim 1, wherein: In step S1, an event listener plug-in is set to monitor the user interface of the game platform in real time. The user interface has an upload button, a delete button and an interactive button; When the user clicks to upload the picture, the upload button attribute is called. The upload button attribute includes a game content upload button and a non-game content upload button.

3. The material review and content security filtering method according to claim 2, wherein: In step S1, the uploaded picture is analyzed according to the upload button attribute. The specific analysis process is as follows: If the upload button attribute is a game content upload button, the uploaded picture is marked as game content, and the review step is skipped; If the upload button attribute is a non-game content upload button, the uploaded picture is marked as a non-game content image, and the non-game content image is preliminarily reviewed.

4. The material review and content security filtering method according to claim 1, wherein: In step S2, the text pixel ratio and the remaining pixel color monotony are standardized and weighted summed to obtain a text content coefficient. The text content coefficient is compared with a preset text threshold value; If the text content coefficient is greater than or equal to the text threshold value, the non-game content image is classified into a text interval; If the text content coefficient is less than the text threshold value, the non-game content image is not classified into the text interval; The non-game content image is continuously input into the image feature capture model to collect suspected graphic numbers of the non-game content image.

5. The material review and content security filtering method according to claim 4, wherein: In step S2, the image is subjected to grayscale and Gaussian smoothing processing, a pixel gradient amplitude matrix is calculated, an edge distribution is calculated using an edge detection operator, and a closed contour region formed by continuous edges is identified based on a connected region marking algorithm, and the number of closed contour regions detected is taken as the suspected graphic number; The suspected graphic number is compared with a preset image threshold value; If the suspected graphic number is greater than or equal to the image threshold value, the non-game content image is classified into an image interval; If the suspected graphic number is less than the image threshold value, the non-game content image is not classified into the image interval; Four results are obtained by judging whether the non-game content image is in the text interval and the image interval, specifically, the image classified into the text interval, the image classified into the image interval, the image in both the text interval and the image interval, and the unclassified image; The non-game content image greater than or equal to both the text threshold value and the image threshold value is the image in both the text interval and the image interval; The non-game content image less than both the text threshold value and the image threshold value is the unclassified image.

6. The material review and content security filtering method according to claim 1, wherein: In step S3, the non-game content image in both the text interval and the image interval is called for intensive review and marked as an intensive image; The intensive image is substituted into a deep learning model, and the image is regionally classified into multiple test regions through a feature extraction layer and a region proposal network of the deep learning model; Multiple layer feature maps of the intensive image are extracted through a convolutional neural network, the gradient response intensity is calculated on each feature map, a candidate region set is generated according to the gradient response intensity, and the test confidence of each candidate region is calculated through the region proposal network.

7. The material review and content security filtering method according to claim 6, wherein: In step S3, the test confidence of each candidate region is compared with a preset test threshold value; If the test confidence of the candidate region is greater than or equal to the test threshold value, the corresponding candidate region is marked as a test region; If the test confidence of the candidate region is less than the test threshold value, the corresponding candidate region is excluded; The candidate regions marked as the test regions are counted to obtain multiple test regions.

8. The material review and content security filtering method according to claim 7, wherein: In step S4, an audit standard library is called to obtain a risk word combination set and a risk pixel standard; The multiple test regions are substituted into the audit standard library to calculate the risk word combination number and the risk pixel density. The text recognition algorithm is used to extract all the text content in the to-be-tested region, a plurality of word combinations are combined from the recognized text, and the word combinations are compared with a risk word combination set. If there is a consistent word combination, it is recorded as one match, and the number of matches is counted as the number of risk word combinations; All pixel features in the to-be-tested region are extracted, and similarity comparison is performed with the risk pixel standard. The pixels with a similarity greater than or equal to a set threshold are recorded as risk pixels and counted. The risk pixel density is calculated by dividing the number of risk pixels by the total number of pixels in the to-be-tested region.

9. The material review and content security filtering method of claim 8, wherein: In step S4, the number of risk word combinations and the risk pixel density of all to-be-tested regions are accumulated to obtain the total number of risk word combinations and the total risk pixel density; The total number of risk word combinations and the total risk pixel density are standardized and substituted into a polynomial regression formula to calculate a review intervention coefficient; The review intervention coefficient is compared with a preset review threshold value for analysis; If the review intervention coefficient is greater than or equal to the review threshold value, manual review intervention is called; If the review intervention coefficient is lower than the review threshold value, manual review intervention is not called.

Images