Content-security-oriented multi-modal multi-stage multi-view community user portrait and content understanding method and system and medium

Abstract

The invention discloses a multi-mode, multi-stage and multi-view community user portrait and content understanding method and system for content security and a medium. The method comprises the following steps: 1) acquiring multi-view feature data of community users; 2) performing grading processing on the multi-view feature data to obtain a difficult sample; 3) labeling the difficult sample to form a labeled data set; 4) constructing a multi-modal large model, and performing adjustment by using the annotation data set to obtain a reference strategy model; 5) performing reinforcement learning optimization on the reference strategy model; and 6) deploying the optimized multi-modal large model at a server, and providing a judgment result for an online content security decision. The system comprises a feature acquisition and fusion module, a rule processing module, a graph calculation and integrated learning module, a data flywheel and knowledge base module, a multi-modal supervision fine tuning module, a multi-modal reinforcement learning module and an online reasoning and service module. According to the method, training from a rule layer to a multi-modal large model and reasoning of full-link interpretability and self-evolution capability are realized.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and content security technology, specifically to a multimodal, multi-stage, and multi-perspective community user profiling and content understanding method, system, and medium for content security. Background Technology

[0002] With the rapid development of content communities, social media, and video platforms, the number of users and the amount of content on these platforms are growing exponentially. The behavioral data, content data, social relationship data, and device environment data generated by users within these platforms are highly complex, diverse, and evolving rapidly. In terms of user profiling and content understanding, these platforms undertake both recommendation and operational optimization functions, as well as content security and compliance governance responsibilities.

[0003] In existing technologies, platforms typically use manual rules, keyword filtering, and several traditional machine learning models for content moderation and account risk identification. While these methods were effective against coarse-grained risks in the early stages, their limitations have become increasingly apparent as black market and gray market technologies continue to evolve.

[0004] First, there's the issue of accuracy and coverage.

[0005] Traditional methods primarily rely on fixed rules, keyword matching, or shallow models, making it difficult to model the deep semantics of the content itself, implicit elements in images, and dynamic behaviors in videos. They are particularly ineffective at identifying highly disguised black market accounts, covert marketing accounts, traffic-driving accounts, and content that evades review through "borderline," slang, or homophones. Attackers often circumvent single-modal checks through synonym substitution, mixed text and images, and mismatched video narration and subtitles, resulting in insufficient accuracy and low recall rates in risk identification.

[0006] Second, there are issues of efficiency and cost.

[0007] While large multimodal models (including large language models and their visual and multimodal extensions) offer stronger semantic understanding and robustness, directly invoking all content and user behavior data in large-scale industrial scenarios incurs extremely high computational costs (computing power, storage, and token costs) and inference latency. This is especially true in online scenarios with timeframes of hours, minutes, or even seconds, where relying solely on large models struggles to meet throughput and latency requirements.

[0008] Third, the issue of interpretability.

[0009] In practice, machine learning models are often considered "black boxes." However, in risk control and content security scenarios, operations and strategy personnel need to explain and trace the decisions made regarding blocking, demotion, and traffic limiting to support strategy iteration, user appeals, and compliance audits. A lack of explainability weakens business trust in the model and hinders the construction of a collaborative decision-making system based on "rules + models."

[0010] Fourth, the problems of data silos and feature fragmentation.

[0011] User profile features, behavioral features, content features, social network features, and device and environmental fingerprint features are often scattered across different systems and data sources. The lack of systematic multi-perspective fusion modeling capabilities results in one-sided user profiles and incomplete scene perception, which has limited effectiveness in complex attack and defense games and the identification of multi-account "gang" behavior.

[0012] Furthermore, existing systems are generally inadequate in terms of "model self-evolution." In real-world business scenarios, new attack methods, cryptic expressions, and cross-modal evasion paths constantly emerge. If these emerging challenging examples cannot be quickly collected, transformed into training data, and fed back into the model and rules, the system performance will quickly degrade.

[0013] Therefore, there is a need for a community user profiling and content understanding method and system that can perform deep fusion and hierarchical processing of multi-source and multi-modal data under controllable cost, while also possessing high accuracy, interpretability, self-evolution capabilities, and supporting industrial-grade online deployment. Summary of the Invention

[0014] The purpose of this invention is to provide a multimodal, multi-stage, and multi-perspective method, system, and medium for community user profiling and content understanding oriented towards content security.

[0015] The technical means adopted to achieve the technical objective of this invention are as follows: a multimodal, multi-stage, and multi-perspective community user profiling and content understanding method for content security, comprising the following steps:

[0016] 1) Obtain multi-perspective feature data of community users.

[0017] 2) Based on the preset rule base, multi-perspective feature data is processed in a hierarchical manner to obtain difficult example samples.

[0018] 3) Difficult sample samples are analyzed and labeled using retrieval enhancement generation method and multi-agent workflow to form labeled dataset.

[0019] 4) Construct a multimodal large model and adjust the multimodal large model using labeled datasets to obtain a reference strategy model.

[0020] 5) The reference policy model is optimized by reinforcement learning to obtain the optimized multimodal large model.

[0021] 6) Deploy the optimized multimodal large model on the server to provide judgment results for online content security decisions.

[0022] Furthermore, the multi-perspective feature data of the community users includes user basic profile features, device fingerprint features, account risk status features, social relationship network features, content publishing behavior features, time-series behavior features, interaction behavior features, and overall statistical features.

[0023] Furthermore, the steps for obtaining difficult example samples are as follows:

[0024] A1 uses a pre-defined rule base to perform preliminary processing on multi-perspective feature data, identifying normal samples and remaining samples.

[0025] The preset rule base stores blacklist rules, whitelist rules, and statistical threshold rules.

[0026] A2 uses a graph computation model and an ensemble learning model to perform secondary processing on the remaining samples, identifying high-risk and difficult samples. The steps are as follows:

[0027] A21 constructs a graph structure based on users' social relationships and interactive behaviors, where nodes represent users and edges represent relationships or behaviors between users.

[0028] A22 uses the GraphSAGE algorithm to generate a vector for each node in the graph structure, as shown below:

[0029] (1)

[0030] In the formula, Indicates the node index. Indicates the aggregation level index. , They represent the first The node at the th , Characteristics of the layer. Indicates the first Layer weight matrix. This represents a non-linear activation function. This represents the join function. This represents an aggregate function. This represents the neighbor node index. Indicates the first The neighboring node is at the _th ... Characteristics of the layer. Indicates the first The set of neighboring nodes of a node.

[0031] A23 uses the Fraudard algorithm to mine abnormal subgraphs in the graph structure and obtains the gang risk characteristics of each node.

[0032] The Fraudar algorithm defines a weighted suspicion function and finds the subgraph with the highest weighted total suspicion score, thereby obtaining the gang risk characteristics of each node.

[0033] The weighted suspicion function is as follows:

[0034] (2)

[0035] In the formula, S represents a subset of candidate suspicious nodes. Indicates the number of nodes. This represents the user node index. Indicates the index of the object node. Indicates the first The user node to the Edges of an object node. Represents the set of edges. Indicates the degree of an object node. This indicates the smoothing term. Indicates a suspicious density. This represents the weighted total suspicion level.

[0036] A24 uses the Personalized PageRank algorithm to find potential samples that are related to known high-risk samples.

[0037] The iterative formula for the Personalized PageRank algorithm is as follows:

[0038] (3)

[0039] In the formula, This represents the steady-state probability vector. This indicates the probability of restarting. Let represent the random walk transition matrix. This represents a personalized vector that is non-zero only on the set of source nodes, which are known high-risk samples.

[0040] A25 stitches together the gang risk characteristics and potential samples of each node into the graph structure, and uses an ensemble learning model to perform secondary processing on the remaining samples to identify high-risk samples and difficult samples.

[0041] The objective function of the ensemble learning model is as follows:

[0042] (4)

[0043] In the formula, This represents the objective function of the ensemble learning model. Indicates the sample index. Indicates the iterative index. Represents a tree index. This represents the total number of samples. This represents the loss function. Indicates the first After the nth iteration The predicted value for each sample. Indicates the first The actual value of each sample.

[0044] Among them, the regularization term As shown below:

[0045] (5)

[0046] In the formula, Indicates the first The number of leaves on a tree. and Both represent regularization coefficients. Indicates the first The weight vector of each leaf node. Represents the norm.

[0047] Furthermore, the retrieval enhancement generation method includes a hybrid retrieval approach that combines text retrieval based on BM25 and vector indexing based on GroupHNSW.

[0048] The scoring function for the BM25-based text retrieval is as follows:

[0049] (6)

[0050] In the formula, This indicates a query. Represents a document. Indicates the document length. express. This indicates a keyword index. This indicates the total number of keywords. Indicates the first in the query One keyword. This represents the inverse document frequency weight. Keywords In the document The number of times it appears in the text. and All of these indicate adjustable parameters. This represents the average length of the document collection.

[0051] The GroupHNSW-based vector index uses a multimodal coding model to map text, image, and video data to the same vector space, and achieves nearest neighbor retrieval of vectors through a grouped hierarchical small-world graph index.

[0052] The multimodal coding model is shown below:

[0053] (7)

[0054] In the formula, This represents a multimodal fusion vector. , , These represent the weights for text, images, and videos, respectively. , , Representing text, image, and video vectors respectively Normalized value.

[0055] in, normalization function As shown below:

[0056] (8)

[0057] In the formula, This represents the Euclidean norm.

[0058] Furthermore, the agents in the multi-agent workflow include, but are not limited to: content analysis agent, behavior pattern agent, social network agent, and rule consistency agent.

[0059] Furthermore, the multimodal large model includes the LLaVA model, the Qwen-VL model, and the Intern-VL model.

[0060] Furthermore, the objective function of the reference strategy model is as follows:

[0061] (9)

[0062] In the formula, This represents a labeled dataset. This indicates the input data. This indicates the output data. Indicates model parameters. The parameter is A multimodal large model. It expresses expectation.

[0063] Furthermore, the reinforcement learning optimization methods include direct preference optimization and group-relative policy optimization.

[0064] The loss function for direct preference optimization is shown below:

[0065] (10)

[0066] In the formula, This indicates the input data. This indicates the preferred output data. This indicates that the output data was rejected. This represents the Sigmoid function. This represents the loss function. The parameter is A multimodal large model. This represents the reference strategy model. It expresses expectation. This represents the temperature coefficient.

[0067] The objective function for optimizing the relative strategy of the group is shown below:

[0068] (11)

[0069] (12)

[0070] (13)

[0071] (14)

[0072] (15)

[0073] In the formula, This represents the objective function for optimizing the relative strategy of the group. This indicates that a query is being entered. This indicates the output index generated from the old strategy. This indicates the query distribution. This indicates the generation of the first generation from the old strategy. One output. This represents the old strategy model. This represents the parameters of the old strategy. Indicates the strategy parameters. It expresses expectation. This represents the total number of outputs generated from the old strategy. This represents the cutoff objective function. This represents the KL penalty coefficient. This represents the KL divergence. The parameter is A multimodal large model. This represents the reference strategy model. Indicates the importance sampling ratio. The representation form has a relative advantage. Indicates the interval parameter. This indicates that the ratio is limited to a range. Inside. Indicates the first The reward for each output. This represents the average reward within the group. This represents the standard deviation of the rewards within the group. Indicates a stable term. , Both indicate adjustable weights. This indicates a general quality award. This indicates a reward based on the rules.

[0074] A system for the above method includes: a feature acquisition and fusion module, a rule processing module, a graph computation and ensemble learning module, a data flywheel and knowledge base module, a multimodal supervised fine-tuning module, a multimodal reinforcement learning module, and an online inference and service module.

[0075] The feature acquisition and fusion module is used to acquire multi-perspective feature data of community users and construct the corresponding graph structure.

[0076] The rule processing module performs preliminary processing on multi-perspective feature data through a preset rule library to identify normal samples and remaining samples.

[0077] The graph computation and ensemble learning module performs secondary processing on the remaining samples through the graph computation model and the ensemble learning model to identify high-risk samples and difficult samples.

[0078] The data flywheel and knowledge base module analyzes and labels difficult sample data through retrieval-enhanced generation and multi-agent workflow to form a labeled dataset.

[0079] The multimodal supervised fine-tuning module is used to construct a large multimodal model and adjust the large multimodal model using a labeled dataset to obtain a reference policy model.

[0080] The multimodal reinforcement learning module is used to perform reinforcement learning optimization on the reference policy model to obtain an optimized multimodal large model.

[0081] The online reasoning and service module is used to deploy the optimized multimodal large model on the server side to provide judgment results for online content security decisions.

[0082] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for multimodal, multi-stage, and multi-perspective community user profiling and content understanding oriented towards content security.

[0083] The technical effectiveness of this invention is undeniable. This invention introduces a hybrid reward signal consisting of preference rewards and rule rewards into GRPO, converting the high-confidence judgments of the first-stage rule layer and machine learning layer into programmable rule rewards. This guides the large model to strictly adhere to risk control rules while learning human preferences. Compared with existing technologies, this invention significantly reduces inference resource costs while maintaining high accuracy and recall, and achieves end-to-end interpretability and self-evolutionary capabilities from the rule layer to multimodal large model training and inference.

[0084] By employing a hierarchical and distributed architecture, graph computing and ensemble learning, multimodal large model supervised fine-tuning, and reinforcement learning optimization combined with rule rewards, this invention significantly reduces overall inference costs while ensuring risk identification accuracy and recall. It provides end-to-end interpretability from rules to large model training and inference, and achieves adaptive evolution of new risk patterns through a data flywheel mechanism.

[0085] The present invention has the following beneficial effects:

[0086] First, the hierarchical and distributed architecture significantly reduces costs. Approximately 90% of samples are processed at the rule-based layer and the traditional machine learning / deep learning layer, while only about 10% of complex samples are handled by the multimodal large model. This achieves a resource allocation of "approximately 50% rule-based processing + approximately 40% machine learning + approximately 10% large model," effectively reducing inference costs and latency while ensuring recognition accuracy. The system can dynamically adjust the proportions of each stage based on business load and risk tolerance, achieving a controllable balance between cost and effectiveness.

[0087] Second, the multimodal large-scale model addresses the shortcomings of traditional methods in understanding images, videos, and other content through deep understanding. Through supervised fine-tuning and reinforcement learning optimization, the model can identify marketing QR codes, external links, and exaggerated or vulgar image elements in images, as well as verbal promotions, suggestive actions, and scenes in videos, achieving semantic-level understanding of complex content, particularly significantly improving recognition capabilities in mixed "text + image" and "text + video" scenarios.

[0088] Third, interpretability is introduced through the rule layer and graph model. Meanwhile, the structured judgment results generated in the first stage are used to construct training data and reward signals, ensuring that the training data sources for the second and third stages are clearly traceable, forming an end-to-end interpretable closed loop. For specific samples, the system can trace back their processing paths and key feature contributions at the rule layer, graph computation layer, and large model layer.

[0089] Fourth, through the RAG and multi-agent data flywheel mechanism, this invention can continuously learn from emerging attack patterns, slang expressions, and cross-modal avoidance paths, accumulating representative difficult examples into a knowledge base and high-quality training samples, enabling the model to continuously iterate, improve the ability to identify new violations, extend the model life cycle, and reduce the cost of frequent manual intervention and retraining.

[0090] Fifth, by introducing hybrid reward signals and group-relative policy optimization, this invention ensures that the multimodal large model does not deviate from the key rule red line while aligning with human preferences during the reinforcement learning stage, avoiding the risk of "sacrificing security for optimizing metrics", which is beneficial for deploying large models in highly sensitive scenarios such as content security. Attached Figure Description

[0091] Figure 1 A schematic diagram of the overall architecture of a multimodal, multi-stage, and multi-perspective community user profiling and content understanding system;

[0092] Figure 2 This is a schematic diagram of the structure of the hybrid data processing and feature engineering layer in Stage 1;

[0093] Figure 3 A schematic diagram of the optimization process for Stage 3 reinforcement learning. Detailed Implementation

[0094] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0095] Example 1:

[0096] See Figures 1 to 3 A multimodal, multi-stage, and multi-perspective method for community user profiling and content understanding oriented towards content security includes the following steps:

[0097] 1) Obtain multi-perspective feature data of community users.

[0098] 2) Based on the preset rule base, multi-perspective feature data is processed in a hierarchical manner to obtain difficult example samples.

[0099] 3) Difficult sample samples are analyzed and labeled using retrieval enhancement generation method and multi-agent workflow to form labeled dataset.

[0100] The annotations include account classification categories and a complete reasoning process similar to a thought chain (cot).

[0101] The account categories include marketing accounts, suspected marketing accounts, low-quality content creators, and normal users.

[0102] 4) Construct a multimodal large model and adjust the multimodal large model using labeled datasets to obtain a reference strategy model.

[0103] 5) The reference policy model is optimized by reinforcement learning to obtain the optimized multimodal large model.

[0104] 6) Deploy the optimized multimodal large model on the server to provide judgment results for online content security decisions.

[0105] The online content includes videos, text, and images shared by community users, as well as user interactions, including likes and comments.

[0106] The determination result includes the account's classification category and a complete reasoning process similar to a thought chain (cot).

[0107] Example 2:

[0108] A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method for content security is described in Example 1. Further, the multi-perspective feature data of the community users includes basic user profile features, device fingerprint features, account risk status features, social relationship network features, content publishing behavior features, time-series behavior features, interaction behavior features, and overall statistical features.

[0109] Example 3:

[0110] A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method for content security is provided. The main technical content is described in any one of Embodiments 1 and 2. Furthermore, the steps for obtaining difficult example samples are as follows:

[0111] A1 uses a pre-defined rule base to perform preliminary processing on multi-perspective feature data, identifying normal samples and remaining samples.

[0112] The term "normal sample" refers to a sample with a clearly defined classification category.

[0113] The preset rule base stores blacklist rules, whitelist rules, and statistical threshold rules.

[0114] A2 uses a graph computation model and an ensemble learning model to perform secondary processing on the remaining samples, identifying high-risk and difficult samples. The steps are as follows:

[0115] A21 constructs a graph structure based on users' social relationships and interactive behaviors, where nodes represent users and edges represent relationships or behaviors between users.

[0116] A22 uses the GraphSAGE algorithm to generate a vector for each node in the graph structure, as shown below:

[0117] (1)

[0118] In the formula, Indicates the node index. Indicates the aggregation level index. , They represent the first The node at the th , Characteristics of the layer. Indicates the first Layer weight matrix. This represents a non-linear activation function. This represents the join function. This represents an aggregate function. This represents the neighbor node index. Indicates the first The neighboring node is at the _th ... Characteristics of the layer. Indicates the first The set of neighboring nodes of a node.

[0119] A23 uses the Fraudard algorithm to mine abnormal subgraphs in the graph structure and obtains the gang risk characteristics of each node.

[0120] The Fraudar algorithm defines a weighted suspicion function and finds the subgraph with the highest weighted total suspicion score, thereby obtaining the gang risk characteristics of each node.

[0121] The weighted suspicion function is as follows:

[0122] (2)

[0123] In the formula, S represents a subset of candidate suspicious nodes. Indicates the number of nodes. This represents the user node index. Indicates the index of the object node. Indicates the first The user node to the Edges of an object node. Represents the set of edges. Indicates the degree of an object node. This indicates the smoothing term. Indicates a suspicious density. This represents the weighted total suspicion level.

[0124] A24 uses the Personalized PageRank algorithm to find potential samples that are related to known high-risk samples.

[0125] The iterative formula for the Personalized PageRank algorithm is as follows:

[0126] (3)

[0127] In the formula, This represents the steady-state probability vector. This indicates the probability of restarting. Let represent the random walk transition matrix. This represents a personalized vector that is non-zero only on the set of source nodes, which are known high-risk samples.

[0128] A25 stitches together the gang risk characteristics and potential samples of each node into the graph structure, and uses an ensemble learning model to perform secondary processing on the remaining samples to identify high-risk samples and difficult samples.

[0129] The term "high-risk sample" refers to samples that account for a relatively high proportion of a certain category during classification.

[0130] The term "difficult sample" refers to samples that cannot be classified when there is no prominent category.

[0131] The objective function of the ensemble learning model is as follows:

[0132] (4)

[0133] In the formula, This represents the objective function of the ensemble learning model. Indicates the sample index. Indicates the iterative index. Represents a tree index. This represents the total number of samples. This represents the loss function. Indicates the first After the nth iteration The predicted value for each sample. Indicates the first The actual value of each sample.

[0134] Among them, the regularization term As shown below:

[0135] (5)

[0136] In the formula, Indicates the first The number of leaves on a tree. and Both represent regularization coefficients. Indicates the first The weight vector of each leaf node. Represents the norm.

[0137] Example 4:

[0138] A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method for content security is provided. The main technical contents are described in any one of Embodiments 1 to 3. Furthermore, the retrieval enhancement generation method includes a hybrid retrieval method based on BM25 text retrieval and based on GroupHNSW vector index.

[0139] The scoring function for the BM25-based text retrieval is as follows:

[0140] (6)

[0141] In the formula, This indicates a query. Represents a document. Indicates the document length. express. This indicates a keyword index. This indicates the total number of keywords. Indicates the first in the query One keyword. This represents the inverse document frequency weight. Keywords In the document The number of times it appears in the text. and All of these indicate adjustable parameters. This represents the average length of the document collection.

[0142] The GroupHNSW-based vector index uses a multimodal coding model to map text, image, and video data to the same vector space, and achieves nearest neighbor retrieval of vectors through a grouped hierarchical small-world graph index.

[0143] The multimodal coding model is shown below:

[0144] (7)

[0145] In the formula, This represents a multimodal fusion vector. , , These represent the weights for text, images, and videos, respectively. , , Representing text, image, and video vectors respectively Normalized value.

[0146] in, normalization function As shown below:

[0147] (8)

[0148] In the formula, This represents the Euclidean norm.

[0149] Example 5:

[0150] A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method for content security is provided. The main technical contents are described in any one of Embodiments 1 to 4. Furthermore, the agents in the multi-agent workflow include, but are not limited to: content analysis agent, behavior pattern agent, social network agent, and rule consistency agent.

[0151] Example 6:

[0152] A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method for content security is provided. The main technical contents are described in any one of Examples 1 to 5. Furthermore, the multimodal large model includes the LLaVA model, the Qwen-VL model, and the Intern-VL model.

[0153] Example 7:

[0154] A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method for content security is provided. The main technical content is described in any one of Examples 1 to 6. Furthermore, the objective function of the reference strategy model is as follows:

[0155] (9)

[0156] In the formula, This represents a labeled dataset. This indicates the input data. This indicates the output data. Indicates model parameters. The parameter is A multimodal large model. It expresses expectation.

[0157] Example 8:

[0158] A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method for content security is provided. The main technical contents are described in any one of Examples 1 to 7. Furthermore, the reinforcement learning optimization method includes direct preference optimization and group relative policy optimization.

[0159] The loss function for direct preference optimization is shown below:

[0160] (10)

[0161] In the formula, This indicates the input data. This indicates the preferred output data. This indicates that the output data was rejected. This represents the Sigmoid function. This represents the loss function. The parameter is A multimodal large model. This represents the reference strategy model. It expresses expectation. This represents the temperature coefficient.

[0162] The objective function for optimizing the relative strategy of the group is shown below:

[0163] (11)

[0164] (12)

[0165] (13)

[0166] (14)

[0167] (15)

[0168] In the formula, This represents the objective function for optimizing the relative strategy of the group. This indicates that a query is being entered. This indicates the output index generated from the old strategy. This indicates the query distribution. This indicates the generation of the first generation from the old strategy. One output. This represents the old strategy model. This represents the parameters of the old strategy. Indicates the strategy parameters. It expresses expectation. This represents the total number of outputs generated from the old strategy. This represents the cutoff objective function. This represents the KL penalty coefficient. This represents the KL divergence. The parameter is A multimodal large model. This represents the reference strategy model. Indicates the importance sampling ratio. The representation form has a relative advantage. Indicates the interval parameter. This indicates that the ratio is limited to a range. Inside. Indicates the first The reward for each output. This represents the average reward within the group. This represents the standard deviation of the rewards within the group. Indicates a stable term. , Both indicate adjustable weights. This indicates a general quality award. This indicates a reward based on the rules.

[0169] Example 9:

[0170] A system for the method described in any one of Embodiments 1 to 8 includes: a feature acquisition and fusion module, a rule processing module, a graph computation and ensemble learning module, a data flywheel and knowledge base module, a multimodal supervised fine-tuning module, a multimodal reinforcement learning module, and an online inference and service module.

[0171] The feature acquisition and fusion module is used to acquire multi-perspective feature data of community users and construct the corresponding graph structure.

[0172] The rule processing module performs preliminary processing on multi-perspective feature data through a preset rule library to identify normal samples and remaining samples.

[0173] The graph computation and ensemble learning module performs secondary processing on the remaining samples through the graph computation model and the ensemble learning model to identify high-risk samples and difficult samples.

[0174] The data flywheel and knowledge base module analyzes and labels difficult sample data through retrieval-enhanced generation and multi-agent workflow to form a labeled dataset.

[0175] The multimodal supervised fine-tuning module is used to construct a large multimodal model and adjust the large multimodal model using a labeled dataset to obtain a reference policy model.

[0176] The multimodal reinforcement learning module is used to perform reinforcement learning optimization on the reference policy model to obtain an optimized multimodal large model.

[0177] The online reasoning and service module is used to deploy the optimized multimodal large model on the server side to provide judgment results for online content security decisions.

[0178] Example 10:

[0179] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the multimodal, multi-stage, multi-perspective community user profiling and content understanding method for content security as described in any one of embodiments 1 to 8.

[0180] Example 11:

[0181] See Figures 1 to 3 A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method, system, and medium for content security, the main technical contents of which include:

[0182] The method includes the following steps:

[0183] (a) Multidimensional feature acquisition step, used to acquire multi-perspective feature data of community users, wherein the multi-perspective feature data includes at least user basic profile features, device fingerprint features, account risk status features, social relationship network features, content publishing behavior features, time-series behavior features, interaction behavior features, and overall statistical features obtained by comparing the above features with the overall user profile.

[0184] (b) A hybrid hierarchical processing step for hierarchically processing the multi-view feature data, including:

[0185] (b1) The multi-view feature data is processed in the first stage through a preset rule filtering layer to identify and directly determine the first type of user or content with high certainty.

[0186] (b2) Input the data that fails to be determined with high confidence in the rule filtering layer into the machine learning / deep learning layer, and perform the second stage processing through the graph computing model and ensemble learning model to identify the second type of user or content, output risk score and classification label, and mark the samples with insufficient confidence or large internal conflict as difficult case data.

[0187] (c) The steps for building a data flywheel, including:

[0188] (c1) Input the data marked as difficult in step (b2) into the data flywheel layer;

[0189] (c2) The data flywheel layer analyzes and annotates the difficult case data with high quality through Retrieval-Augmented Generation (RAG) and multi-agent workflow. RAG recalls cases related to the difficult case data from the historical knowledge base by combining a hybrid retrieval method based on BM25 text retrieval and a vector retrieval method based on GroupHNSW index. The multi-agent workflow uses the cases to perform collaborative reasoning and annotation on the difficult case data to form an annotated dataset.

[0190] (d) Supervised fine-tuning steps for multimodal models, including:

[0191] (d1) Using the labeled dataset, supervised fine-tuning of the basic multimodal large model is performed to adapt it to the understanding of multimodal content of text, images and videos in community scenarios, and a reference strategy model is obtained;

[0192] (e) Reinforcement learning optimization steps, including:

[0193] (e1) Construct a preference dataset and perform reinforcement learning optimization on the reference policy model through Direct Preference Optimization (DPO) or Group Relative Policy Optimization (GRPO) to enhance the model's decision details and generalization ability on boundary cases; wherein, when using GRPO, a set of outputs and corresponding rewards are generated for each query, and a relative advantage function is constructed based on the in-group rewards, and the magnitude is updated by shearing the target and constraining the KL divergence penalty.

[0194] The graph computation model in step (b2) includes analyzing the features of the social relationship network using the GraphSAGE algorithm and a personalized PageRank algorithm to generate user node embeddings and influence scores for relatively high-risk accounts, wherein GraphSAGE satisfies:

[0195]

[0196] In step (b2), the machine learning / deep learning layer detects high-density abnormal subgraphs by running the Fraudar algorithm on the constructed social relationship graph and interaction behavior graph, thereby identifying group mutual following, group likes, or group traffic-driving behaviors, and uses the corresponding group risk features as one of the input features to input the ensemble learning model for comprehensive judgment.

[0197] In step (c2), the hybrid retrieval method for RAG uses the following BM25 scoring function:

[0198]

[0199] in, For query, For the first in the query One keyword, Keywords in the document The number of times it appears in For document length, The average length of the document collection. and It is an adjustable parameter. This is the inverse document frequency weight.

[0200] In step (e1), when Group Relative Policy Optimization (GRPO) is used, the reward signal used to calculate the advantage function is... For a mixed reward signal, the following conditions must be met:

[0201]

[0202] in, The quality reward is based on a preference model or preference optimization. Based on the rule-based reward function constructed in step (b1) and the machine learning / deep learning layer in step (b2), when the model output conflicts with the high-confidence rule judgment, Take a larger negative value to strongly penalize the output during GRPO updates.

[0203] The relative advantage function in the GRPO The results were obtained by normalizing the within-group mean and standard deviation:

[0204]

[0205] in, For the first A mixed reward for each output. and These are the mean and standard deviation of the rewards for the same group, respectively. It is a stable term.

[0206] The objective function of the GRPO includes a shearing term and a KL penalty term, and its expected form is:

[0207]

[0208] in, The importance sampling ratio, For shearing function, In the query The KL divergence between the new strategy and the reference strategy. KL is the penalty coefficient.

[0209] In step (b), the number of samples processed by the rule filtering layer accounts for approximately 50% of the total number of samples, the number of samples processed by the machine learning / deep learning layer accounts for approximately 40% of the total number of samples, and the remaining approximately 10% of the samples enter the data flywheel construction step in step (c); the proportions can be adjusted according to business needs.

[0210] A community user profiling and content understanding system for implementing the above method, the system comprising:

[0211] (1) Feature acquisition and fusion module, used to collect and preprocess multi-perspective feature data of community users and construct the corresponding graph structure;

[0212] (2) Rule processing module, used to perform first-stage processing on the multi-view feature data according to the preset rule base, and output high-confidence rule judgment results and rule features;

[0213] (3) Graph computation and ensemble learning module, which is used to perform graph algorithms such as GraphSAGE, Fraudar, and personalized PageRank based on social relationship graphs and interaction behavior graphs, and output risk scores and classification labels in combination with ensemble learning models, while labeling hard sample samples;

[0214] (4) Data Flywheel and Knowledge Base Module, used to perform retrieval enhancement generation and multi-agent collaborative annotation on the difficult sample, and to build and maintain a high-quality labeled dataset and case knowledge base;

[0215] (5) Multimodal supervised fine-tuning module, used to train the labeled dataset to obtain a reference policy model;

[0216] (6) A multimodal reinforcement learning module, used to perform DPO and / or GRPO reinforcement learning optimization on the reference policy model based on the preference dataset and mixed reward signals to obtain the reinforced multimodal model;

[0217] (7) Online reasoning and service module, used to deploy the enhanced multimodal model, and together with the rule processing module and graph computing and ensemble learning module, to provide judgment results and strategy suggestions for online content security decision-making.

[0218] Example 12:

[0219] See Figures 1 to 3 A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method, system, and medium for content security, the main technical contents of which include:

[0220] Based on processing complexity and resource consumption, the architecture divides user profiling and content understanding tasks into three core stages, and achieves closed loops between stages through a data flywheel.

[0221] The first stage is a hybrid data processing and feature engineering layer (Stage 1). This stage, built upon a unified data layer, integrates multi-perspective user features, which are then processed sequentially through a rule-based filtering layer and a machine learning / deep learning layer. This process splits approximately 90% of the tasks, which have high determinism and low complexity. Specifically, the rule-based filtering layer handles about 50% of the highly deterministic samples, the machine learning / deep learning layer handles about 40% of the moderately complex samples, and the remaining approximately 10% of highly uncertain samples are designated as "hard cases" and enter the data flywheel layer.

[0222] The second stage is the multimodal large model supervised fine-tuning layer (Stage 2). Utilizing the high-quality structured labeled data produced in the first stage, domain-supervised fine-tuning is performed on the selected basic multimodal large model. This enables the model to accurately understand specific community slang, multimodal marketing formats (such as QR codes and referral cards), and short video ad-libs for attracting traffic, and to output interpretable risk assessment results.

[0223] The third stage is the multimodal large-scale model reinforcement learning optimization layer (Stage 3). Based on the supervised fine-tuning model, reinforcement learning from human feedback (RLHF) methods such as Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO) are used. In particular, a programmable rule reward function derived from the rule layer and machine learning layer in the first stage is introduced to further align the model with expert judgment criteria and improve the decision precision, stability and generalization ability in boundary scenarios.

[0224] In the above multi-stage architecture, the key technical solutions proposed in this embodiment include:

[0225] 1) The data layer design integrates multiple perspective features, including basic user profiles, device fingerprints, account risk status, social relationship networks, content publishing behavior, time-series behavior, interactive behavior, and overall statistical features, to achieve the transition from single-point features to three-dimensional profiles.

[0226] 2) A hybrid decision-making logic based on rules, graph computation, and ensemble learning, including using graph algorithms such as GraphSAGE, Fraudar, and Personalized PageRank (PPR) to identify gangs and mine relationships in social networks and interactive networks, and outputting comprehensive risk scores and class labels through models such as XGBoost or LightGBM.

[0227] 3) Based on the data flywheel constructed by Retrieval Augmentation Generation (RAG) and multi-agent workflow, the samples that cannot be determined in the first stage are recalled by the hybrid retrieval technology that combines BM25 and vector retrieval. Then, high-quality, multi-dimensional annotations are generated by multi-agent collaborative analysis to drive supervised fine-tuning and reinforcement learning of multimodal large models.

[0228] 4) Under the GRPO framework, a hybrid reward signal combining preference reward and rule reward is introduced to construct a normalized form of relative advantage. This not only learns human preferences but also strictly constrains the model to comply with high-confidence rules and risk control requirements, achieving a unity of "flexible preferences + rigid rules".

[0229] Example 13:

[0230] See Figures 1 to 3 A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method, system, and medium for content security, the main technical contents of which include:

[0231] System Overall Architecture

[0232] like Figure 1 As shown, the multimodal, multi-stage, and multi-perspective community user profiling and content understanding system for content security in this embodiment may include: a feature acquisition and fusion module, a rule processing module, a graph computation and ensemble learning module, a data flywheel and knowledge base module, a multimodal supervised fine-tuning module, a multimodal reinforcement learning module, and an online reasoning and service module, etc.

[0233] The feature acquisition and fusion module is used to collect and clean multi-perspective user features from data sources such as the community platform's log system, content storage system, social relationship system, and device information system, and encode them uniformly into structured feature vectors and graph structured data. This module may include a feature synchronization submodule, a feature verification submodule, and a feature service interface submodule, supporting offline batch processing and online feature querying.

[0234] The rule processing module is used to quickly determine the input samples based on a preset rule base, initially classify high-confidence normal samples and high-risk samples, and output some direct handling results and rule features. The rule base can be managed through the policy configuration platform, supporting canary releases and version rollbacks.

[0235] The graph computing and ensemble learning module, based on user social relationship graphs, interaction behavior graphs, and related attribute features, utilizes algorithms such as GraphSAGE, Fraudar, and personalized PageRank to generate node embeddings and gang risk indicators. It then combines these with ensemble learning models such as XGBoost or LightGBM to output a comprehensive risk score and category label. This module can be deployed on graph computing frameworks or distributed computing engines and supports periodic batch updates of graph embeddings.

[0236] The data flywheel and knowledge base module collects difficult examples that the rule layer and ensemble learning layer cannot determine with high confidence. It then performs in-depth analysis and annotation using retrieval enhancement generation and multi-agent workflows, transforming these difficult examples into high-quality training samples and searchable case studies. This module maintains a versioned knowledge base index and records the effective date and scope of each knowledge entry, facilitating time-series management of historical case studies.

[0237] The multimodal supervised fine-tuning module is used to perform supervised fine-tuning of the basic multimodal large model based on the aforementioned high-quality labeled data, enabling it to have targeted content security judgment capabilities for multimodal content such as images, text, and short videos. This module may include a data cleaning submodule, a sample construction submodule, and a training scheduling submodule.

[0238] The multimodal reinforcement learning module, based on supervised fine-tuning of the model, uses algorithms such as DPO and GRPO to construct a hybrid reward signal combining preference and rule-based rewards to enhance and optimize the model. This makes the model's judgments on risk levels, violation types, and handling suggestions in complex boundary scenarios more consistent with expert standards and platform rules. This module supports offline reinforcement learning experiments and some online A / B validation.

[0239] The online inference and service module is used to deploy the multimodal large model optimized by reinforcement learning as an online service. Together with the rule processing module and the graph computing and ensemble learning module, it forms an online decision engine to realize functions such as user profile tag generation, content quality assessment, risk identification and policy triggering, and feeds online feedback back to the data flywheel module for further training.

[0240] Through the modular design described above, this embodiment can be flexibly deployed according to different platform scales and business models. It can be centrally deployed in a single data center or deployed across multiple data centers in different regions, and supports multi-business calls through feature service and model service interfaces.

[0241] Example 14:

[0242] See Figures 1 to 3 A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method, system, and medium for content security, the main technical contents of which include:

[0243] Stage 1 Hybrid Data Processing and Feature Engineering Layer

[0244] Multidimensional feature input

[0245] In a preferred embodiment, the data layer of Stage 1 performs multi-view feature modeling on the user, specifically including but not limited to the following categories:

[0246] (1) Basic user profile features. These include username, personal signature, avatar type and suspiciousness, account authentication status, account registration time, registration channel, characteristics of bound mobile phone number or email address, and regional information, which are used to depict the user's basic attributes and registration background.

[0247] (2) Device fingerprint features. These include common device types (operating system, model), unique device identifier or fingerprint, device registration time, common IP range, IP location, IP risk profile (e.g., whether it hits a high-risk Autonomous System (AS)), device sharing degree (number of multiple accounts under the same device), device replacement frequency, etc., which are used to identify abnormal sharing and batch account farming behavior at the device level.

[0248] (3) Account risk status characteristics. These include historical ban records, traffic restriction or demotion records, being marked on black and gray market lists, historical appeals and review results, etc., which are used as prior risk information.

[0249] (4) Social network characteristics. These include user following lists, follower lists, mutual following relationships, following and unfollowing speeds, follower growth rate, shortest path lengths or co-occurrence communities with known high-risk accounts, etc., which are used to construct social relationship graphs and form the basis for gang identification.

[0250] (5) Content publishing behavior characteristics. This includes the cumulative number of published content, the number and percentage of deleted content, the publishing ratio of different content types (text, images, short videos, long articles, etc.), the distribution of publishing time, commonly used publishing city IPs, and the proportion of new content containing external links, QR codes, and contact information, which are used to identify centralized traffic diversion and mass production behavior.

[0251] (6) Temporal behavioral characteristics. These include the number of daily logins, the distribution of login time periods, the number of times IP or device is frequently switched, the proportion of content published in the early morning period (e.g., 0:00–5:00), and the number of times multiple pieces of content are published in a short period of time, which are used to characterize whether there are scripted or batch operation behaviors.

[0252] (7) Interactive behavior characteristics. These include the number of private messages sent, the number of private messages received, the number and percentage of private messages reported as spam or harassment by users, the number of comments and replies, the rhythm and distribution of liking, collecting, and following others' content, and the distribution of search keywords, etc., which are used to identify abnormal interaction trajectories and "spying" behavior patterns.

[0253] (8) Overall statistics and percentile characteristics. Compare the above behavioral statistics with the overall user population, calculate the percentile of the corresponding indicator on the overall population (e.g., a certain indicator is at the P90, P95 or P99 percentile), and the degree of deviation relative to similar user groups (such as the same registration time, the same vertical creators), to strengthen the significance of abnormal behavior.

[0254] The aforementioned features undergo missing value imputation, discretization, normalization, and embedding encoding through a feature governance process, and are uniformly input into Stage 1 in the form of vector features and graph structures.

[0255] Rule-based filtering layer

[0256] The rule-based filtering layer prioritizes samples with simple structures that can be clearly determined by rules. Its design goal is to cover at least approximately 50% of the samples while ensuring the recall of both high-risk and high-normal samples. This layer may include the following rule types:

[0257] (1) Blacklist rules. When a user's device fingerprint or IP hits the high-risk blacklist, the account has been confirmed as a member of a black and gray industry group, or the user has been banned multiple times in the past for serious violations, the account and its content are directly marked as high-risk without further large-scale model reasoning.

[0258] (2) Whitelist rules. For platform-certified institutional accounts, government accounts, and key cooperative brand accounts, in the absence of obvious abnormal behavior, they can be given priority as trusted accounts. If necessary, only sampling review will be conducted to save subsequent model resources.

[0259] (3) Statistical threshold rules. When the proportion of content published in the early morning exceeds the preset threshold and the content type is highly concentrated in the external link guidance type, it can be judged as a suspected marketing account; when the number of accounts registered and logged in by the same device in a short period of time exceeds the preset limit, it can be judged as a suspected mass account farming behavior; when similar private message content is sent to a large number of new users in a short period of time, it can be judged as a suspected spam private message behavior.

[0260] The output of the rule filtering layer includes: providing direct risk labels and handling suggestions for some samples; and adding rule triggering features (such as trigger rule ID, trigger count, and trigger time) to samples that cannot be judged with high confidence, which are then passed to subsequent machine learning / deep learning layers.

[0261] Machine learning / deep learning layers

[0262] For the approximately 40% of samples that the rule-based filtering layer failed to determine with high confidence, this embodiment uses a combination of graph computation and ensemble learning for unified modeling and scoring.

[0263] First, a graph-structured data structure is constructed based on users' social relationships and interactive behaviors. The nodes of the graph can be user accounts, and the edges can represent follow relationships, interaction relationships (comments, likes, shares), or common attributes (such as the same external link pointing to the same user). The edges can also have weights to indicate the intensity of interaction or similarity.

[0264] Secondly, the GraphSAGE algorithm is used to generate a vector representation for each node in the graph. A single-layer aggregation of GraphSAGE can be represented as:

[0265]

[0266] in, Represents a node In the Layer representation, For the original features of the node, For nodes The set of neighboring nodes, For aggregation functions (e.g., average aggregation). It is a non-linear activation function. This is a trainable weight matrix.

[0267] Next, the Fraudar algorithm is used to perform high-density anomaly subgraph mining on the above graph structure. Fraudar identifies suspicious groups with concentrated mutual following, concentrated likes, or concentrated backlinks by defining a weighted suspicion function to find the subgraph with the highest score under given constraints. In this embodiment, the algorithm is run on the social relationship graph and interaction graph to obtain the group risk score and suspicious subgraph affiliation label for each node, which are used as feature inputs for subsequent models.

[0268] The Fraudar algorithm is as follows:

[0269] Let the bipartite graph of platform interaction relationships be: ,in: For a set of user nodes, A collection of object nodes (such as content or accounts that have been liked, followed, or commented on). Let be the set of edges, representing the interaction relationships between users and objects.

[0270] object node The degree is (i.e., the number of edges connected to node j), introducing a constant c>0 as a smoothing term, the column degree weighting function is defined as:

[0271]

[0272] Therefore, for any edge User-object pairs, with their edge weights defined as follows:

[0273]

[0274] in Representing an edge Contribution to overall suspiciousness, highly popular objects ( (Larger) will be due to Larger values ​​are automatically downgraded, thus weakening the masquerade effect.

[0275] For a given subset of candidate suspicious nodes ,remember Let S be the number of nodes. Then, the Fraudard log weighted total suspicion degree and suspicion density used in this invention are defined as follows:

[0276]

[0277]

[0278] In practical algorithm applications, by searching among all candidate subsets S, it is possible to find the... The largest subgraph can identify suspicious groups in areas with significantly high local edge density that are not easily diluted by popular nodes, and can be used to discover suspected groups of accounts involved in traffic boosting, online trolling, or black and gray market activities.

[0279] Furthermore, using the Personalized PageRank (PPR) algorithm, a set of known high-risk accounts is used as a personalized "source node." The influence scores of other nodes in the graph relative to the source node are calculated to identify potential associates or affected users highly correlated with existing high-risk accounts. The iterative form of PPR is as follows:

[0280]

[0281] in, Let be the steady-state probability vector. Let be the random walk transition matrix. For the probability of restarting, A personalized vector that is non-zero only on the set of source nodes.

[0282] After obtaining the aforementioned graph embedding and graph risk features, this embodiment concatenates them with user basic profile features, behavioral statistical features, and rule triggering features, and inputs them into an ensemble learning model, such as XGBoost. XGBoost uses an additive model with a regularization term, and its objective function can be expressed as:

[0283]

[0284] in, Indicates the first Samples after round of iterations The predicted value, For loss function, Indicates the first A tree, As a regularization term, we can take:

[0285]

[0286] in, For the first The number of leaves on the trees, This is the weight vector of the leaf node. and The regularization coefficient is used to control the complexity of the model.

[0287] The output of the machine learning / deep learning layer includes: risk scores and classification labels for most samples (such as "marketing account", "suspected marketing account", "low-quality content producer", "normal user" etc.), and a very small number of samples with insufficient confidence or many internal conflicts are marked as "difficult cases" and enter the data flywheel layer.

[0288] Data Flywheel Layer and RAG Retrieval

[0289] For the approximately 10% of samples that are difficult to determine with high confidence by both the rule layer and the ensemble learning layer, this embodiment uses them as key "difficult cases" to construct a RAG-driven data flywheel.

[0290] First, a multimodal knowledge base containing typical historical cases is constructed. The sample records in this knowledge base include: user multi-perspective features, original content and multimodal information (text, images, video summaries), conclusions annotated by humans or experts, handling results, and key explanatory information.

[0291] Secondly, a hybrid retrieval method was used for case recall.

[0292] Based on BM25 keyword retrieval, the BM25 scoring function can be expressed as:

[0293]

[0294] in, For query, For the first in the query One keyword, Keywords in the document The number of times it appears in For document length, The average length of the document collection. and It is an adjustable parameter. This is the standard inverse document frequency weight.

[0295] Semantic recall based on vector retrieval utilizes a multimodal coding model to map text, images, and video summaries to the same vector space, and achieves nearest neighbor retrieval of high-dimensional vectors through a Hierarchical Navigable Small WorldGraph Index with Grouping (GroupHNSW). GroupHNSW organizes nodes according to a grouping strategy based on the classic HNSW structure to improve retrieval efficiency and load balancing on large-scale vector databases.

[0296] The multimodal coding model is as follows:

[0297] 1. Multimodal embedding coding

[0298] For the k-th community content item, its text, image, and video inputs are denoted as follows:

[0299]

[0300] By mapping text encoders, image encoders, and video encoders to the same vector space, we obtain...

[0301]

[0302] in, , , These are text, image, and video encoding functions, respectively. , , For the corresponding trainable parameters; For the embedded dimension; The dimension is The real vector space, that is, each vector has Each real number component.

[0303] To standardize the scale, the modal vectors were processed. Normalization:

[0304]

[0305] in This represents the Euclidean norm.

[0306] Multimodal fusion representation, linearly fused according to modal weights and then normalized again:

[0307]

[0308] in , , These are the weighting coefficients for text, image, and video modalities; Indicates to proceeding again Normalization; vector This is the unified multimodal embedding representation of the content, where the weight of a certain modality can be 0 when it is missing.

[0309] 2. Contrastive Learning Loss (InfoNCE)

[0310] A contrastive learning loss based on InfoNCE is used to train the multimodal encoder. The similarity function is defined as follows:

[0311]

[0312]

[0313] in The normalized vector, For inner product, It is a temperature coefficient used to adjust the smoothness of the distribution.

[0314] In a training mini-batch containing N image and text samples, the normalized text and image vectors of the nth sample are respectively... The InfoNCE loss for text retrieval in the image orientation is defined as:

[0315]

[0316] in The numerator represents the number of samples within the batch, the denominator is the similarity between correctly paired images and text, and the denominator is the sum of the similarities between the text and all images within the batch, achieving the effect of "bringing positive samples closer and pushing negative samples further away." A symmetrical loss can be defined for the image retrieval text direction, and the weighted sum of the two yields the overall multimodal contrastive loss. .

[0317] 3. Parameter Update

[0318] Let the parameter set of the entire multimodal encoder be .

[0319]

[0320] The parameters are updated using a gradient descent-based approach, with the following iterative form:

[0321]

[0322] in The parameter values ​​are for the k-th iteration. Let be the learning rate in the k-th round. This represents the gradient vector obtained over all parameters. Through multiple iterations, the contrastive loss is... The convergence results in a multimodal coding model aligned in a common vector space.

[0323] Then, a lightweight rearrangement model is used to refine the candidate cases obtained from the above hybrid retrieval, and the most relevant set of historical samples and their judgment reasons are output as the "external memory" of the current difficult cases.

[0324] Building upon this foundation, this embodiment constructs a multi-agent workflow for high-quality annotation of difficult examples. The multi-agents include, but are not limited to: a content analysis agent (focusing on the semantics of text, images, and video content), a behavior pattern agent (focusing on behavior sequences and abnormal patterns), a social network agent (focusing on social relationships and group structures), and a rule consistency agent (focusing on the degree of conformity with existing rule and policy bases). Each agent shares the aforementioned RAG retrieval results and knowledge base information, and through serial or parallel collaborative reasoning, provides comprehensive judgment conclusions and explanatory basis for difficult examples, forming structured annotated data and providing high-quality samples for Stage 2 and Stage 3 training.

[0325] The data flywheel operates as follows: new samples enter → Stage 1 judgment → hard examples are accumulated → RAG retrieval of historical cases → multi-agent collaborative annotation → training data generation → training or updating the Stage 2 / 3 model → changes in sample distribution after model update → re-identification of new hard examples. Through this cycle, this embodiment can continuously absorb new attack styles and multimodal evasion techniques, enabling the system to maintain stable risk identification performance.

[0326] Example 15:

[0327] A method, system, and medium for multimodal, multi-stage, and multi-perspective community user profiling and content understanding oriented towards content security are described in Embodiment 14. Further, in this embodiment, a basic model with visual-linguistic or multimodal capabilities is selected as the multimodal basic large model, such as LLaVA, Qwen-VL, or Intern-VL.

[0328] The training data is sourced from the high-quality labeled samples generated by the Stage 1 data flywheel. Each sample can contain the following: an input part and an output part.

[0329] The input part includes text content (such as posts, comments, private message fragments), corresponding images or video key frames, user multi-perspective feature summaries (such as risk scores, important behavior statistics), and necessary context information (several interaction records before and after). The output part includes judgment results in a unified format (risk level, violation type, handling suggestions) and explanatory notes (such as key clues triggered, similarities with historical cases).

[0330] During the supervised fine-tuning process, standard autoregressive or instruction fine-tuning objectives are adopted to perform conditional generation training on the model, enabling it to output standardized judgment conclusions and explanations when given multi-modal inputs. The training objective can be formalized as:

[0331]

[0332] where, is the labeled dataset, is the multi-modal input, is the target output, is the parameter of the multi-modal large model.

[0333] Through this step, the multi-modal large model obtains domain knowledge for the content ecosystem of a specific community platform, can understand local jargon (such as "transfer money to me", "check the homepage", "get on the bus", etc.), deformed homophonic words, and implicit marketing elements in images, and makes up for the understanding defects of traditional models in complex multi-modal content.

[0334] The multi-modal model obtained after SFT training is denoted as the reference policy and serves as the benchmark model for KL constraint in the subsequent reinforcement learning stage. <​​​​​​​​​​​​​(Including content and user characteristics).

[0338] Direct Preference Optimization (DPO) is an algorithm that fine-tunes a policy model directly based on preference data without explicitly training the reward model. Its loss function can be written as:

[0339]

[0340] in, For the strategy model to be optimized, As a reference model after SFT, For temperature coefficient, This refers to the Sigmoid function.

[0341] By minimizing the aforementioned DPO loss, the policy model's ability to generate preferred outputs can be improved. This reduces the probability of generating rejected outputs. The probability is used to fine-tune the model along the preference direction, making its decision closer to the expert's choice.

[0342] Group relative strategy optimization and relative advantage based on GRPO

[0343] In a preferred embodiment, to further improve the model's performance in complex inference tasks and high-dimensional output spaces, a Group Relative Policy Optimization (GRPO) algorithm is introduced. Unlike traditional PPO dependency value function estimation, GRPO queries each input... Generate a set of outputs Then, construct an advantage function based on the relative performance within the group.

[0344] For a given query From the old strategy sampling Output And calculate the reward for each output. This embodiment constructs the relative advantage using the normalized group mean and standard deviation:

[0345]

[0346] in, and These are the mean and standard deviation of the group rewards, respectively. To prevent stable terms with a denominator of zero.

[0347] Define importance sampling ratio:

[0348]

[0349] Similar to the shearing objective function of PPO, this embodiment adopts the following shearing advantage form:

[0350]

[0351]

[0352] in, This indicates that the ratio is limited to a range. This approach aligns with the pruning strategy proposed in PPO, effectively suppressing excessive policy updates and improving training stability.

[0353] Based on this, this embodiment presents the optimization objective of GRPO:

[0354]

[0355] in, To query the distribution, The KL divergence between the new strategy and the reference strategy under this query. KL is the penalty coefficient, used to constrain the policy from deviating excessively from the SFT reference policy.

[0356] Hybrid reward signal and rule-based reward construction

[0357] One of the core innovations of this embodiment lies in the reward signal in GRPO. The construction of this method is as follows. Traditional RLHF often relies solely on rewards generated by the preference model, while this embodiment explicitly incorporates the outputs of the rule layer and machine learning layer in the first stage into the reward function to maintain strict adherence to platform rules and hard constraints during reinforcement learning.

[0358] This embodiment defines the hybrid reward signal as follows:

[0359]

[0360] in, This represents the general quality reward obtained based on preference models or implicit preference estimation. This represents the rule-based reward constructed based on the first-stage rule layer and machine learning layer. These are adjustable weighting coefficients.

[0361] In a preferred embodiment, the rule-based reward function The construction includes the following principles:

[0362] (1) When the model outputs When the judgment result is consistent with the hard rule judgment given by the rule layer and has a high confidence level, a positive reward is given (e.g. (or higher), encouraging models to follow well-validated rules.

[0363] (2) When there is a significant conflict between the model output and the hard rule judgment, for example, the rule layer has determined that the account or device has been placed on a serious blacklist but the model outputs "normal user", a larger negative reward is given (e.g. ), and severely punish violations of key rules.

[0364] (3) When there is still uncertainty in the rule layer and the machine learning layer, only neutral or slight rewards are given, mainly based on preference rewards. This determines the optimization direction for the sample.

[0365] Through the above hybrid reward design, this embodiment incorporates "programmable rule reward" and "human or preference model reward" into the construction of the advantage function during the GRPO training process. This allows the model to learn nuanced human preferences and platform risk tolerance on the one hand, and strictly follow key risk control rules on the other, achieving a balance between high security and high generalization.

[0366] Example 17:

[0367] A multimodal, multi-stage, and multi-perspective community user profiling and content understanding method, system, and medium for content security are disclosed. The main technical details are described in any one of embodiments 14 to 16. Furthermore, in a typical implementation, the method of this embodiment can be implemented in a distributed computing environment. The system includes at least one processor, a memory, and a network interface. The memory stores a computer program that can run on the processor. When executed by the processor, this program implements the community user profiling and content understanding method described in the foregoing embodiments.

[0368] In terms of engineering deployment, the rule processing and graph computation and ensemble learning modules of Stage 1 can be deployed in a high-throughput, low-latency online service cluster to ensure that most requests are processed in milliseconds to tens of milliseconds. The large model inference capabilities of Stage 2 and Stage 3 can be deployed in the form of microservices or inference acceleration clusters, and only called on the selected complex samples, thereby controlling peak resource consumption while ensuring the effect.

Claims

1. A multi-modal multi-stage multi-view community user profiling and content understanding method for content security, characterized in that, Includes the following steps: 1) Obtain multi-perspective feature data of community users; 2) Based on a pre-defined rule base, multi-perspective feature data is processed in a hierarchical manner to obtain difficult example samples; 3) Difficult example samples are analyzed and labeled using retrieval enhancement generation and multi-agent workflow to form a labeled dataset; 4) Construct a multimodal large model and adjust the multimodal large model using labeled datasets to obtain a reference strategy model; 5) Optimize the reference policy model using reinforcement learning to obtain the optimized multimodal large model; 6) Deploy the optimized multimodal large model on the server to provide judgment results for online content security decisions.

2. The method of claim 1, wherein, The multi-perspective feature data of community users includes user basic profile features, device fingerprint features, account risk status features, social relationship network features, content publishing behavior features, time-series behavior features, interaction behavior features, and overall statistical features.

3. The method for multimodal, multi-stage, and multi-perspective community user profiling and content understanding oriented towards content security, as described in claim 1, is characterized in that... The steps for obtaining difficult example samples are as follows: A1 uses a pre-defined rule base to perform preliminary processing on multi-perspective feature data, identifying normal samples and remaining samples; The preset rule base stores blacklist rules, whitelist rules, and statistical threshold rules; A2 uses a graph computation model and an ensemble learning model to perform secondary processing on the remaining samples, identifying high-risk and difficult samples. The steps are as follows: A21 constructs a graph structure based on users' social relationships and interactive behaviors, where nodes represent users and edges represent relationships or behaviors between users. A22 uses the GraphSAGE algorithm to generate a vector for each node in the graph structure, as shown below: （1） In the formula, Indicates the node index. Indicates the aggregation level index. , They represent the first The node at the th , Features of the layer; Indicates the first Layer weight matrix; Represents a non-linear activation function; Indicates the join function; Represents aggregate functions; Indicates the neighbor node index; Indicates the first The neighboring node is at the _th ... Features of the layer; Indicates the first The set of neighboring nodes of a node; A23 uses the Fraudard algorithm to mine abnormal subgraphs in the graph structure and obtains the gang risk characteristics of each node; The Fraudar algorithm defines a weighted suspicion function and finds the subgraph with the highest weighted total suspicion score, thereby obtaining the gang risk characteristics of each node. The weighted suspicion function is as follows: （2） In the formula, S represents a subset of candidate suspicious nodes; Indicates the number of nodes; Indicates the user node index; Indicates the index of the object node; Indicates the first The user node to the Edges of an object node; Represents the set of edges; Indicates the degree of an object node; Indicates the smoothing term; Indicates a suspicious density; Indicates the weighted total suspiciousness; A24 uses the Personalized PageRank algorithm to find potential samples that are related to known high-risk samples; The iterative formula for the Personalized PageRank algorithm is as follows: （3） wherein, denotes the steady state probability vector; denotes the restart probability; denotes the random walk transition matrix; denotes the personalized vector that is non-zero only on the source node set, which are known high risk samples; A25 stitches together the gang risk characteristics and potential samples of each node into the graph structure, and uses an ensemble learning model to perform secondary processing on the remaining samples to identify high-risk samples and difficult samples. The objective function of the ensemble learning model is as follows: （4） In the formula, This represents the objective function of the ensemble learning model; Indicates the sample index; Indicates an iterative index; Represents a tree index; Indicates the total number of samples; Represents the loss function; Indicates the first After the nth iteration Predicted values ​​for each sample; Indicates the first The actual value of each sample; where the regularization term As shown below: （5） In the formula, Indicates the first The number of leaves on a tree; and Both represent regularization coefficients; Indicates the first Leaf node weight vectors; Represents the norm.

4. The content security oriented multi-modal multi-stage multi-view community user profiling and content understanding method according to claim 1, characterized in that, The retrieval enhancement generation method includes a hybrid retrieval approach that combines text retrieval based on BM25 and vector indexing based on GroupHNSW. The scoring function for the BM25-based text retrieval is as follows: （6） In the formula, Indicates a query; Represents a document. Indicates document length; express; Indicates a keyword index; Indicates the total number of keywords; Indicates the first in the query One keyword; Indicates the inverse document frequency weight; Keywords In the document The number of times it appears in; and All of these indicate adjustable parameters; This represents the average length of the document collection; The GroupHNSW-based vector index uses a multimodal coding model to map text, image, and video data to the same vector space, and achieves nearest neighbor retrieval of vectors through a grouped hierarchical small-world graph index. The multimodal coding model is shown below: （7） In the formula, Represents a multimodal fusion vector; , , These represent the weights for text, images, and videos, respectively. , , Representing text, image, and video vectors respectively Normalized value; in, normalization function As shown below: （8） In the formula, denotes the Euclidean norm.

5. The content security oriented multi-modal multi-stage multi-view community user profiling and content understanding method according to claim 1, characterized in that, The agents in the multi-agent workflow include, but are not limited to: content analysis agent, behavior pattern agent, social network agent, and rule consistency agent.

6. The content security oriented multi-modal multi-stage multi-view community user profiling and content understanding method according to claim 1, characterized in that, The multimodal large model includes the LLaVA model, the Qwen-VL model, and the Intern-VL model.

7. The method for multimodal, multi-stage, and multi-perspective community user profiling and content understanding oriented towards content security, as described in claim 1, is characterized in that... The objective function of the reference strategy model is as follows: （9） In the formula, This represents a labeled dataset. Indicates input data; Indicates the output data; Indicates model parameters; The parameter is A multimodal large model; It expresses expectation.

8. The content security oriented multi-modal multi-stage multi-view community user profiling and content understanding method according to claim 1, characterized in that, The reinforcement learning optimization methods include direct preference optimization and group-relative policy optimization. The loss function for direct preference optimization is shown below: （10） In the formula, Indicates input data; Indicates the preferred output data; This indicates that the output data was rejected. This represents the Sigmoid function; Represents the loss function; The parameter is A multimodal large model; Represents the reference strategy model; Expressing expectations; Indicates the temperature coefficient; The objective function for optimizing the relative strategy of the group is shown below: （11） （12） （13） （14） （15） In the formula, This represents the objective function for optimizing the group-relative strategy. Indicates the input query; This represents the output index generated from the old strategy; Indicates query distribution; This indicates the generation of the first generation from the old strategy. One output; Represents the old strategy model; Indicates the old strategy parameters; Indicates the strategy parameters; Expressing expectations; This represents the total number of outputs generated from the old strategy; Represent the objective function for shearing; Indicates the KL penalty coefficient; Indicates the KL divergence; The parameter is A multimodal large model; Represents the reference strategy model; Indicates the importance sampling ratio; The relative advantage of representational construction; Indicates an interval parameter; This indicates that the ratio is limited to a range. Inside; Indicates the first The reward for each output; This represents the average reward within the group; This represents the standard deviation of the group's rewards; Indicates a stable term; , Both indicate adjustable weights; Indicates a general quality award; This indicates a reward based on the rules.

9. A system for use in the method according to any one of claims 1 to 8, characterized in that, include: The module includes: Feature Acquisition and Fusion, Rule Processing, Graph Computation and Ensemble Learning, Data Flywheel and Knowledge Base, Multimodal Supervised Fine-tuning, Multimodal Reinforcement Learning, and Online Inference and Service. The feature acquisition and fusion module is used to acquire multi-perspective feature data of community users and construct the corresponding graph structure; The rule processing module performs preliminary processing on multi-view feature data through a preset rule library to identify normal samples and remaining samples; The graph computation and ensemble learning module performs secondary processing on the remaining samples through the graph computation model and the ensemble learning model to identify high-risk samples and difficult samples. The data flywheel and knowledge base module analyzes and labels difficult example samples through retrieval-enhanced generation and multi-agent workflow to form a labeled dataset; The multimodal supervised fine-tuning module is used to construct a large multimodal model and adjust the large multimodal model using a labeled dataset to obtain a reference policy model; The multimodal reinforcement learning module is used to perform reinforcement learning optimization on the reference policy model to obtain an optimized multimodal large model; The online reasoning and service module is used to deploy the optimized multimodal large model on the server side to provide judgment results for online content security decisions.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the multimodal, multi-stage, multi-perspective community user profiling and content understanding method for content security as described in any one of claims 1-8.

Images