Business content processing method and device, program product and electronic equipment

Abstract

The invention discloses a business content processing method and device, a computer program product and electronic equipment, and relates to the technical field of data processing. The business content processing method comprises the following steps: acquiring business content data, and clustering the business content data to obtain a plurality of clusters; according to the target business content data in each cluster, judging whether each cluster has a content risk or not; the target business content data is content data of a clustering center; determining a content publishing user corresponding to the service content data in the first cluster as an illegal user; the first cluster is a cluster with content risk in a plurality of clusters; and pushing the service content data in the second cluster. The second clustering cluster is a clustering cluster without content risk in the plurality of clustering clusters. According to the method and the device, the business content data is clustered, risk identification is carried out on the cluster, the risk content with high aggregation can be continuously sensed and locked, and meanwhile, the risk identification efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of data processing technology. More specifically, the embodiments of the present disclosure relate to a business content processing method, a business content processing device, a computer program product, and an electronic device. Background Art

[0002] This section is intended to provide a background or context to embodiments of the invention that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] When users publish content on an Internet platform, the platform needs to detect whether it contains content security risks in order to maintain the network environment.

[0004] In related technologies, sensitive words, rules, deep learning and other technologies are usually used to intercept single content. Not only is the interception efficiency low, but it is also difficult to intercept risky content that is highly confrontational and changes rapidly.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present disclosure provides a business content processing method, a business content processing device, a computer program product and an electronic device, thereby at least to a certain extent solving the problem of low efficiency and great difficulty in intercepting risky content in related technologies.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to a first aspect of the present disclosure, a business content processing method is provided, the method comprising: acquiring business content data, and clustering the business content data to obtain a plurality of clusters; judging whether each of the clusters has content risks based on target business content data in each of the clusters; the target business content data is the content data of the cluster center; determining a content publishing user corresponding to the business content data in a first cluster as an illegal user; the first cluster is a cluster among the plurality of clusters that has content risks; pushing the business content data in a second cluster; the second cluster is a cluster among the plurality of clusters that has no content risks.

[0009] In an exemplary embodiment of the present disclosure, the acquiring of business content data and clustering of the business content data to obtain a plurality of clustering clusters include: acquiring business content data corresponding to a target business; filtering the business content data corresponding to the target business according to preset rules to obtain filtered business content data; and clustering the filtered business content data to obtain a plurality of clustering clusters.

[0010] In an exemplary embodiment of the present disclosure, the business content data corresponding to the target business is screened according to preset rules to obtain the screened business content data, including: determining a content publishing user set according to the business content data corresponding to the target business; determining a suspected user from the content publishing user set according to user characteristics of each user in the content publishing user set; and screening out the business content data published by the suspected user from the business content data corresponding to the target business to obtain the screened business content data.

[0011] In an exemplary embodiment of the present disclosure, judging whether each of the clusters has content risks based on the target business content data in each of the clusters includes: performing similarity matching between each sample content data in the white sample library and the target business content data in the cluster to be detected; the white sample library stores content data with a risk lower than a first preset threshold; the cluster to be detected is any cluster among the multiple clusters; if the white sample library contains sample content data that matches the target business content data in the cluster to be detected, determining that the cluster to be detected has no content risk.

[0012] In an exemplary embodiment of the present disclosure, the method further includes: if there is no sample content data in the white sample library that matches the target business content data in the cluster to be detected, performing similarity matching on each sample content data in the black sample library and the target business content data in the cluster to be detected; the black sample library stores content data with a risk higher than a second preset threshold; if there is sample content data in the black sample library that matches the target business content data in the cluster to be detected, determining that there is a content risk in the cluster to be detected.

[0013] In an exemplary embodiment of the present disclosure, the method also includes: if there is no sample content data in the black sample library that matches the target business content data in the cluster to be detected, risk identification is performed on the target business content data in the cluster to be detected through a classification model and / or a large model to obtain a risk identification result of the cluster to be detected.

[0014] In an exemplary embodiment of the present disclosure, the method also includes: if the risk identification result of the cluster to be detected is that there is no risk, determining the characteristic statistical data of the cluster to be detected according to the attribute characteristics corresponding to each business content data in the cluster to be detected; if the characteristic statistical data of the cluster to be detected meets the preset alarm condition, pushing the cluster to be detected to a manual review terminal so that the cluster to be detected can be manually reviewed by the manual review terminal.

[0015] According to a second aspect of the present disclosure, a business content processing device is disclosed, the device comprising: a content clustering module, used to acquire business content data, and cluster the business content data to obtain a plurality of clusters; a risk judgment module, used to judge whether each of the clusters has content risks according to target business content data in each of the clusters; the target business content data is the content data of the cluster center; a violation determination module, used to determine a content publishing user corresponding to the business content data in a first cluster as a violating user; the first cluster is a cluster with content risks among the plurality of clusters; a content push module, used to push the business content data in a second cluster; the second cluster is a cluster without content risks among the plurality of clusters.

[0016] According to a third aspect of an embodiment of the present disclosure, a computer program product is disclosed, including a computer program, which, when executed by a processor, implements the business content processing method of the first aspect and possible implementation methods thereof.

[0017] According to the fourth aspect of an embodiment of the present disclosure, an electronic device is disclosed, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the business content processing method of the above-mentioned first aspect and its possible implementation method by executing the executable instructions.

[0018] The technical solution disclosed in this disclosure has the following beneficial effects:

[0019] In the above business content processing process, business content data is obtained, and the business content data is clustered to obtain multiple clusters; based on the target business content data in each cluster, it is determined whether each cluster has content risks; the target business content data is the content data of the cluster center; the content publishing user corresponding to the business content data in the first cluster is determined as a violating user; the first cluster is a cluster with content risks among multiple clusters; the business content data in the second cluster is pushed; the second cluster is a cluster without content risks among multiple clusters. The present disclosure clusters business content data and identifies risks for clusters, which can continuously perceive and lock risky content with strong aggregation, while improving risk identification efficiency; by batch stripping normal business data, the user content monitoring cost can be further reduced.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram showing a flow chart of a method for processing business content in this exemplary embodiment;

[0023] Figure 2 A schematic diagram of a process of clustering business content data in this exemplary embodiment is shown;

[0024] Figure 3 A method for identifying business content risks in this exemplary embodiment is shown;

[0025] Figure 4 A schematic diagram of a structure of risk identification based on text clustering in this exemplary embodiment is shown;

[0026] Figure 5 A structural block diagram of a business content processing device in this exemplary embodiment is shown;

[0027] Figure 6 An electronic device for implementing the above-mentioned service content processing method in this exemplary embodiment is shown.

[0028] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0029] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0031] According to an embodiment of the present disclosure, a business content processing method, a business content processing device, a computer program product, and an electronic device are proposed.

[0032] Any number of elements in the drawings is for illustration and not limitation, and any naming is for distinction only and does not have any limiting meaning.

[0033] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0034] In the related technologies disclosed in the present invention, single content is intercepted through sensitive words, rules, deep learning and other technologies. Not only is the interception efficiency low, but it is also difficult to intercept risky content that is highly confrontational and changes rapidly.

[0035] Based on the above problems, the present invention proposes a business content processing method, which clusters business content data and identifies risks in clusters. It can continuously perceive and lock risk content with strong aggregation, while improving risk identification efficiency; by batch stripping normal business data, the user content monitoring cost can be further reduced.

[0036] Application Scenario Overview

[0037] It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this regard. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario. For example, it can be applied to social content review scenarios with strong confrontation, fast-changing speech, but with a certain degree of aggregation, and the present disclosure does not specifically limit this.

[0038] Exemplary Methods

[0039] In combination with the above application scenarios, refer to Figure 1 The business content processing method according to the exemplary embodiment of the present disclosure is described.

[0040] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for processing business content according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the business content processing method may include:

[0041] Step S110, acquiring business content data, and clustering the business content data to obtain a plurality of clusters;

[0042] Step S120, judging whether each cluster has content risk according to the target business content data in each cluster; the target business content data is the content data of the cluster center;

[0043] Step S130, determining the content publishing user corresponding to the business content data in the first cluster as a violating user; the first cluster is a cluster with content risks among the multiple clusters;

[0044] Step S140: Pushing the business content data in the second cluster; the second cluster is a cluster without content risk among the multiple clusters.

[0045] Implementation Figure 1 The business content processing method shown clusters business content data and identifies risks in clusters. It can continuously sense and lock risk content with strong aggregation, while improving risk identification efficiency. By stripping normal business data in batches, user content monitoring costs can be further reduced.

[0046] These steps are described in detail below.

[0047] In step S110 , business content data is acquired, and the business content data is clustered to obtain a plurality of clusters.

[0048] Among them, business content data can be content data published by users on the Internet platform, which may include but is not limited to text, pictures, voice, video and other types of content.

[0049] Optionally, business content data may be obtained from sources such as business real-time databases and event logs.

[0050] Optionally, a specific type of business content data can be obtained in order to review a specific type of business content. Exemplarily, text data within a specified time range under a specified business in a target network platform can be obtained. Specifically, based on a data warehouse tool (e.g., Hive), the required part can be extracted from the data stream of the target network platform.

[0051] Optionally, when acquiring business content data, the business content data may also be associated with the user characteristics of the content publisher and / or the traffic characteristics at the time of publishing, etc., to assist in risk identification and positioning. User characteristics may include but are not limited to user level, nickname, historical penalties, registration time, etc., and traffic characteristics may include but are not limited to ip (Internet Protocol, Internet Protocol), system type, system version, device type, device identification, application software version, etc.

[0052] In an optional embodiment, if Figure 2 As shown, the above-mentioned acquisition of business content data and clustering of the business content data to obtain multiple clusters can be achieved through the following steps:

[0053] Step S210, obtaining business content data corresponding to the target business;

[0054] Step S220, filtering the business content data corresponding to the target business according to preset rules to obtain filtered business content data;

[0055] Step S230: clustering the filtered business content data to obtain a plurality of clusters.

[0056] The target business may be a pre-specified Internet business that needs to be reviewed, such as business related to user messages, user interactions, etc., which is not specifically limited in the present disclosure.

[0057] The preset rule may be a pre-set data cleaning rule. Optionally, the business content data may be deduplicated and abnormal time fields may be deleted to clean the business content data.

[0058] After obtaining the business content data corresponding to the target business, preliminary screening of the obtained business content data can further reduce the amount of audit tasks and improve audit efficiency.

[0059] In an optional implementation, the above-mentioned filtering of the business content data corresponding to the target business according to preset rules to obtain the filtered business content data can be achieved through the following steps: determining a content publishing user set according to the business content data corresponding to the target business; determining suspected users from the content publishing user set according to user characteristics of each user in the content publishing user set; and filtering out the business content data published by the suspected users from the business content data corresponding to the target business to obtain the filtered business content data.

[0060] Exemplarily, the content publisher of the acquired business content data may be determined according to the user identifier associated with the business content data corresponding to the target business, and the content publishing user set may include the content publisher of the acquired business content data.

[0061] Among them, suspected users refer to users who may be risky and can be determined based on the user characteristics of each user in the content publishing user set.

[0062] For example, it can be determined whether the user characteristics of each user in the content publishing user set meet the pre-set suspicious user determination conditions, and the users who meet the suspicious user determination conditions can be regarded as suspicious users. The suspicious user determination conditions may include but are not limited to the existence of historical penalty records, abnormal user activity status, unusual login, registration time less than the preset duration, etc., which are not specifically limited in this disclosure.

[0063] The business content data published by the suspected user is screened out from the business content data corresponding to the target business, so as to conduct targeted review and processing on the business content data published by the suspected user, thereby reducing the amount of review and processing tasks.

[0064] Taking the business content data as text as an example, the business content data can be clustered by text clustering technology to obtain multiple clusters. It should be noted that text clustering is an unsupervised learning method used to divide a set of text data into several groups (clusters) so that the texts in the same cluster are similar in a certain sense, while the texts in different clusters are not similar.

[0065] Taking the business content data as text as an example, in an optional implementation, the above-mentioned clustering of the business content data to obtain multiple clusters can be achieved through the following steps: a text-to-vector model can be used to vectorize the business content data to obtain a text vector of the business content data, and a clustering algorithm can be used to cluster the text vector of the business content data to obtain multiple clusters.

[0066] Optionally, sample data required for training the text-steering model may be collected, including training data and test data; based on the training data set, the text-steering model may be trained using unsupervised, semi-supervised or supervised learning methods; and the trained text-steering model may be tested based on the test data.

[0067] Exemplarily, when the model is trained in an unsupervised manner, the training data collection process is simple, less manpower is required, and only the text samples need to be cleaned to avoid too many repeated similar samples. Optionally, the SimCSE (SimpleContrastive Learning of Sentence Embe ddings, based on simple contrast learning sentence embedding) method can be used to train the bert (Bidirectional Encoder Representations from Transformers, bidirectional encoder representation) model. It should be noted that the SimCSE method is a contrast learning method for generating high-quality sentence embeddings. The unsupervised SimCSE needs to input a sentence and then predict itself in the contrast learning framework, using standard dropout (random inactivation) as noise to generate different embeddings of the same text as similar sample pairs; using embeddings of different texts as different sample pairs. Among them, embedding is a technology for mapping data to a vector space, which is widely used in fields such as natural language processing and computer vision, and is mainly used to represent the characteristics of data so that these features can be processed more effectively by machine learning models.

[0068] Exemplarily, when a supervised learning method is used to train a model, text pairs with annotation information may be prepared, including similar text pairs and different text pairs. Similar text pairs may be obtained through manual annotation or data enhancement.

[0069] Optionally, when testing the trained text vector model, sample pairs with labeled information can be used for testing to compare whether the distances between similar sample pairs and the distances between different sample pairs are different. The text vector distance can be calculated using methods such as Euclidean distance and cosine distance, which are not specifically limited in this disclosure.

[0070] Optionally, similar sample pairs can use the same type of black market rhetoric.

[0071] Optionally, different clustering algorithms such as k-means (k-means clustering algorithm), DBSCAN (Density-Based Spatial Clustering of Applications with Noise), hierarchical clustering, etc. can be used to cluster the text vectors of the business content data to obtain multiple clusters.

[0072] Among them, the k-means clustering algorithm is an iterative clustering analysis algorithm that can repeatedly divide data into multiple clusters according to a certain distance function.

[0073] Among them, the DBSCAN clustering algorithm can classify semantically dense texts into one category and regard semantically discrete texts as noise, and the total number of categories does not need to be pre-defined, which is in line with business logic. Exemplarily, a machine learning library (such as python Scikit-learn) can be used to implement DBSCAN clustering.

[0074] Among them, hierarchical clustering completes clustering by generating a series of nested clustering trees.

[0075] Optionally, after the business content data is clustered, characteristic statistical data of each cluster may be determined according to attribute characteristics corresponding to each business content data in the cluster to be detected, such as user characteristics, traffic characteristics, etc., for subsequent use.

[0076] Optionally, feature statistical data may include but are not limited to: the number of samples contained in the cluster sample_cnt, the number of users contained in the cluster user_cnt, the representative content of the center of the cluster center_text, the text vector of the representative content of the center of the cluster center center_vector, the proportion of illegal user traffic in the cluster user_risk_rate, the proportion of abnormal device traffic in the cluster device_risk_rate, the average number of Chinese characters in the cluster content average_chinese_cnt, the proportion of whitelist user traffic in the cluster content white_rate, the number of low-level users, the number of IPs, etc. The present disclosure does not make specific limitations on this.

[0077] In step S120, it is determined whether each cluster has content risk according to the target business content data in each cluster; the target business content data is the content data of the cluster center.

[0078] Exemplarily, it can be determined whether the target business content data of each cluster has content risks. If the target business content data has content risks, the cluster can be considered a cluster with content risks; if the target business content data does not have risks, the cluster can be considered a cluster without content risks.

[0079] The content data of the cluster center can be the most representative content data in the cluster, for example, the representative content of the cluster center is center_text. Since the content similarity within the same cluster is high, judging whether the cluster has risks based on the target business content data in the cluster can reduce the risk identification cost to a certain extent.

[0080] In an optional implementation, the above-mentioned determination of whether each cluster has content risk based on the target business content data in each cluster can be achieved through the following steps: performing similarity matching between each sample content data in the white sample library and the target business content data in the cluster to be detected; storing content data with a risk lower than a first preset threshold in the white sample library; the cluster to be detected is any cluster among multiple clusters; if there is sample content data in the white sample library that matches the target business content data in the cluster to be detected, it is determined that the cluster to be detected does not have content risk.

[0081] Among them, the first preset threshold can be a critical value for measuring whether the content does not have risks. When the content risk is lower than the critical value, it can be considered that the content does not have risks. When the content risk is equal to or higher than the critical value, it can be considered that the content has the possibility of risks. The specific setting can be based on experience, and the present disclosure does not make specific limitations on this.

[0082] In actual application, the white sample library can store some pre-set content that is known not to involve risks, such as "good morning", "hello" and other known low-risk content, so as to filter out clusters that do not involve risks through the white sample library, so as to avoid misjudgment of risks, reduce accidental damage to regular content data, and further narrow the scope of risk identification.

[0083] Specifically, by traversing the clusters, the target service content data in each cluster is matched with each sample content data in the white sample library for similarity, so as to filter out clusters without risks.

[0084] Specifically, when performing similarity matching between the sample content data in the white sample library and the target business content data in the cluster to be detected, the target business content data in the cluster to be detected can be compared with the sample content data in the white sample library for similarity. If the similarity detection results between the sample content data in the white sample library and the target business content data in the cluster to be detected are similar, it can be considered that there is no content risk in the cluster to be detected.

[0085] Optionally, the distance between the vector of the target business content data in the cluster to be detected and the vector of any sample content data in the white sample library can be calculated, such as cosine distance, Euclidean distance, etc., which is not specifically limited in the present disclosure; if the calculated distance is less than or equal to a preset distance threshold, it can be considered that the similarity detection result between the target business content data in the cluster to be detected and the sample content data is similar.

[0086] For example, the cosine distance between the text vector center_vector representing the center of the cluster and the vector white_vector of any sample content data in the white sample library can be obtained by calculating 1-cosine (center_vector, white_vector). When the cosine distance is less than or equal to the preset distance threshold, it can be considered that the similarity detection result between the target service content data in the cluster to be detected and the sample content data is similar.

[0087] Where cosine is the vector cosine angle, which can be calculated using the following formula (1).

[0088]

[0089] Among them, center_vector·white_vector can be calculated by the following formula (2).

[0090]

[0091] Wherein, |center_vector| can be calculated by the following formula (3).

[0092]

[0093] Wherein, |white_vector| can be calculated by the following formula (4).

[0094]

[0095] Among them, n is the vector dimension, center_vector i represents the i-th dimension of center_vector, white_vector i Represents the i-th dimension of white_vector vector.

[0096] Optionally, if there is sample content data in the white sample library that matches the target business content data in the cluster to be detected, it can be determined whether the cluster to be detected has abnormal fluctuations or exhibits obvious risk characteristics. If not, it can be determined that there is no content risk in the cluster to be detected; if so, human intervention can be performed to further confirm the risk situation to improve the accuracy of risk identification.

[0097] Furthermore, if the characteristic statistical data of the cluster to be detected meets the preset low-risk condition, it can be considered that there is no risk content in the cluster to be detected. Exemplarily, the preset low-risk condition may include but is not limited to: the average number of Chinese characters in the cluster content average_chinese_cnt is less than or equal to the corresponding threshold CT, the proportion of whitelist user traffic in the cluster content white_rate is greater than or equal to the corresponding threshold WRT, etc. Among them, the threshold CT and the threshold WRT can be set in advance based on experience, and the present disclosure does not specifically limit this.

[0098] In an optional implementation, the following steps may also be performed: if there is no sample content data in the white sample library that matches the target business content data in the cluster to be detected, perform similarity matching between each sample content data in the black sample library and the target business content data in the cluster to be detected; the black sample library stores content data with a risk higher than a second preset threshold; if there is sample content data in the black sample library that matches the target business content data in the cluster to be detected, determine that there is a content risk in the cluster to be detected.

[0099] Among them, the second preset threshold can be a critical value for measuring the risk of content. When the content risk is higher than the critical value, it can be considered that the content is risky. The specific setting can be based on experience, and the present disclosure does not make specific limitations on this.

[0100] In actual application, some pre-set known risk content can be stored in the black sample library, such as fraudulent content, unhealthy advertising content, illegal information, violent content, etc., so as to filter out risky clusters through the black sample library.

[0101] Specifically, the target service content data in the to-be-detected clusters may be matched with each sample content data in the black sample library for similarity, so as to filter out clusters with risks.

[0102] Specifically, when the sample content data in the black sample library is matched with the target business content data in the cluster to be detected, the target business content data in the cluster to be detected can be compared with the sample content data in the black sample library for similarity. If the similarity detection result between the sample content data in the black sample library and the target business content data in the cluster to be detected is similar, it can be considered that the cluster to be detected has content risk.

[0103] Optionally, the distance between the vector of the target business content data in the cluster to be detected and the vector of any sample content data in the black sample library can be calculated, such as cosine distance, Euclidean distance, etc., which is not specifically limited in the present disclosure; if the calculated distance is less than or equal to a preset distance threshold, it can be considered that the similarity detection result between the target business content data in the cluster to be detected and the sample content data is similar, thereby improving the accuracy of the risk perception capability.

[0104] For example, the cosine distance between the text vector center_vector representing the content of the cluster center and the vector black_vector of any sample content data in the black sample library can be obtained by calculating 1-cosine (center_vector, black_vector). When the cosine distance is less than or equal to the preset distance threshold, it can be considered that the similarity detection result between the target service content data in the cluster to be detected and the sample content data is similar.

[0105] In an optional implementation, the following steps can also be performed: if there is no sample content data in the black sample library that matches the target business content data in the cluster to be detected, the target business content data in the cluster to be detected is identified for risk through a classification model and / or a large model to obtain a risk identification result for the cluster to be detected.

[0106] Optionally, different classification models may be used for detection of different types of risks, such as fraud, sensitive topics, etc., which is not specifically limited in the present disclosure.

[0107] For example, historical online risk content data of volume brushing can be collected so as to train a classification model based on the historical risk content data of volume brushing. Specifically, for specific risk types, keywords or regular expressions can be used to intercept and obtain historical risk content data of volume brushing; models such as Bert can be used to supervise the training of classification models so as to identify risks of target business content data in the clusters to be detected through the trained classification model.

[0108] Optionally, large models such as llama3 and Qwen2 can be deployed locally to identify risks of target business content data in the clusters to be detected.

[0109] For example, lora (Low-Rank Adaptation, lightweight model fine-tuning) technology can be used to fine-tune the large model.

[0110] For example, a prompt word can be developed for the risk type to be detected in combination with the characteristics of historical risk content, and the large model can be used to identify risks based on the prompt word. When it is difficult to meet the recognition accuracy requirements by relying solely on the prompt word, the SFT (Supervised Fine-Tuning) method can be used to fine-tune the large model. SFT is a method that further trains the pre-trained model through supervised data, and is usually used for fine-tuning of a specific task to improve the performance of the model on the task.

[0111] Exemplarily, during the fine-tuning of a large model, PEFT methods, such as prompt tuning, prefix tuning, etc., can also be used to improve the model recognition effect, which is not specifically limited in the present disclosure. Among them, PEFT is a method for fine-tuning by adjusting fewer parameters based on a pre-trained model, with the goal of reducing the number of parameters that need to be adjusted during the fine-tuning process, thereby reducing computing costs and storage requirements, while maintaining or approaching the performance of full parameter fine-tuning.

[0112] It should be noted that in actual applications, for different types of risks, the difficulty of risk identification and confrontation can be judged, so as to select a mode that only uses traditional semantic recognition capabilities, a mode that only uses the recognition capabilities of large models, or a superposition mode for risk identification. For example, large models are better at dealing with risk types that are highly confrontational and good at changing expressions. For this type of risk, only large models can be used without introducing traditional semantic recognition capabilities to avoid omissions caused by traditional semantic models.

[0113] In an optional implementation, the following steps can also be performed: if the risk identification result of the cluster to be detected is that there is no risk, determine the characteristic statistical data of the cluster to be detected based on the attribute characteristics corresponding to each business content data in the cluster to be detected; if the characteristic statistical data of the cluster to be detected meets the preset alarm conditions, push the cluster to be detected to the manual review terminal, so that the cluster to be detected can be manually reviewed through the manual review terminal.

[0114] Exemplarily, the preset alarm conditions may include but are not limited to: the average number of Chinese characters in the cluster content average_chinese_cnt is greater than or equal to the alarm threshold ACT, the sample volume sample_cnt contained in the cluster is greater than or equal to the alarm threshold SCT, the proportion of illegal user traffic in the cluster user_risk_rate is greater than or equal to the alarm threshold URT, etc. Exemplarily, data analysis of online historical situations or experience can be performed to set various alarm thresholds, and this disclosure does not specifically limit this.

[0115] Clusters that meet the preset alarm conditions can be pushed to the manual review terminal for manual review and processing. Optional manual processing operations include but are not limited to: punishing specified content, punishing clusters, punishing related users, blacklisting related IPs, blacklisting related devices, etc. Clusters that do not meet the preset alarm conditions can have their business data content pushed to the platform for display in a normal state.

[0116] In step S130, the content publishing user corresponding to the business content data in the first cluster is determined as an illegal user; the first cluster is a cluster with content risks among the multiple clusters.

[0117] Specifically, after determining the first cluster, the content publishing user corresponding to the business content data in the first cluster can be determined as a violating user and punished for the violation. For example, for a specific high-risk violation type, the violating user who has been hit by multiple penalties can be pushed to the user penalty queue for human review or cross-verification with other user policies for automatic punishment.

[0118] In step S140, the business content data in the second cluster is pushed; the second cluster is a cluster without content risk among the multiple clusters.

[0119] Specifically, after the second cluster is determined, the business content data in the second cluster may be pushed.

[0120] For example, Figure 3 As shown, a business content risk identification method is provided, which may specifically include the following:

[0121] Step S301, obtaining business content data and preprocessing the business content data;

[0122] Among them, the preprocessing operations may include but are not limited to data deduplication, abnormal data removal and other data cleaning operations.

[0123] Step S302, clustering the pre-processed business content data to obtain clusters;

[0124] Step S303, determining whether the cluster is hit by the white sample library, if it is hit, executing step S304, if not, executing step S305;

[0125] Step S304, hiding the business content data in the cluster;

[0126] Under normal circumstances, hidden data will no longer be of concern, unless the level of hidden data fluctuates abnormally or exhibits obvious risk characteristics, in which case human intervention will be required to confirm the risk situation.

[0127] Step S305, determining whether the cluster is hit by the black sample library, if it is hit, executing step S311, if not, executing step S306;

[0128] Step S306, determining whether the cluster is hit by the traditional semantic recognition capability, if it is hit, executing step S311, if not, executing step S307; the traditional semantic recognition capability identifies whether the cluster has risks based on the classification model;

[0129] Step S307, determining whether the cluster is hit by the semantic recognition capability of the large model, if it is hit, executing step S311, if not, executing step S308; the semantic recognition capability of the large model is used to identify whether the cluster has risks;

[0130] Step S308, determining whether the characteristic statistical data of the clusters meets the preset alarm condition, if so, executing step S309, if not, executing step S310;

[0131] Step S309, triggering an alarm and pushing the business content data in the cluster to a manual review terminal;

[0132] Step S310, routinely push the business content data in the cluster to the display platform;

[0133] Step S311: Perform illegal processing on the business content data in the cluster.

[0134] For example, Figure 4 As shown, a structural diagram of risk identification based on text clustering is provided, including a data storage unit, a processing unit, and a model reasoning unit.

[0135] The data storage unit can store and manage data, including service content data, service feature data (such as user features, traffic features, etc.), black sample library, white sample library, etc. Optionally, the data storage unit can be implemented using a server, a cloud server, or other devices.

[0136] The processing unit can be used to implement data processing, analysis, logical judgment, calculation, etc. Exemplarily, the processing unit can implement functions such as data preprocessing, clustering, feature integration, vector comparison, penalty push, and alarm rule determination. Optionally, the processing unit can be implemented based on hardware such as spark clusters and CPU (Central Processing Unit) servers. Spark clusters are distributed computing frameworks.

[0137] The model inference unit can be used to implement the inference of the deep learning model. For example, it can implement model inference functions such as text steering, traditional model inference, and large model inference. Optionally, the model inference unit can be implemented using hardware devices such as GPU (graphics processing unit) servers.

[0138] The data storage unit provides business content data, which is pre-processed by the processing unit and sent to the model reasoning module to realize text conversion. After that, the vectors are clustered in the processing unit, and the business feature data (such as the attribute features corresponding to the business content data) provided by the data storage unit are integrated in the processing unit. After that, the processing unit compares and determines the vectors in the cluster cluster with the vectors in the white sample library and the black sample library, and pushes them to the model reasoning unit to obtain the detection results of the traditional model and the large model. According to the detection results, the cluster cluster is either automatically punished or pushed to the alarm detection logic to decide whether to alarm. The final results of risk perception and identification are output to the outside.

[0139] Figure 4 The structure shown can continuously sense and lock on risk content that is confrontational but highly clustered. The clusters screened by clustering and business feature data can be used to strip normal business content data, reducing the amount of over-inspection of large models and other models, reducing costs and accidental damage to regular samples caused by illusion problems, and improving the accuracy of risk perception capabilities.

[0140] Exemplary Devices

[0141] After introducing the method of the exemplary embodiment of the present disclosure, next, refer to Figure 5 A business content processing device according to an exemplary embodiment of the present disclosure will be described.

[0142] See also Figure 5 , Figure 5 FIG. 1 is a block diagram of a business content processing device according to an exemplary embodiment of the present disclosure. Figure 5 As shown, a business content processing device 500 of an exemplary embodiment of the present disclosure includes: a content clustering module 510, a risk judgment module 520, a violation determination module 530, and a content push module 540, wherein:

[0143] The content clustering module 510 is used to obtain business content data and cluster the business content data to obtain multiple clusters;

[0144] The risk judgment module 520 is used to judge whether each cluster has content risk according to the target business content data in each cluster; the target business content data is the content data of the cluster center;

[0145] The violation determination module 530 is used to determine the content publishing user corresponding to the business content data in the first cluster as a violating user; the first cluster is a cluster with content risks among the multiple clusters;

[0146] The content push module 540 is used to push the business content data in the second cluster; the second cluster is a cluster without content risk among the multiple clusters.

[0147] In an optional implementation, based on the aforementioned scheme, the content clustering module 510 includes: a data acquisition module, used to acquire business content data corresponding to the target business; a data screening module, used to screen the business content data corresponding to the target business according to preset rules to obtain the screened business content data; a cluster determination module, used to cluster the screened business content data to obtain multiple clusters.

[0148] In an optional implementation, based on the aforementioned scheme, the data screening module can be configured to: determine a content publishing user set based on the business content data corresponding to the target business; determine suspected users from the content publishing user set based on user characteristics of each user in the content publishing user set; and screen out business content data posted by suspected users from the business content data corresponding to the target business to obtain screened business content data.

[0149] In an optional implementation, based on the above solution, the risk judgment module 520 may include: a first matching module. The first matching module may be configured to: perform similarity matching between each sample content data in the white sample library and the target business content data in the cluster to be detected; the white sample library stores content data with a risk lower than a first preset threshold; the cluster to be detected is any cluster among the multiple clusters; if there is sample content data in the white sample library that matches the target business content data in the cluster to be detected, it is determined that the cluster to be detected does not have content risk.

[0150] In an optional implementation, based on the above solution, the risk judgment module 520 further includes: a second matching module. The second matching module can be configured as follows: if there is no sample content data in the white sample library that matches the target business content data in the cluster to be detected, perform similarity matching between each sample content data in the black sample library and the target business content data in the cluster to be detected; the black sample library stores content data with a risk higher than a second preset threshold; if there is sample content data in the black sample library that matches the target business content data in the cluster to be detected, determine that the cluster to be detected has content risk.

[0151] In an optional embodiment, based on the aforementioned scheme, the risk judgment module 520 also includes: a model identification module, which is used to identify the risks of the target business content data in the cluster to be detected through a classification model and / or a large model if there is no sample content data in the black sample library that matches the target business content data in the cluster to be detected, so as to obtain a risk identification result of the cluster to be detected.

[0152] In an optional implementation, based on the above solution, the risk judgment module 520 further includes: a risk warning module. The risk warning module can be configured as follows: if the risk identification result of the cluster to be detected is that there is no risk, determine the characteristic statistical data of the cluster to be detected according to the attribute characteristics corresponding to each business content data in the cluster to be detected; if the characteristic statistical data of the cluster to be detected meets the preset alarm condition, push the cluster to be detected to the manual review terminal, so that the cluster to be detected is manually reviewed by the manual review terminal.

[0153] By implementing the embodiments of the present disclosure, business content data is clustered and risks are identified for clusters, which can continuously perceive and lock risk content with strong aggregation, while improving risk identification efficiency; by batch stripping normal business data, user content monitoring costs can be further reduced.

[0154] It should be noted that although several modules or units of the business content processing device are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0155] Exemplary Program Products

[0156] After introducing the apparatus according to the exemplary embodiment of the present disclosure, next, a program product according to the exemplary embodiment of the present disclosure will be described.

[0157] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned business content processing method is implemented.

[0158] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0159] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0160] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0161] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure. Exemplarily, the following steps may be included:

[0162] Acquire business content data, and cluster the business content data to obtain multiple clusters;

[0163] According to the target business content data in each cluster, determine whether each cluster has content risk; the target business content data is the content data of the cluster center;

[0164] Determine the content publishing user corresponding to the business content data in the first cluster as a violating user; the first cluster is a cluster with content risks among the multiple clusters;

[0165] The business content data in the second cluster is pushed; the second cluster is a cluster without content risk among the multiple clusters.

[0166] In an optional implementation, based on the aforementioned scheme, the above-mentioned acquisition of business content data and clustering of the business content data to obtain multiple clustering clusters can be achieved through the following steps: acquiring the business content data corresponding to the target business; filtering the business content data corresponding to the target business according to preset rules to obtain filtered business content data; clustering the filtered business content data to obtain multiple clustering clusters.

[0167] In an optional implementation, based on the aforementioned scheme, the business content data corresponding to the target business is screened according to preset rules to obtain the screened business content data, which can be achieved through the following steps: determining a content publishing user set based on the business content data corresponding to the target business; determining suspected users from the content publishing user set based on user characteristics of each user in the content publishing user set; and screening out the business content data published by the suspected users from the business content data corresponding to the target business to obtain the screened business content data.

[0168] In an optional implementation, based on the aforementioned scheme, the above-mentioned determination of whether each cluster has content risk according to the target business content data in each cluster can be achieved through the following steps: performing similarity matching between each sample content data in the white sample library and the target business content data in the cluster to be detected; storing content data with a risk lower than a first preset threshold in the white sample library; the cluster to be detected is any cluster among multiple clusters; if there is sample content data in the white sample library that matches the target business content data in the cluster to be detected, it is determined that the cluster to be detected does not have content risk.

[0169] In an optional implementation, based on the aforementioned scheme, the following steps may also be performed: if there is no sample content data in the white sample library that matches the target business content data in the cluster to be detected, perform similarity matching between each sample content data in the black sample library and the target business content data in the cluster to be detected; the black sample library stores content data with a risk higher than a second preset threshold; if there is sample content data in the black sample library that matches the target business content data in the cluster to be detected, determine that there is a content risk in the cluster to be detected.

[0170] In an optional implementation, based on the aforementioned scheme, the following steps may also be performed: if there is no sample content data in the black sample library that matches the target business content data in the cluster to be detected, the target business content data in the cluster to be detected is identified for risk through a classification model and / or a large model to obtain a risk identification result for the cluster to be detected.

[0171] In an optional implementation, based on the aforementioned scheme, the following steps may also be performed: if the risk identification result of the cluster to be detected is that there is no risk, determine the characteristic statistical data of the cluster to be detected according to the attribute characteristics corresponding to each business content data in the cluster to be detected; if the characteristic statistical data of the cluster to be detected meets the preset alarm conditions, push the cluster to be detected to the manual review terminal so that the cluster to be detected can be manually reviewed through the manual review terminal.

[0172] By implementing the embodiments of the present disclosure, business content data is clustered and risks are identified for clusters, which can continuously perceive and lock risk content with strong aggregation, while improving risk identification efficiency; by batch stripping normal business data, user content monitoring costs can be further reduced.

[0173] Exemplary Electronic Devices

[0174] After introducing the service content processing method, apparatus, and program product according to the exemplary embodiment of the present disclosure, next, an electronic device according to another exemplary embodiment of the present disclosure is introduced.

[0175] The exemplary embodiment of the present disclosure also provides an electronic device capable of implementing the above-mentioned business content processing method. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program code. The processor executes the method in the exemplary embodiment by executing the executable instructions. In addition, the electronic device may also include a display for displaying a graphical user interface.

[0176] Reference below Figure 6 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Figure 6 The electronic device 600 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0177] like Figure 6 As shown, the electronic device 600 may include: a processor 610 , a memory 620 , a bus 630 , an I / O (input / output) interface 640 , a network adapter 650 , and a display 660 .

[0178] The memory 620 may include a volatile memory, such as a RAM 621, a cache unit 622, and may also include a non-volatile memory, such as a ROM 623. The memory 620 may also include one or more program modules 624, such program modules 624 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 624 may include each module in the above-mentioned device.

[0179] The processor 610 may include one or more processing units, for example, the processor 610 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0180] The processor 610 may be configured to execute executable instructions stored in the memory 620 , such as executing any one or more method steps in this exemplary embodiment.

[0181] Exemplarily, the processor 610 may perform the following steps:

[0182] Acquire business content data, and cluster the business content data to obtain multiple clusters;

[0183] According to the target business content data in each cluster, determine whether each cluster has content risk; the target business content data is the content data of the cluster center;

[0184] Determine the content publishing user corresponding to the business content data in the first cluster as a violating user; the first cluster is a cluster with content risks among the multiple clusters;

[0185] The business content data in the second cluster is pushed; the second cluster is a cluster without content risk among the multiple clusters.

[0186] In an optional implementation, based on the aforementioned scheme, the above-mentioned acquisition of business content data and clustering of the business content data to obtain multiple clustering clusters can be achieved through the following steps: acquiring the business content data corresponding to the target business; filtering the business content data corresponding to the target business according to preset rules to obtain filtered business content data; clustering the filtered business content data to obtain multiple clustering clusters.

[0187] In an optional implementation, based on the aforementioned scheme, the business content data corresponding to the target business is screened according to preset rules to obtain the screened business content data, which can be achieved through the following steps: determining a content publishing user set based on the business content data corresponding to the target business; determining suspected users from the content publishing user set based on user characteristics of each user in the content publishing user set; and screening out the business content data published by the suspected users from the business content data corresponding to the target business to obtain the screened business content data.

[0188] In an optional implementation, based on the aforementioned scheme, the above-mentioned determination of whether each cluster has content risk according to the target business content data in each cluster can be achieved through the following steps: performing similarity matching between each sample content data in the white sample library and the target business content data in the cluster to be detected; storing content data with a risk lower than a first preset threshold in the white sample library; the cluster to be detected is any cluster among multiple clusters; if there is sample content data in the white sample library that matches the target business content data in the cluster to be detected, it is determined that the cluster to be detected does not have content risk.

[0189] In an optional implementation, based on the aforementioned scheme, the following steps may also be performed: if there is no sample content data in the white sample library that matches the target business content data in the cluster to be detected, perform similarity matching between each sample content data in the black sample library and the target business content data in the cluster to be detected; the black sample library stores content data with a risk higher than a second preset threshold; if there is sample content data in the black sample library that matches the target business content data in the cluster to be detected, determine that there is a content risk in the cluster to be detected.

[0190] In an optional implementation, based on the aforementioned scheme, the following steps may also be performed: if there is no sample content data in the black sample library that matches the target business content data in the cluster to be detected, the target business content data in the cluster to be detected is identified for risk through a classification model and / or a large model to obtain a risk identification result for the cluster to be detected.

[0191] In an optional implementation, based on the aforementioned scheme, the following steps may also be performed: if the risk identification result of the cluster to be detected is that there is no risk, determine the characteristic statistical data of the cluster to be detected according to the attribute characteristics corresponding to each business content data in the cluster to be detected; if the characteristic statistical data of the cluster to be detected meets the preset alarm conditions, push the cluster to be detected to the manual review terminal so that the cluster to be detected can be manually reviewed through the manual review terminal.

[0192] By implementing the embodiments of the present disclosure, business content data is clustered and risks are identified for clusters, which can continuously perceive and lock risk content with strong aggregation, while improving risk identification efficiency; by batch stripping normal business data, user content monitoring costs can be further reduced.

[0193] The bus 630 is used to realize the connection between different components of the electronic device 600, and may include a data bus, an address bus, and a control bus.

[0194] The electronic device 600 can communicate with one or more external devices 700 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 640 .

[0195] The electronic device 600 can communicate with one or more networks through the network adapter 650. For example, the network adapter 650 can provide mobile communication solutions such as 3G / 4G / 5G, or provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 650 can communicate with other modules of the electronic device 600 through the bus 630.

[0196] The electronic device 600 may display a graphical user interface and the like through the display 660 .

[0197] although Figure 6 Not shown, other hardware and / or software modules may also be provided in the electronic device 600, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID (Redundant Arrays of Independent Disks) system, a tape drive, and a data backup storage system.

[0198] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0199] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module" or "system". Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0200] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A business content processing method, characterized in that: The method comprises: Acquire business content data, and cluster the business content data to obtain multiple clusters; According to the target business content data in each of the clusters, judging whether each of the clusters has content risk; the target business content data is the content data of the cluster center; Determine a content publishing user corresponding to the business content data in a first cluster as a violating user; the first cluster is a cluster with content risks among the multiple clusters; The business content data in the second cluster is pushed; the second cluster is a cluster without content risk among the multiple clusters.

2. The method according to claim 1, characterized in that: The acquiring of the business content data and clustering of the business content data to obtain a plurality of clusters includes: Obtain business content data corresponding to the target business; According to preset rules, the business content data corresponding to the target business is screened to obtain screened business content data; The filtered business content data is clustered to obtain a plurality of clusters.

3. The method according to claim 2, characterized in that The filtering of the business content data corresponding to the target business according to the preset rules to obtain the filtered business content data includes: Determining a content publishing user set according to the business content data corresponding to the target business; determining a suspected user from the content publishing user set according to user characteristics of each user in the content publishing user set; The business content data published by the suspected user is screened out from the business content data corresponding to the target business to obtain screened business content data.

4. The method according to claim 1, characterized in that: The step of judging whether each cluster has content risk according to the target business content data in each cluster includes: Perform similarity matching on each sample content data in the white sample library and the target business content data in the cluster to be detected; the white sample library stores content data with a risk lower than a first preset threshold; the cluster to be detected is any cluster among the multiple clusters; If there is sample content data in the white sample library that matches the target service content data in the cluster to be detected, it is determined that there is no content risk in the cluster to be detected.

5. The method according to claim 4, characterized in that The method further comprises: If there is no sample content data in the white sample library that matches the target service content data in the cluster to be detected, similarity matching is performed between each sample content data in the black sample library and the target service content data in the cluster to be detected; the black sample library stores content data with a risk higher than a second preset threshold; If there is sample content data in the black sample library that matches the target service content data in the cluster to be detected, it is determined that the cluster to be detected has content risk.

6. The method according to claim 5, characterized in that The method further comprises: If there is no sample content data in the black sample library that matches the target business content data in the cluster to be detected, risk identification is performed on the target business content data in the cluster to be detected through a classification model and / or a large model to obtain a risk identification result of the cluster to be detected.

7. The method according to claim 6, characterized in that The method further comprises: If the risk identification result of the cluster to be detected is that there is no risk, determining characteristic statistical data of the cluster to be detected according to the attribute characteristics corresponding to each business content data in the cluster to be detected; If the characteristic statistical data of the cluster to be detected meets the preset alarm condition, the cluster to be detected is pushed to the manual review terminal, so that the cluster to be detected is manually reviewed by the manual review terminal.

8. A business content processing device, characterized in that: The device comprises: A content clustering module, used to obtain business content data and cluster the business content data to obtain multiple clusters; A risk judgment module, used to judge whether each of the clusters has content risk according to target business content data in each of the clusters; the target business content data is the content data of the cluster center; A violation determination module, used to determine a content publishing user corresponding to the business content data in a first cluster as a violating user; the first cluster is a cluster with content risks among the multiple clusters; The content push module is used to push the business content data in the second cluster; the second cluster is a cluster without content risk among the multiple clusters.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

Images