Content risk identification method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a content risk identification method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a multimedia content which comprises at least two sub-contents in different modes; performing feature extraction on the sub-contents in the at least two modes to obtain sub-content feature information corresponding to different modes; performing fusion processing on the sub-content feature information corresponding to the different modes to obtain fusion feature information of the multimedia content; and performing content risk identification on the multimedia content according to the fused feature information to obtain risk information of the multimedia content. According to the embodiment of the invention, the sub-content feature information of the sub-contents in different modes in the multimedia content is fused to obtain the fused feature information capable of representing the features of the multimedia content in different modes, and then content risk identification is carried out on the multimedia content according to the fused feature information. The accuracy of identifying the content risk of the multimedia content can be improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a content risk identification method, apparatus, computer equipment and computer-readable storage medium, wherein the storage medium is a computer-readable storage medium. Background Art

[0002] With the development of Internet technology and the rise of self-media, the content on social platforms has become more colorful. At the same time, it is more important to review the content on social platforms. Reviewing the content on social platforms can avoid pushing risky content to users. The review of content is usually done manually, which is slow and obviously subjective. Different reviewers have different review standards, resulting in low accuracy in identifying content risks. Summary of the invention

[0003] Embodiments of the present application provide a content risk identification method, apparatus, computer device, and storage medium, which can improve the accuracy of identifying content risks of multimedia content.

[0004] A content risk identification method provided in an embodiment of the present application includes:

[0005] Acquire multimedia content, where the multimedia content includes at least two sub-contents in different modalities;

[0006] Extracting features of the sub-contents in the at least two modes respectively to obtain feature information of the sub-contents corresponding to the different modes;

[0007] Fusing the sub-content feature information corresponding to different modalities to obtain fused feature information of the multimedia content;

[0008] Content risk identification is performed on the multimedia content according to the fusion feature information to obtain risk information of the multimedia content.

[0009] Accordingly, an embodiment of the present application further provides a content risk identification device, including:

[0010] An acquisition unit, configured to acquire multimedia content, wherein the multimedia content includes at least two sub-contents in different modes;

[0011] An extraction unit, configured to extract features of the sub-contents in the at least two modes respectively, to obtain feature information of the sub-contents corresponding to the different modes;

[0012] A fusion unit, used for fusing sub-content feature information corresponding to different modalities to obtain fused feature information of the multimedia content;

[0013] The identification unit is used to perform content risk identification on the multimedia content according to the fused feature information to obtain risk information of the multimedia content.

[0014] In one embodiment, the at least two modalities include a text modality, the text modality corresponds to at least one text sub-content, and the extraction unit includes:

[0015] A word set acquisition subunit is used to acquire a sensitive word set;

[0016] A detection subunit, configured to perform sensitive word detection on each text sub-content in the text modality according to the sensitive word set, and determine the position of the sensitive word in each text sub-content;

[0017] A marking subunit, used to mark the sensitive word position of each text sub-content according to the position, and obtain the sensitive word position information corresponding to each text sub-content;

[0018] The feature extraction subunit is used to extract text features from each text sub-content based on the sensitive word position information to obtain sub-content feature information of the text modality.

[0019] In one embodiment, the feature extraction subunit includes:

[0020] A text feature extraction module, used to extract text features of each text sub-content based on the sensitive word position information, to obtain text feature information of each text sub-content;

[0021] A weight determination module, used to determine the content weight of each text sub-content according to the display position of each text sub-content in the multimedia content;

[0022] The weighted fusion module is used to perform weighted fusion processing on the text feature information of each text sub-content according to the content weight of each text sub-content, so as to obtain the sub-content feature information of the text modality.

[0023] In one embodiment, the sensitive word set includes sensitive word subsets corresponding to different risk types, and the marking subunit includes:

[0024] A sensitive word marking module, used to map non-sensitive content in the sub-text content to content feature elements and map sensitive words to sensitive word marking feature elements corresponding to the marking method according to the position and the marking method corresponding to the sensitive word subset, to obtain a position feature sequence corresponding to the sub-text content;

[0025] An information acquisition module is used to use the position feature sequence as the sensitive word position information.

[0026] In one embodiment, the at least two modalities include an image modality, the image modality corresponds to at least one image sub-content, and the extraction unit includes:

[0027] An image feature extraction subunit, configured to extract image features from the at least one image sub-content respectively to obtain image feature information of each image sub-content;

[0028] The fusion processing subunit is used to perform fusion processing on the image feature information of each image sub-content to obtain the sub-content feature information corresponding to the image modality.

[0029] In one embodiment, the at least two modalities include a text modality and an image modality, and the fusion unit includes:

[0030] A quantity acquisition subunit, used to acquire the number of text units contained in the sub-content of the text modality in the multimedia content, and to acquire the number of images of the sub-content of the image modality;

[0031] a weight setting subunit, configured to determine a first weight of the sub-content feature information of the text modality and a second weight of the sub-content feature information of the image modality according to a quantitative relationship between the number of the text units and the number of the images;

[0032] The modality fusion subunit is used to perform weighted fusion processing on the sub-content feature information corresponding to the different modalities based on the first weight and the second weight to obtain the fused feature information.

[0033] In one embodiment, there are multiple multimedia contents, the risk information includes risk type and risk level, and the content risk identification device further includes:

[0034] A classification unit, used for classifying each multimedia content according to its risk type, and obtaining multimedia content under each risk type;

[0035] The filtering unit is used to filter the multimedia content under each risk type according to the risk level to obtain the multimedia content to be reviewed for each risk type.

[0036] Correspondingly, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any content risk identification method provided in the embodiment of the present application.

[0037] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute any one of the content risk identification methods provided in the embodiments of the present application.

[0038] The embodiment of the present application obtains multimedia content, which includes sub-contents in at least two different modes; extracts features of the sub-contents in at least two modes respectively to obtain sub-content feature information corresponding to different modes; fuses the sub-content feature information corresponding to different modes to obtain fused feature information of the multimedia content; and identifies content risks of the multimedia content based on the fused feature information to obtain risk information of the multimedia content. The embodiment of the present application fuses the sub-content feature information of sub-contents in different modes in the multimedia content to obtain fused feature information that can characterize the features of the multimedia content in different modes, and then identifies content risks of the multimedia content based on the fused feature information, thereby improving the accuracy of identifying content risks of the multimedia content. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 is a flow chart of a content risk identification method provided by an embodiment of the present application;

[0041] Figure 2 It is a schematic diagram of the model structure provided in the embodiment of the present application;

[0042] Figure 3 is a schematic diagram of the network structure of the fusion network provided in the embodiment of the present application;

[0043] Figure 4 is a schematic diagram of a content risk identification device provided in an embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0046] The embodiments of the present application provide a content risk identification method, device, computer device and computer readable storage medium. The content risk identification device can be integrated in a computer device, which can be a server or a terminal.

[0047] The terminal may include a mobile phone, a wearable smart device, a tablet computer, a laptop computer, a personal computer (PC), and a vehicle-mounted computer.

[0048] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0049] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0050] This embodiment will be described from the perspective of a content risk identification device, which may be integrated into a computer device, which may be a server or a terminal.

[0051] The present application provides a method for identifying content risks. Figure 1 As shown, the specific process of the content risk identification method can be as follows:

[0052] 101. Acquire multimedia content, where the multimedia content includes at least two sub-contents in different modes.

[0053] The multimedia content may be content including multiple media forms, for example, it may include sub-contents in at least two different modes among text, image, audio and video.

[0054] For example, multimedia content may be obtained from a database to identify whether the multimedia content is illegal or in violation of regulations, and determine risk information of the multimedia content. For details of the specific implementation process, see the description of the relevant content of steps 102 to 104.

[0055] 102. Perform feature extraction on sub-contents in at least two modes respectively to obtain sub-content feature information corresponding to different modes.

[0056] For example, sub-contents of different modalities can be feature extracted through different neural networks. The sub-contents of the image modality can be feature extracted through the Resnet network to obtain the sub-content feature information corresponding to the image modality; the sub-contents of the text modality can be feature extracted through the Bert network to obtain the sub-content feature information corresponding to the text modality; the sub-contents of the audio modality can be feature extracted through the Transformer network to obtain the sub-content feature information corresponding to the audio modality.

[0057] Feature extraction can also be performed through other neural networks, which is not limited here.

[0058] In one embodiment, the multimedia content includes a sub-content belonging to a text mode, and the sub-content may be referred to as a text sub-content. The sensitive words in the text sub-content may be marked according to a sensitive word set, so that when extracting features from the text sub-content, more attention may be paid to the sensitive words in the text sub-content, thereby improving the recognition effect of risk information. That is, in the embodiment, at least two modes include a text mode, and the text mode corresponds to at least one text sub-content. The step of "extracting features from the sub-contents under at least two modes respectively to obtain sub-content feature information corresponding to different modes" may include:

[0059] Get the sensitive word set;

[0060] Perform sensitive word detection on each text sub-content in the text mode according to the sensitive word set, and determine the position of the sensitive word in each text sub-content;

[0061] Mark the sensitive word position of each text sub-content according to the position, and obtain the sensitive word position information corresponding to each text sub-content;

[0062] Based on the sensitive word location information, content features are extracted for each text sub-content to obtain sub-content feature information of the text modality.

[0063] The sensitive word set may be a pre-set word set, which includes sensitive words related to content risks.

[0064] The text mode may have at least one text sub-content, for example, it may include a title, a body, and text recognized from a video or an image by optical character recognition (OCR) technology.

[0065] Sensitive words are searched from each text sub-content according to the sensitive word set, the position of the sensitive words in the sensitive word set in each text sub-content is determined, and then the position of the sensitive words is marked to obtain the sensitive word position information corresponding to each text content.

[0066] In one embodiment, each text unit in the text sub-content corresponds to an identifier, which can indicate whether the text unit at that position is a keyword. For example, 0 indicates that the text unit at that position is not a keyword, and 1 indicates that the text unit at that position is a keyword. The sensitive word position information can be obtained based on the identifier corresponding to each text unit in the text content.

[0067] Assume that there is a text sub-content titled "Which is more bitter, bitter melon or American coffee?", in which bitter melon is a sensitive word in the sensitive word set. Then the sensitive word position information corresponding to the text sub-unit can be 11000000, where 1 indicates that the text unit at this position is a sensitive word, and 0 indicates that the sensitive word at this position is a non-sensitive word. The first text unit and the second text unit in the text sub-content are determined to be sensitive words based on the sensitive word position information.

[0068] Each text sub-content and the corresponding sensitive word position information can be input into a neural network for feature extraction to obtain text feature information of each text sub-content, and the text feature information of each text sub-content can be used as sub-content feature information of the text modality; optionally, the text feature information of each text sub-content can also be subjected to feature fusion processing to obtain sub-content feature information of the sub-content of the text modality.

[0069] In one embodiment, the Figure 2 The fusion network in the model structure shown in the figure fuses each text feature information to obtain the sub-content feature information of the text modality. The network structure of the fusion network can be entered into Figure 3 As shown, the feature fusion layer is used to fuse at least two text feature information to obtain initial sub-content features, and then the global pooling layer performs pooling on the initial sub-content features, performs linear transformation through the fully connected layer, maps the input information to a preset numerical range through the ReLU layer, processes it, performs linear transformation again through the fully connected layer, maps the input information to a preset numerical range through the Sigmod layer, and finally scales the size of the text feature information to an appropriate size for calculation through the scaling layer.

[0070] The feature fusion processing is performed on the text feature information of at least two text sub-contents of the text modality, and the feature fusion processing can be performed on at least two text feature information by means of concatenate or add; corresponding weights can be set for different text sub-contents. For example, the OCR text of the title and the cover image of the multimedia content is usually the first text sub-content that users look at, so compared with other multimedia content, it is more urgent to filter the multimedia content containing risky content in the title or cover image. The text sub-content weight can be assigned according to the display position of the text sub-content in the multimedia content. That is, in one embodiment, the step of "based on the sensitive word position information, extracting content features for each text sub-content to obtain sub-content feature information of the text modality" can specifically include:

[0071] Based on the sensitive word location information, text feature extraction is performed on each text sub-content to obtain text feature information of each text sub-content;

[0072] Determining the content weight of each text sub-content according to the display position of each text sub-content in the multimedia content;

[0073] According to the content weight of each text sub-content, the text feature information of each text sub-content is weighted and fused to obtain the sub-content feature information of the text modality.

[0074] For example, a correspondence between the display position and the content weight can be preset, and the content weight of each text sub-content can be determined according to the display position of each text sub-content in the multimedia content. For example, the content weight of the OCR text of the title and cover image can be set to 1, and the content weight of the text can be set to 0.8, etc.

[0075] Based on the sensitive word location information, text feature extraction is performed on each text sub-content to obtain text feature information of each text sub-content. Then, according to the content weight of each text sub-content, weighted fusion processing is performed on the text feature information of each text sub-content to obtain sub-content feature information of the text modality.

[0076] According to the sub-content feature information corresponding to different modes of multimedia content, the risk information of the multimedia content can be identified. The risk information may include the risk level of the multimedia content, and may also include the risk type of the multimedia content, such as the violence type and the pornography type, etc. Therefore, the sensitive word set may include sensitive word subsets corresponding to different risk types. Different sensitive word subsets may be marked in different ways. For example, different identifiers may be used to indicate that sensitive words belong to different sensitive word subsets. That is, in one embodiment, the sensitive word set includes sensitive word subsets corresponding to different risk types. The step of "marking the sensitive word position of each text sub-content according to the position to obtain the sensitive word position information corresponding to each text sub-content" may include:

[0077] According to the marking method corresponding to the position and sensitive word subset, the non-sensitive content in the sub-text content is mapped to the content feature element, and the sensitive words are mapped to the sensitive word marking feature element corresponding to the marking method, so as to obtain the position feature sequence corresponding to the sub-text content;

[0078] The position feature sequence is used as the sensitive word position information.

[0079] For example, different sensitive word marking feature elements can be used to mark sensitive words belonging to different sensitive word subsets in the text sub-content. Assuming that the sensitive word set contains three sensitive word subsets, the sensitive words in the sensitive word set in the text sub-content can be marked with 1, 2, and 3 respectively.

[0080] According to the marking method corresponding to the position and sensitive word subset, the text units in each text sub-content that are not sensitive words are mapped to content feature elements, and the text units that are sensitive words are mapped to marking feature elements. Using different methods to mark different sensitive word subsets can make the neural network pay more attention to the sensitive words in the text sub-content and the risk type to which the sensitive words belong, which can improve the accuracy and speed of determining the risk information of multimedia content.

[0081] Exemplarily, a text sub-content is "Which is more bitter, bitter melon or American style", where bitter melon is a sensitive word in sensitive word subset A, and American style is a sensitive word in sensitive word subset B. The sensitive words in sensitive word subset A are marked with 1, and those in sensitive word subset B are marked with 2. Then the sensitive word position information corresponding to the text sub-unit can be 11022000, where 0 indicates that the sensitive word at this position is a non-sensitive word, 1 indicates that the text unit at this position is a sensitive word in sensitive word subset A, and 2 indicates that the text unit at this position is a sensitive word in sensitive word subset B. According to the sensitive word position information, it is determined that the sensitive words of the first text unit and the second text unit in the text sub-content belong to sensitive word set A, and the sensitive words of the fourth text unit and the fifth text unit belong to sensitive word set B.

[0082] The multimedia content may include sub-content of text modality and may also include sub-content of image modality. The sub-content of image modality may be referred to as image sub-content. By performing feature extraction on the image sub-content, image feature information of each image sub-content may be used as sub-content feature information corresponding to the image modality. Alternatively, the image feature information of each image sub-content may be fused to obtain sub-content feature information. That is, in one embodiment, at least two modalities include an image modality, and the image modality corresponds to at least one image sub-content. The step of “perform feature extraction on sub-contents under at least two modalities respectively to obtain sub-content feature information corresponding to different modalities” may include:

[0083] Extracting image features from at least one image sub-content respectively to obtain image feature information of each image sub-content;

[0084] The image feature information of each image sub-content is fused to obtain the sub-content feature information corresponding to the image modality.

[0085] Among them, an image sub-content can be an image, a complete video, or multiple video frames extracted from a video, for example, it can include a cover image of multimedia content, a cover frame and first frame of a video, etc.

[0086] For example, image features may be extracted from at least one image sub-content under the image modality to obtain image feature information of each image sub-content, and the image feature information of each image sub-content may be fused to obtain sub-content feature information corresponding to the image modality.

[0087] Optionally, weights can be set for different image sub-contents to perform weighted fusion processing on the image feature information of different image sub-contents to obtain the sub-content feature information corresponding to the image modality. For example, the content weights can be set according to the display order of the image sub-contents. The content weights of the cover image and the cover frame or the first frame of the video can be set to 1, and the content weights of other image sub-contents can be set to 0.7, etc.

[0088] In one embodiment, the Figure 2 The fusion network in the model structure shown performs fusion processing on each image feature information to obtain sub-content feature information of the image modality.

[0089] 103. Fusing the sub-content feature information corresponding to different modalities to obtain fused feature information of the multimedia content.

[0090] For example, the sub-content feature information corresponding to each modality may be fused to obtain fused feature information of the multimedia content.

[0091] In one embodiment, the Figure 2 The fusion network in the model structure shown fuses the sub-content feature information corresponding to each modality to obtain fused feature information of the multimedia content.

[0092] In one embodiment, different weights may be set for sub-contents of different modalities to perform weighted fusion processing, that is, at least two modalities include a text modality and an image modality, and the step of "fusion processing of sub-content feature information corresponding to different modalities to obtain fusion feature information of multimedia content" may include:

[0093] Obtain the number of text units contained in the sub-content of the text mode in the multimedia content, and obtain the number of images contained in the sub-content of the image mode;

[0094] Determining a first weight of sub-content feature information in a text modality and a second weight of sub-content feature information in an image modality according to a quantitative relationship between the number of text units and the number of images;

[0095] Based on the first weight and the second weight, weighted fusion processing is performed on the sub-content feature information corresponding to different modalities to obtain fused feature information.

[0096] For example, the number of text units contained in the text sub-content in the multimedia content is obtained, and the number of images in the image sub-content is obtained; the ratio between the number of text units and the number of images is calculated, and the ratio is equivalent to the number of text units corresponding to one image.

[0097] If the ratio is greater than the first preset threshold and less than the second preset threshold, it means that the amount of information contained in the image sub-content is equivalent to the amount of information contained in the text sub-content. In this case, the first weight of the sub-content feature information of the text modality and the second weight of the sub-content feature information of the image modality can be set equal. For example, the first weight and the second weight are both 1.

[0098] If the ratio is greater than the second preset threshold, indicating that the amount of information contained in the image sub-content is less than the amount of information contained in the text sub-content, the first weight of the sub-content feature information in the text modality can be set greater than the second weight of the sub-content feature information in the image modality.

[0099] If the ratio is less than the first preset threshold, indicating that the image sub-content contains more information than the text sub-content, the first weight of the sub-content feature information in the text mode may be set smaller than the second weight of the sub-content feature information in the image mode.

[0100] Based on the first weight and the second weight, weighted fusion processing is performed on the sub-content feature information corresponding to different modalities to obtain fused feature information.

[0101] 104. Perform content risk identification on the multimedia content according to the fused feature information to obtain risk information of the multimedia content.

[0102] For example, content risk identification is performed on multimedia content based on fused feature information to obtain risk information of the multimedia content.

[0103] Among them, the risk information may include a risk score or risk level of the multimedia content. After determining the risk score or risk level of the multimedia content, the multimedia content may be filtered according to the risk score or risk level to avoid recommending risky multimedia content to users.

[0104] Optionally, multimedia content that requires manual review can also be determined based on the risk score or risk level. For example, multimedia content with a higher risk score or risk level can be directly filtered, and multimedia content with a low risk score or risk level can be directly recommended to users, reducing the amount of content that requires manual review. For multimedia content with a medium risk score or risk level, manual review can be performed to determine whether it can be recommended to users.

[0105] Optionally, the risk information may further include the risk type of the multimedia content, which may be sorted according to the risk type and the risk level to determine the multimedia content to be filtered. That is, in one embodiment, there are multiple multimedia contents, and the risk information includes the risk type and the risk level. After the step of "identifying the content risk of the multimedia content according to the fusion feature information to obtain the risk information of the multimedia content", the content risk identification method provided in the embodiment of the present application further includes:

[0106] Classify and process each multimedia content according to its risk type, and obtain multimedia content under each risk type;

[0107] The multimedia content under each risk type is filtered according to the risk level to obtain the multimedia content to be reviewed.

[0108] For example, each multimedia content can be classified according to its risk type to obtain the multimedia content of each risk type; the multimedia content of each risk type can be filtered according to the risk level to obtain the multimedia content to be reviewed for each risk type; the multimedia content to be reviewed for each risk type can be pushed to different reviewers for review, thereby improving the review efficiency.

[0109] As can be seen from the above, the embodiments of the present application obtain multimedia content, which includes sub-contents of at least two different modalities; extract features of the sub-contents under at least two modalities respectively to obtain sub-content feature information corresponding to different modalities; fuse the sub-content feature information corresponding to different modalities to obtain fused feature information of the multimedia content; and identify content risks of the multimedia content based on the fused feature information to obtain risk information of the multimedia content. The embodiments of the present application fuse the sub-content feature information of sub-contents of different modalities in the multimedia content to obtain fused feature information that can characterize the features of the multimedia content in different modalities, and then identify content risks of the multimedia content based on the fused feature information, thereby improving the accuracy of identifying content risks of the multimedia content.

[0110] In order to better implement the content risk identification method provided in the embodiment of the present application, a content risk identification device is also provided in one embodiment. The meanings of the terms are the same as those in the above content risk identification method, and the specific implementation details can refer to the description in the method embodiment.

[0111] The content risk identification device can be integrated into a computer device, such as Figure 4 As shown, the content risk identification device may include: an acquisition unit 301, an extraction unit 302, a fusion unit 303 and an identification unit 304, which are specifically as follows:

[0112] (1) Acquisition unit 301: used to acquire multimedia content, where the multimedia content includes at least two sub-contents in different modes.

[0113] (2) Extraction unit 302: used to extract features of sub-contents in at least two modes respectively, and obtain sub-content feature information corresponding to different modes.

[0114] In one embodiment, the at least two modalities include a text modality, and the text modality corresponds to at least one text sub-content. The extraction unit 302 includes:

[0115] Word set acquisition subunit: used to acquire sensitive word sets;

[0116] Detection subunit: used to perform sensitive word detection on each text sub-content in the text mode according to the sensitive word set, and determine the position of the sensitive word in each text sub-content;

[0117] Marking subunit: used to mark the sensitive word position of each text sub-content according to the position, and obtain the sensitive word position information corresponding to each text sub-content;

[0118] Feature extraction subunit: used to extract content features of each text sub-content based on the sensitive word position information, and obtain sub-content feature information of the text modality.

[0119] In one embodiment, the feature extraction subunit includes:

[0120] Text feature extraction module: used to extract text features of each text sub-content based on the sensitive word location information, and obtain text feature information of each text sub-content;

[0121] A weight determination module: used to determine the content weight of each text sub-content according to the display position of each text sub-content in the multimedia content;

[0122] Weighted fusion module: used to perform weighted fusion processing on the text feature information of each text sub-content according to the content weight of each text sub-content, so as to obtain the sub-content feature information of the text modality.

[0123] In one embodiment, the sensitive word set includes sensitive word subsets corresponding to different risk types, and the marking subunit includes:

[0124] Sensitive word marking module: used to map non-sensitive content in the sub-text content into content feature elements and sensitive words into sensitive word marking feature elements corresponding to the marking method according to the marking method corresponding to the position and sensitive word subset, and obtain the position feature sequence corresponding to the sub-text content;

[0125] Information acquisition module: used to use the position feature sequence as the sensitive word position information.

[0126] In one embodiment, the at least two modalities include an image modality, and the image modality corresponds to at least one image sub-content. The extraction unit 302 includes:

[0127] An image feature extraction subunit: used to extract image features from at least one image sub-content respectively to obtain image feature information of each image sub-content;

[0128] Fusion processing subunit: used to perform fusion processing on the image feature information of each image sub-content to obtain the sub-content feature information corresponding to the image modality.

[0129] (3) Fusion unit 303: used to fuse the sub-content feature information corresponding to different modalities to obtain fused feature information of the multimedia content.

[0130] In one embodiment, the at least two modalities include a text modality and an image modality, and the fusion unit 303 includes:

[0131] A quantity acquisition subunit: used to acquire the number of text units contained in the sub-content of the text mode in the multimedia content, and to acquire the number of images of the sub-content of the image mode;

[0132] A weight setting subunit: used to determine a first weight of sub-content feature information in text mode and a second weight of sub-content feature information in image mode according to a quantitative relationship between the number of text units and the number of images;

[0133] Modality fusion subunit: used for performing weighted fusion processing on sub-content feature information corresponding to different modalities based on the first weight and the second weight to obtain fused feature information.

[0134] (4) Identification unit 304: used to identify the content risk of the multimedia content according to the fused feature information to obtain the risk information of the multimedia content.

[0135] In one embodiment, there are multiple multimedia contents, the risk information includes risk type and risk level, and the content risk identification device further includes:

[0136] Classification unit: used for classifying each multimedia content according to its risk type, and obtaining multimedia content under each risk type;

[0137] Filtering unit: used to filter the multimedia content under each risk type according to the risk level, and obtain the multimedia content to be reviewed for each risk type.

[0138] As can be seen from the above, the content risk identification device of the embodiment of the present application obtains multimedia content through the acquisition unit 301, and the multimedia content includes at least two sub-contents of different modes; the extraction unit 302 performs feature extraction on the sub-contents under at least two modes respectively, and obtains the sub-content feature information corresponding to the different modes; the fusion unit 303 fuses the sub-content feature information corresponding to the different modes to obtain the fusion feature information of the multimedia content; the identification unit 304 performs content risk identification on the multimedia content according to the fusion feature information to obtain the risk information of the multimedia content. The embodiment of the present application fuses the sub-content feature information of the sub-contents of different modes in the multimedia content to obtain the fusion feature information that can characterize the features of the multimedia content in different modes, and then performs content risk identification on the multimedia content according to the fusion feature information, which can improve the accuracy of identifying the content risk of the multimedia content.

[0139] The present application also provides a computer device, which may be a terminal or a server. Figure 5 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0140] The computer device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will appreciate that Figure 5 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0141] The processor 1001 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 1002 and calling data stored in the memory 1002, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and computer programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1001.

[0142] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0143] The computer device also includes a power supply 1003 for supplying power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1003 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0144] The computer device may further include an input unit 1004, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0145] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically, in this embodiment, the processor 1001 in the computer device will load the executable files corresponding to the processes of one or more computer programs into the memory 1002 according to the following instructions, and the processor 1001 will run the computer programs stored in the memory 1002, thereby realizing various functions, as follows:

[0146] Acquire multimedia content, where the multimedia content includes at least two sub-contents in different modes;

[0147] Extracting features of sub-contents in at least two modes respectively to obtain feature information of sub-contents corresponding to different modes;

[0148] The sub-content feature information corresponding to different modalities is fused to obtain fused feature information of the multimedia content;

[0149] The content risk of multimedia content is identified according to the fused feature information to obtain the risk information of the multimedia content.

[0150] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0151] As can be seen from the above, the computer device of the embodiment of the present application can obtain multimedia content, the multimedia content includes at least two sub-contents of different modes; extract features of the sub-contents under at least two modes respectively, and obtain sub-content feature information corresponding to different modes; fuse the sub-content feature information corresponding to different modes to obtain fused feature information of the multimedia content; identify content risks of the multimedia content based on the fused feature information to obtain risk information of the multimedia content. The embodiment of the present application fuses the sub-content feature information of sub-contents of different modes in the multimedia content to obtain fused feature information that can characterize the features of the multimedia content in different modes, and then identifies content risks of the multimedia content based on the fused feature information, so as to improve the accuracy of identifying content risks of the multimedia content.

[0152] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in various optional implementations in the above-mentioned embodiments.

[0153] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0154] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any one of the invention name methods provided in the embodiments of the present application.

[0155] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0156] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0157] Since the computer program stored in the computer-readable storage medium can execute any one of the invention name methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the invention name methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0158] The above is a detailed introduction to an invention method, apparatus, computer device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A content risk identification method, characterized in that: include: Acquire multimedia content, where the multimedia content includes at least two sub-contents in different modalities; Extracting features of the sub-contents in the at least two modes respectively to obtain feature information of the sub-contents corresponding to the different modes; Fusing the sub-content feature information corresponding to different modalities to obtain fused feature information of the multimedia content; Content risk identification is performed on the multimedia content according to the fusion feature information to obtain risk information of the multimedia content.

2. The method according to claim 1, characterized in that The at least two modes include a text mode, and the text mode corresponds to at least one text sub-content. The feature extraction of the sub-contents under the at least two modes is performed to obtain sub-content feature information corresponding to different modes, including: Get the sensitive word set; Performing sensitive word detection on each text sub-content in the text mode according to the sensitive word set, and determining the position of the sensitive word in each text sub-content; Marking the sensitive word position of each text sub-content according to the position to obtain sensitive word position information corresponding to each text sub-content; Based on the sensitive word position information, content feature extraction is performed on each text sub-content to obtain sub-content feature information of the text modality.

3. The method according to claim 2, characterized in that The extracting content features of each text sub-content based on the sensitive word position information to obtain sub-content feature information of the text modality includes: Based on the sensitive word position information, extract text features from each text sub-content to obtain text feature information of each text sub-content; Determining the content weight of each text sub-content according to the display position of each text sub-content in the multimedia content; According to the content weight of each text sub-content, the text feature information of each text sub-content is weighted and fused to obtain the sub-content feature information of the text modality.

4. The method according to claim 2, characterized in that: The sensitive word set includes sensitive word subsets corresponding to different risk types, and the sensitive word position information corresponding to each text sub-content is obtained by marking the sensitive word position of each text sub-content according to the position, including: According to the marking method corresponding to the position and the sensitive word subset, the non-sensitive content in the sub-text content is mapped to content feature elements, and the sensitive words are mapped to sensitive word marking feature elements corresponding to the marking method, so as to obtain a position feature sequence corresponding to the sub-text content; and the position feature sequence is used as the sensitive word position information.

5. The method according to claim 1, characterized in that The at least two modalities include an image modality, and the image modality corresponds to at least one image sub-content. The feature extraction of the sub-contents under the at least two modalities is performed to obtain sub-content feature information corresponding to different modalities, including: Extracting image features from the at least one image sub-content respectively to obtain image feature information of each image sub-content; The image feature information of each image sub-content is fused to obtain the sub-content feature information corresponding to the image modality.

6. The method according to claim 1, characterized in that The at least two modalities include a text modality and an image modality, and the fusing of sub-content feature information corresponding to different modalities to obtain fused feature information of the multimedia content includes: Obtaining the number of text units contained in the sub-content of the text modality in the multimedia content, and obtaining the number of images contained in the sub-content of the image modality; Determining a first weight of the sub-content feature information of the text modality and a second weight of the sub-content feature information of the image modality according to a quantitative relationship between the number of the text units and the number of the images; Based on the first weight and the second weight, weighted fusion processing is performed on the sub-content feature information corresponding to the different modalities to obtain the fused feature information.

7. The method according to any one of claims 1 to 6, characterized in that: There are multiple multimedia contents, the risk information includes risk type and risk level, and after content risk identification is performed on the multimedia contents according to the fusion feature information and the risk information of the multimedia contents is obtained, the method further includes: Classify and process each multimedia content according to its risk type, and obtain multimedia content under each risk type; The multimedia content under each risk type is filtered according to the risk level to obtain the multimedia content to be reviewed for each risk type.

8. A content risk identification device, characterized in that: include: An acquisition unit, configured to acquire multimedia content, wherein the multimedia content includes at least two sub-contents in different modes; An extraction unit, configured to extract features of the sub-contents in the at least two modes respectively, to obtain feature information of the sub-contents corresponding to the different modes; A fusion unit, used for fusing sub-content feature information corresponding to different modalities to obtain fused feature information of the multimedia content; The identification unit is used to perform content risk identification on the multimedia content according to the fused feature information to obtain risk information of the multimedia content.

9. A computer device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the content risk identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the content risk identification method according to any one of claims 1 to 7.

Images