Content detection method and system

Abstract

The content detection method and system provided in the specification, after at least one content image of the target content is acquired, the at least one content image is input to a content detection model to obtain a fake content probability of the target content and at least one fake content feature corresponding to the fake content probability, training samples of the content detection model include a plurality of fake content samples generated based on diffusion of reference content by a plurality of expert model groups, the plurality of expert model groups include a plurality of complementary image-text matching models, and based on the fake content probability and the at least one fake content feature, a risk detection result of the target content is determined, and the risk detection result is output; the scheme can improve the detection performance of content detection.

Description

Technical Field

[0001] This specification relates to the field of image recognition, and in particular to a content detection method and system. Background Technology

[0002] In recent years, with the rapid development of internet technology, content generation has become increasingly convenient. For example, artificial intelligence (AI) technology can be used to generate images or videos with specific artistic styles based on input text. However, this type of AI-generated image or video content cannot be copyrighted. Therefore, content detection is necessary for content security. Existing content detection methods often rely on methods based on original images or camera watermarks.

[0003] In the process of researching and practicing existing technologies, the inventors of this application found that the forgery detection method based on original images requires a large amount of labeled data for model training. When the number of forgery samples is limited, this greatly restricts the detection capability of forgery detection. Furthermore, the camera watermarking method requires the introduction of additional watermarks during the acquisition stage, resulting in low content detection coverage and making it difficult to apply to all scenarios. Therefore, the detection performance of content detection is low. Summary of the Invention

[0004] This specification provides a content detection method and system with higher detection performance.

[0005] In a first aspect, this specification provides a content detection method, comprising: acquiring at least one content image of target content; inputting the at least one content image into a content detection model to obtain a forged content probability of the target content and at least one forged content feature corresponding to the forged content probability, wherein the training samples of the content detection model include multiple forged content samples generated after diffusion of reference content based on a multi-expert model group, the multi-expert model group including multiple complementary image-text matching models; and determining a risk detection result of the target content based on the forged content probability and the at least one forged content feature, and outputting the risk detection result.

[0006] In some embodiments, the target content includes at least one of image content or video content.

[0007] In some embodiments, the training process of the multi-expert model group includes the following steps: obtaining content sample pairs and inputting the content sample pairs into a preset multi-model group to obtain the feature set and feature similarity set corresponding to the content sample pairs; and based on the feature set and the feature similarity set, converging the preset multi-model group to obtain the trained multi-expert model group.

[0008] In some embodiments, the content sample pair includes image samples and text samples, the preset multi-model group includes multiple preset image-text matching models and a feature transfer network; and the step of inputting the content sample pair into the preset multi-model group to obtain the feature set and feature similarity set corresponding to the content sample pair includes: inputting the content sample pair into the multiple preset image-text matching models respectively to obtain a sample feature set and a feature similarity set, the sample feature set including the sample image features of the image sample and the sample text features of the text sample output by each preset image-text matching model in the multiple preset image-text matching models, the feature similarity set including the feature similarity between the sample image features and the sample text features output by each preset image-text matching model, inputting each sample feature in the sample feature set into the feature transfer network to obtain a transferred feature set, the transferred feature set including transferring the sample features corresponding to any one preset image-text matching model in the multiple preset image-text matching models to the sample features corresponding to other preset image-text matching models in the multiple preset image-text matching models, and using the transferred feature set and the sample feature set as the feature set.

[0009] In some embodiments, the step of converging the preset multi-model group to obtain the trained multi-expert model group includes: obtaining the annotation matching results of the content sample pairs, and determining feature matching loss information based on the annotation matching results, the feature similarity set, and the sample feature set; determining feature complementarity loss information among the multiple preset text-image matching models based on the sample feature set; determining feature transfer loss information based on the sample feature set and the transfer feature set; and converging the preset multi-model group based on the feature matching loss information, the feature complementarity loss information, and the feature transfer loss information to obtain the multi-expert model group.

[0010] In some embodiments, determining feature matching loss information based on the annotation matching result, the feature similarity set, and the sample feature set includes: determining a first preset similarity threshold corresponding to the content sample pair based on the annotation matching result; selecting the target feature similarity corresponding to each preset image-text matching model in the feature similarity set, and comparing the target feature similarity with the first preset similarity threshold to obtain first feature matching loss information corresponding to each preset image-text matching model; determining second feature matching loss information corresponding to the preset multi-model group based on the sample feature set and the first preset similarity threshold; and using the first feature matching loss information and the second feature matching loss information as the feature matching loss information.

[0011] In some embodiments, determining the second feature matching loss information corresponding to the preset multi-model group based on the sample feature set and the first preset similarity threshold includes: selecting the sample image features from the sample feature set to obtain a sample image feature set, and selecting the sample text features from the sample feature set to obtain a sample text feature set; accumulating the sample image features in the sample image feature set to obtain target sample image features, and accumulating the sample text features in the sample text feature set to obtain target sample text features; and obtaining the current feature similarity between the target sample image features and the target sample text features, and comparing the current feature similarity with the first preset similarity threshold to obtain the second feature matching loss information.

[0012] In some embodiments, determining the feature complementarity loss information among the plurality of preset image-text matching models based on the sample feature set includes: selecting sample features of the same type output by different image-text matching models from the sample feature set to obtain at least one sample feature pair; obtaining the feature similarity between features in each sample feature pair in the at least one sample feature pair to obtain a complementary feature similarity set; and comparing the feature similarity in the complementary feature similarity set with a second preset similarity threshold to obtain the feature complementarity loss information, wherein the constraint condition of the feature complementarity loss information is that the feature similarity between sample features of the same type output by different image-text matching models is less than the second preset similarity threshold.

[0013] In some embodiments, determining feature transfer loss information based on the sample feature set and the transfer feature set includes: selecting candidate sample features corresponding to each transfer feature in the transfer feature set from the sample feature set; obtaining the feature similarity between the candidate sample features and the corresponding transfer features to obtain a transfer feature similarity set; and comparing the feature similarity in the transfer feature similarity set with a third preset similarity threshold to obtain the feature transfer loss information, wherein the constraint condition of the feature transfer loss information is that the feature similarity between sample features of different preset image-text matching models during transfer is less than the third preset similarity threshold.

[0014] In some embodiments, the training process of the content detection model includes the following steps: obtaining content image sample pairs corresponding to target content samples in the training samples; inputting the content image sample pairs into a preset content detection model to obtain sample forged content features and predicted content categories; and converging the preset content detection model based on the sample forged content features and the predicted content categories to obtain the trained content detection model.

[0015] In some embodiments, inputting the content image sample pair into a preset content detection model to obtain sample forged content features and predicted content category includes: linearly superimposing the content image samples in the content image sample pair to obtain a mixed content image sample; extracting features from the content image sample pair and the mixed content image sample respectively to obtain a candidate sample image feature set; and extracting the sample forged content features from the candidate sample image feature set, and determining the predicted content category of the target content sample based on the sample forged content features.

[0016] In some embodiments, extracting the sample forged content features from the candidate sample image feature set includes: linearly superimposing the sample image features in the candidate sample image feature set to obtain mixed sample image features; extracting the initial forged content features corresponding to each sample image feature from the candidate sample image feature set and the mixed sample image features respectively; and using the initial forged content features as the sample forged content features corresponding to the target content sample.

[0017] In some embodiments, the step of converging the preset content detection model to obtain the trained content detection model includes: obtaining the labeled content category of the target content sample; comparing the labeled content category with the predicted content category to obtain content classification loss information; determining the feature prediction loss information of the target content sample based on the labeled content category and the sample forged content features; and fusing the feature prediction loss information and the content classification loss information, and converging the preset content detection model based on the fused target content loss information to obtain the content detection model.

[0018] In some embodiments, the labeled content category includes normal content samples and fake content samples; and determining the feature prediction loss information of the target content sample based on the labeled content category and the fake content features of the sample includes: determining a preset feature category corresponding to the target content sample based on the labeled content category, determining a current feature category corresponding to the fake content features of the sample, and comparing the preset feature category with the current feature category to obtain the feature prediction loss information, wherein the constraint condition of the feature prediction loss information is that the fake content features of the sample corresponding to the normal content sample are all zero features.

[0019] In some embodiments, before obtaining the content image sample pair corresponding to the target content sample in the training samples, the method further includes: obtaining the reference content; inputting the reference content into the multi-expert model group and the diffusion model respectively to obtain multiple fake content samples; and obtaining multiple normal content samples, and using the multiple normal content samples and the multiple fake content samples as the training samples.

[0020] In some embodiments, the reference content includes a text set and a reference image; and the step of inputting the reference content into the multi-expert model group and the diffusion model respectively to obtain multiple fake content samples includes: inputting the reference image into the diffusion model to obtain an initial diffusion content image; inputting the initial diffusion content image and the target text in the text set into the multi-expert model group to obtain noise increment images corresponding to multiple diffusion directions of the initial diffusion content image; and inputting the noise increment images into the diffusion model to obtain the multiple fake content samples.

[0021] In some embodiments, the training process of the diffusion model includes the following steps: obtaining reference image samples and keyword samples corresponding to the reference image samples, and inputting the reference image samples into a preset diffusion model to obtain initial diffusion content image samples; inputting the keyword samples and the initial diffusion content image samples into the multi-expert model group to obtain image-text similarity and multiple sample diffusion directions corresponding to the initial diffusion content image samples, wherein the image-text similarity includes the similarity between the keyword samples and the initial diffusion content samples; inputting the noise set corresponding to the multiple sample diffusion directions into the preset diffusion model to obtain the current diffusion content image sample set corresponding to the multiple sample diffusion directions; and converging the preset diffusion model based on the image-text similarity, the noise set, and the current diffusion content image sample set to obtain the trained diffusion model.

[0022] In some embodiments, the step of converging the preset diffusion model to obtain the trained diffusion model includes: inputting the current diffusion content image set into a preset feature extraction model to obtain the evaluation features of each diffusion content image sample in the current diffusion content image sample set; determining target diffusion loss information based on the image-text similarity, the noise set, the current diffusion content image sample set, and the evaluation features; and converging the preset diffusion model based on the target diffusion loss information to obtain the diffusion model.

[0023] In some embodiments, determining the target diffusion loss information includes: acquiring the labeled image-text similarity between the reference image sample and the keyword sample, and comparing the labeled image-text similarity with the image-text similarity to obtain image-text similarity loss information; determining the sample evaluation loss information between diffusion content image samples in the current diffusion content image sample set based on the evaluation features, wherein the constraint condition for the sample evaluation loss information is that the feature difference between the evaluation features of different diffusion content image samples is less than a preset difference threshold; determining the diffusion distance loss information between different sample diffusion directions in the plurality of sample diffusion directions based on the noise set, wherein the constraint condition for the diffusion distance loss information is that the diffusion distance between different sample diffusion directions exceeds a preset distance threshold; and fusing the image-text similarity loss information, the sample evaluation loss information, and the diffusion distance loss information to obtain the target diffusion loss information.

[0024] In some embodiments, the evaluation features include style features and content features; and determining the sample evaluation loss information between diffused content image samples in the current diffused content image sample set based on the evaluation features includes: obtaining the feature similarity of style features and the feature similarity of content features between the diffused content image samples to obtain a style feature similarity set and a content feature similarity set; comparing the feature similarity in the style feature similarity set with a preset style feature similarity threshold to obtain style loss information; comparing the feature similarity in the content feature similarity set with a preset content feature similarity threshold to obtain content loss information; and fusing the style loss information and the content loss information to obtain the sample evaluation loss information.

[0025] In some embodiments, determining the diffusion distance loss information between different diffusion directions of the plurality of samples includes: extracting features from the noise in the noise set to obtain noise features corresponding to each diffusion direction of the plurality of samples; obtaining the feature distance between the noise features to obtain the diffusion distance between the plurality of samples; and comparing the diffusion distance with a preset distance threshold to obtain the diffusion distance loss information.

[0026] In some embodiments, the risk detection result includes either fake content or normal content, wherein the fake content is text-generated image content or video content with an artistic style; and determining the risk detection result of the target content includes: obtaining the feature mean of the at least one fake content feature, and determining the risk detection result of the target content as fake content when the probability of fake content is greater than a preset probability threshold, or when the feature mean is greater than a preset feature mean threshold.

[0027] In some embodiments, the method further includes: when the forged content is less than the preset probability threshold and the mean value of the features is less than the preset mean value of the features, determining the risk detection result of the target content as normal content.

[0028] Secondly, this specification also provides a content detection system, comprising: at least one storage medium storing at least one instruction set for performing content detection; and at least one processor communicatively connected to the at least one storage medium, wherein, when the content detection system is running, the at least one processor reads the at least one instruction set and executes the content detection method described in the first aspect of this specification according to the instructions of the at least one instruction set.

[0029] As can be seen from the above technical solutions, the content detection method and system provided in this specification, after acquiring at least one content image of the target content, inputs at least one content image into the content detection model to obtain the forgery probability of the target content and at least one forgery feature corresponding to the forgery probability. The training samples of the content detection model include multiple forgery samples generated after the reference content is diffused based on a multi-expert model group. The multi-expert model group includes multiple complementary image-text matching models, and the risk detection result of the target content is determined based on the forgery probability and at least one forgery feature, and the risk detection result is output. Since this solution guides the diffusion of the reference content through a multi-expert model group, and the multi-expert model includes multiple complementary image-text matching models, the randomness of the generated data is increased. In addition, no other additional factors need to be introduced, thereby improving the diversity and coverage of the generated forgery samples, and thus obtaining a content detection model with high coverage and high performance. Therefore, the detection performance of content detection can be improved.

[0030] Other functions of the content detection methods and systems provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the content detection methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A schematic diagram illustrating an application scenario of a content detection system provided according to an embodiment of this specification is shown.

[0033] Figure 2 A hardware structure diagram of a computing device provided according to an embodiment of this specification is shown;

[0034] Figure 3 A flowchart of a content detection method provided according to an embodiment of this specification is shown;

[0035] Figure 4 A schematic diagram of the overall process of content detection according to embodiments of this specification is shown; and

[0036] Figure 5A schematic diagram illustrating the detection of forged content generated by the DiscoDiffusion method according to an embodiment of this specification is shown. Detailed Implementation

[0037] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0038] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0039] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0040] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0041] For ease of description, the terms that will appear in the following descriptions will be explained as follows:

[0042] Multi-expert model: refers to a method that uses multiple independent and complementary models for decision-making and reasoning. For example, multiple expert models can be trained to perform content security checks at the same time, and the results can be fused together as the final result.

[0043] Disco Diffusion is an AI-powered method for generating artistic content based on user-input text. This method uses a large-scale pre-trained model CLIP to guide the optimization of the generation model, and the generated content is mostly artistic creation that is difficult for the human eye to distinguish.

[0044] Content security: Authenticity of content is detected using artificial intelligence algorithms, for example, distinguishing between images generated by Disco Diffusion and images created by artists.

[0045] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0046] The content detection methods provided in this manual can be applied to any content detection scenario. For example, in content review scenarios, the content detection methods in this manual can be used to detect content awaiting review on a content platform; in content publishing scenarios, the content detection methods in this manual can be used to detect content to be published; in content sharing scenarios, the content detection methods in this manual can be used to detect content to be shared; and they can also be applied to any other content detection scenario, which will not be elaborated on here.

[0047] Those skilled in the art should understand that the content detection methods and systems described in this specification are also within the scope of protection of this specification when applied to other use scenarios.

[0048] Figure 1 This diagram illustrates an application scenario of a content detection system 001 provided according to an embodiment of this specification. The content detection system 001 (hereinafter referred to as System 001) can be applied to content detection in any scenario, such as content detection in content review scenarios, content detection in content publishing scenarios, content detection in content sharing scenarios, etc. Figure 1 As shown, system 001 may include target user 100, client 200, server 300 and network 400 within the target space.

[0049] Target user 100 can be the user who triggers the identification of target content. Target user 100 can perform content detection operations on client 200.

[0050] Client 200 can be a device for performing content detection on target content in response to a content detection operation by target user 100. In some embodiments, the content detection method can be executed on client 200. In this case, client 200 may store data or instructions for executing the content detection method described herein, and may execute or be used to execute said data or instructions. In some embodiments, client 200 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to operate. Figure 1 As shown, client 200 can communicate with server 300. In some embodiments, server 300 can communicate with multiple clients 200. In some embodiments, client 200 can interact with server 300 through network 400 to receive or send messages, such as receiving or sending target content or content images. In some embodiments, client 200 may include mobile devices, tablets, laptops, built-in devices in motor vehicles, or similar content, or any combination thereof. In some embodiments, the mobile device may include smart home devices, smart mobile devices, virtual reality devices, augmented reality devices, or similar devices, or any combination thereof. In some embodiments, the smart home device may include smart TVs, desktop computers, etc., or any combination thereof. In some embodiments, the smart mobile device may include smartphones, personal digital assistants, gaming devices, navigation devices, etc., or any combination thereof. In some embodiments, the virtual reality device or augmented reality device may include virtual reality headsets, virtual reality glasses, virtual reality controllers, augmented reality headsets, augmented reality glasses, augmented reality controllers, or similar content, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, head-mounted displays, VR, etc. In some embodiments, the built-in device in the motor vehicle may include an in-vehicle computer, an in-vehicle TV, etc. In some embodiments, the client 200 may include an image acquisition device for acquiring an image of the target content, thereby obtaining at least one content image. In some embodiments, the image acquisition device may be a two-dimensional image acquisition device (such as an RGB camera), or a combination of a two-dimensional image acquisition device (such as an RGB camera) and a depth image acquisition device (such as a 3D structured light camera, a laser detector, etc.). In some embodiments, the client 200 may be a device with positioning technology for locating the position of the client 200.

[0051] In some embodiments, the client 200 may have one or more applications (APPs) installed. The APPs provide the target user 100 with the ability and interface to interact with the outside world via the network 400. The APPs include, but are not limited to: web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email clients, social media platform software, etc. In some embodiments, the client 200 may have a target APP installed. The target APP can capture target content or content images of the target content for the client 200, thereby obtaining at least one content image. In some embodiments, the target user 100 can also trigger a content detection request through the target APP. The target APP can respond to the content detection request by executing the content detection method described in this specification. The content detection method will be described in detail later.

[0052] Server 300 may be a server providing various services, such as a backend server supporting content detection of at least one content image of target content collected on client 200. In some embodiments, the content detection method may be executed on server 300. In this case, server 300 may store data or instructions for executing the content detection method described herein, and may execute or be used to execute said data or instructions. In some embodiments, server 300 may include hardware devices with data processing capabilities and the necessary programs required to drive the hardware devices. Server 300 may communicate with multiple clients 200 and receive data sent by clients 200.

[0053] Network 400 serves as a medium to provide a communication connection between client 200 and server 300. Network 400 facilitates the exchange of information or data. For example... Figure 1 As shown, client 200 and server 300 can connect to network 400 and transmit information or data to each other through network 400. In some embodiments, network 400 can be any type of wired or wireless network, or a combination thereof. For example, network 400 may include cable networks, wired networks, fiber optic networks, telecommunications networks, intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), and Bluetooth. TM Network, ZigBee TMA network, a near-field communication (NFC) network, or a similar network. In some embodiments, network 400 may include one or more network access points. For example, network 400 may include wired or wireless network access points, such as base stations or internet exchange points, through which one or more components of client 200 and server 300 can connect to network 400 to exchange data or information.

[0054] It should be understood that Figure 1 The number of clients 200, servers 300, and networks 400 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 200, servers 300, and networks 400.

[0055] It should be noted that the content detection method can be executed entirely on the client 200, entirely on the server 300, or partially on the client 200 and partially on the server 300.

[0056] Figure 2 A hardware structure diagram of a computing device 600 according to an embodiment of this specification is shown. The computing device 600 can execute the content detection method described in this specification. The content detection method is described in other parts of this specification. When the content detection method is executed on a client 200, the computing device 600 can be the client 200. When the content detection method is executed on a server 300, the computing device 600 can be the server 300. When the content detection method can be executed partly on the client 200 and partly on the server 300, the computing device 600 can be both the client 200 and the server 300.

[0057] like Figure 2 As shown, the computing device 600 may include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 may also include a communication port 650 and an internal communication bus 610. Additionally, the computing device 600 may include I / O components 660.

[0058] The internal communication bus 610 can connect different system components, including storage medium 630, processor 620 and communication port 650.

[0059] I / O component 660 supports input / output between computing device 600 and other components.

[0060] Communication port 650 is used for data communication between computing device 600 and external sources. For example, communication port 650 can be used for data communication between computing device 600 and network 400. Communication port 650 can be a wired communication port or a wireless communication port.

[0061] Storage medium 630 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. Storage medium 630 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the content detection methods provided in this specification.

[0062] At least one processor 620 can be communicatively connected to at least one storage medium 630 and a communication port 650 via an internal communication bus 610. The at least one processor 620 is used to execute the at least one instruction set described above. When the computing device 600 is running, the at least one processor 620 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the content detection method provided in this specification. The processor 620 can execute all the steps included in the content detection method. The processor 620 can be in the form of one or more processors. In some embodiments, the processor 620 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 620 is described in this specification for the computing device 600. However, it should be noted that the computing device 600 in this specification may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 620 of the computing device 600 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 620 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0063] Figure 3A flowchart of a content detection method P100 provided according to an embodiment of this specification is shown. As previously described, the computing device 600 can execute the content detection method P100 of this specification. Specifically, the processor 620 can read an instruction set stored in its local storage medium and then execute the content detection method P100 of this specification according to the instructions in the instruction set. Figure 3 As shown, method P100 may include:

[0064] S110: Obtain at least one content image of the target content.

[0065] The target content includes at least one of image content or video content. The content image can be an image containing the target content. For example, when the target content is image content, the content image can be the image content itself; when the target content is video content, the content image can be at least one video frame from the video content, and so on.

[0066] There are several ways to obtain at least one image of the target content, including the following:

[0067] For example, the processor 620 can directly acquire at least one content image of the target content uploaded by the target user 100 through the client 200 or terminal; or, it can acquire the target content and then acquire at least one content image of the target content through an image acquisition device; or, it can acquire at least one content image of the target content from the network or a content database; or, when the number of target contents is large or the memory is large, it can also receive a content detection request, which carries the storage address of the target content or at least one content image of the target content, and acquire at least one content image of the target content based on the storage address, and so on.

[0068] S120: Input at least one content image into the content detection model to obtain the forgery probability of the target content and at least one forgery feature corresponding to the forgery probability.

[0069] The content detection model is used to detect risks or forgery in target content. This model may include an image fusion network, a feature encoding network, a feature fusion network, and a forgery prediction network. The image fusion network is configured to linearly superimpose two content images to obtain a mixed content image. The feature encoding network is configured to encode image features of the content image and the mixed content image to obtain image features of the content image and the mixed content image. The feature fusion network is configured to linearly superimpose the image features of the content image and the mixed content image to obtain mixed image features. The forgery prediction network is configured to determine the forgery features corresponding to the image features and the mixed image features, and based on these forgery features, determine the probability that the target content is forgery. The forgery features can be feature information characterizing the target content as forgery. The forgery content can be text-generated image content or video content with an artistic style, etc. The forgery probability is the probability that the target content is forgery. The training samples of the content detection model include multiple forgery samples generated after diffusing reference content based on a multi-expert model group, where the multi-expert model group includes multiple complementary image-text matching models. The reference content includes a set of text and a reference image, which can be the original image before diffusion.

[0070] There are several ways to input at least one content image into the content detection model to obtain the probability of forged content of the target content and at least one forged content feature corresponding to the probability of forged content, which can be as follows:

[0071] For example, processor 620 can select content image pairs from at least one content image, input the content image pairs into an image mixing network to obtain a mixed content image, input the content image pairs and the mixed content image into a feature encoding network, and input an image feature set, which includes the image features of the content image pairs and the image features of the mixed content image. Then, the image feature set is input into a feature mixing network to obtain mixed features. The mixed features and the image feature set are input into a fake content prediction network to obtain a fake content probability and at least one fake content feature.

[0072] The training process of the content detection model may include the following: the processor 620 can acquire sample content image pairs corresponding to the target content samples in the training samples, input the sample content image pairs into the preset content detection model to obtain sample forged content features and predicted content categories, and converge the preset content detection model based on the sample forged content features and predicted content categories to obtain the trained content detection model.

[0073] There are several ways to obtain the content image sample pair corresponding to the target content sample in the training samples. For example, the processor 620 can randomly select at least one content sample as the target content sample in the training samples, then obtain the content image sample set of the target content sample, and randomly select two content image samples from the content image sample set to obtain the content image sample pair. Alternatively, it can select a content sample of a specific category as the target content sample, then obtain the content image sample set of the target content sample, and randomly select two content image samples from the content image sample set to obtain the content image sample pair, and so on.

[0074] After acquiring the content image sample pairs corresponding to the target content sample in the training samples, the processor 620 can input the content image sample pairs into a preset content detection model to obtain sample forgery content features and predicted content categories. There are several ways to input the content image sample pairs into the preset content detection model to obtain sample forgery content features and predicted content categories. For example, the processor 620 can linearly superimpose the content image samples in the content image sample pairs to obtain mixed content image samples, extract features from both the original content image samples and the mixed content image samples to obtain a candidate sample image feature set, extract sample forgery content features from the candidate sample image feature set, and determine the predicted content category of the target content sample based on the sample forgery content features.

[0075] There are several ways to linearly superimpose the content image samples in the content image sample pair. For example, the processor 620 can directly linearly superimpose the content image sample pair and then linearly superimpose the content tags corresponding to the content image samples to obtain a mixed content image sample. Here, the content tags can be labeled content categories.

[0076] After linearly superimposing the content image samples and extracting features, the processor 620 can extract the sample forged content features from the extracted candidate sample image feature set. There are several ways to extract sample forged content features from the candidate sample image feature set. For example, the processor 620 can linearly superimpose the sample image features in the candidate sample image feature set to obtain mixed sample image features. It can then extract the initial forged content features corresponding to each sample image feature from both the candidate sample image feature set and the mixed sample image feature set, and use these initial forged content features as the sample forged content features corresponding to the target content sample. Then, based on the sample forged content features, the predicted content category of the target content sample is determined.

[0077] After determining the sample forged content features and predicted content category of the target content sample, the processor 620 can converge the preset content detection model based on these features to obtain the trained content detection model. There are several ways to converge the preset content detection model. For example, the processor 620 can obtain the labeled content category of the target content sample, compare the labeled content category with the predicted content category to obtain content classification loss information, determine the feature prediction loss information of the target content sample based on the labeled content category and the sample forged content features, fuse the feature prediction loss information and the content classification loss information, and converge the preset content detection model based on the fused target content loss information to obtain the content detection model.

[0078] The content classification loss information refers to the loss information generated when classifying the target content samples. There are various ways to compare the labeled content category with the predicted content category. For example, the processor 620 can compare the labeled content category with the predicted content category based on the cross-entropy loss function to obtain the content classification loss information. Alternatively, it can compare the labeled content category with the predicted content category based on other loss functions to obtain the content classification loss information, and so on.

[0079] The labeled content categories include normal content samples and fake content samples. The fake content samples can be the content samples corresponding to fake content. The feature prediction loss information can be the loss information generated when predicting the features of fake content. The constraint condition for the feature prediction loss information is that the fake content features corresponding to the normal content samples are all zero features. There are multiple ways to determine the feature prediction loss information. For example, the processor 620 can determine the preset feature category corresponding to the target content sample based on the labeled content categories, determine the current feature category corresponding to the sample fake content features, and compare the preset feature category with the current feature category to obtain the feature prediction loss information. Feature categories can include all-zero features and non-all-zero features; the all-zero features can be features where all feature elements are 0.

[0080] After determining the content classification loss information and feature prediction loss information, the processor 620 can fuse the content classification loss information and feature prediction loss information to obtain the fused target content loss information. There are several ways to fuse the content classification loss information and feature prediction loss information. For example, the processor 620 can directly add the content classification loss information to obtain the target content loss information; alternatively, it can obtain content weights and, based on these weights, weight the content classification loss information and feature prediction loss information separately, and then fuse the weighted content classification loss information and the weighted feature prediction loss information to obtain the target content loss information, and so on.

[0081] After fusing the content classification loss information and the feature prediction loss information, the processor 620 can converge the preset content detection model based on the fused target content loss information, thereby obtaining the trained content detection model. There are several ways to converge the preset content detection model. For example, the processor 620 can use a gradient descent algorithm to update the network parameters of each network layer of the preset content detection model based on the target content loss information and / or the content classification loss information and feature prediction loss information within the target content loss information, and then return to the step of obtaining the content image sample pairs corresponding to the target content samples in the training samples, until the preset content detection model converges, thus obtaining the trained content detection model. Alternatively, it can use other network parameter update algorithms to update the network parameters of each network layer of the preset content detection model based on the target content loss information and / or the content classification loss information and feature prediction loss information within the target content loss information, and then return to the step of obtaining the content image sample pairs corresponding to the target content samples in the training samples, until the preset content detection model converges, thus obtaining the trained content detection model, and so on.

[0082] In some embodiments, before acquiring the content image sample pair corresponding to the target content sample in the training samples, the processor 620 may also generate or acquire training samples. The processor 620 may generate or acquire training samples in various ways. For example, the processor 620 may acquire reference content, input the reference content into at most a group of expert models and a diffusion model to obtain multiple fake content samples, and acquire multiple normal content samples, using the multiple normal content samples and the multiple fake content samples as training samples.

[0083] The reference content includes a text set and a reference image. There are several ways to obtain the reference content. For example, the processor 620 can randomly generate at least one text to obtain a text set, randomly select an image from a network or image database as a reference image, or it can receive reference content uploaded by the target user 100 through the client 200 or a terminal, and so on.

[0084] After acquiring the reference content, the processor 620 can input the reference content into the multi-expert model group and the diffusion model respectively to obtain multiple fake content samples. There are several ways to input the reference content into the multi-expert model group and the diffusion model respectively to obtain multiple fake content samples. For example, the processor 620 can input the reference image into the diffusion model to obtain an initial diffusion content image, input the initial diffusion content and the target text in the text set into the multi-expert model group to obtain noise increment images corresponding to multiple diffusion directions of the initial diffusion content image, and input the noise increment images into the diffusion model to obtain multiple fake content samples.

[0085] The noise increment image can be an increment image generated based on noise corresponding to each of multiple diffusion directions. The diffusion direction can be understood as the direction in which the reference image (or the initial diffused content image) is adjusted, modified, or diffused. This diffusion direction can be represented by noise. For example, generating a noise increment image based on the noise corresponding to a certain diffusion direction, and fusing the noise increment image with the reference image or the initial diffused content image, yields the diffused content image diffused in that direction. Furthermore, it should be noted that the diffused content image can be understood as the content image of the forged content sample. By using a multi-expert model group to diffuse in multiple directions, the randomness of the generated forged content samples can be increased, thereby improving the diversity and coverage of the generated content samples.

[0086] The training process of the diffusion model may include the following steps: the processor 620 can acquire reference image samples and keyword samples corresponding to the reference image samples, and input the reference image samples into a preset diffusion model to obtain initial diffusion content image samples. The keyword samples and initial diffusion content image samples are input into a multi-expert model group to obtain image-text similarity and multiple sample diffusion directions corresponding to the initial diffusion content image samples. The noise set corresponding to the multiple sample diffusion directions is input into the preset diffusion model to obtain the current diffusion content image sample set corresponding to the multiple sample diffusion directions. Based on image-text similarity, noise set and current diffusion content image sample set, the preset diffusion model is converged to obtain the trained diffusion model.

[0087] The keyword samples are text describing the style or art of the image, such as "Van Gogh's style." The image-text similarity includes the similarity between the keyword samples and the initial diffusion content image samples. The initial diffusion content image samples are obtained by randomly selecting a direction to diffuse the reference content using an initialized preset diffusion model.

[0088] Specifically, based on image-text similarity, noise set, and the current set of images containing the content to be spread, a pre-defined diffusion model is converged to obtain the trained diffusion model. There are several ways to converge the pre-defined diffusion model. For example, the processor 620 can input the current set of images containing the content to be spread into a pre-defined feature extraction model to obtain the evaluation features of each image sample in the current set of images containing the content to be spread. Based on image-text similarity, noise set, the current set of images containing the content to be spread, and the evaluation features, the target diffusion loss information is determined. Based on the target diffusion loss information, the pre-defined diffusion model is converged to obtain the diffusion model.

[0089] The evaluation features are those extracted by the feature extraction model for evaluating the diffused content image samples. Evaluation features can include style features and content features. The network structure of the feature extraction model can be varied, including CNN (Convolutional Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), or other network structures. The feature extraction model can be a local feature extraction model integrated within the processor 620, or a third-party feature extraction model.

[0090] After extracting the evaluation features of each diffused content image sample, the processor 620 can determine the target diffusion loss information based on image-text similarity, noise set, current diffused content image sample set, and evaluation features. The target diffusion loss information can be the loss information generated by a preset diffusion model when diffusing reference image samples or initial content diffused image samples. There are multiple ways to determine the target diffusion loss information. For example, the processor 620 can obtain the labeled image-text similarity between reference image samples and keyword samples, compare the labeled image-text similarity with the image-text similarity to obtain image-text similarity loss information, determine the sample evaluation loss information between diffused content image samples in the current diffused content image sample set based on evaluation features, determine the diffusion distance loss information between different sample diffusion directions based on the noise set, and fuse the image-text similarity loss information, sample evaluation loss information, and diffusion distance loss information to obtain the target diffusion loss information.

[0091] The sample evaluation loss information can be the feature information used to evaluate the style and content of diffused content image samples generated from multiple diffusion directions. The constraint condition for the sample evaluation loss information is that the feature difference between the evaluation features of different diffused content image samples is less than a preset difference threshold. This means that the sample evaluation loss information is used to constrain the style and content features of diffused content image samples generated from different diffusion directions to remain consistent. There are several ways to determine the sample evaluation loss information. For example, the processor 620 can separately acquire the feature similarity scores of style features and content features between diffused content image samples to obtain a style feature similarity set and a content feature similarity set. The feature similarity scores in the style feature similarity set are compared with a preset style feature similarity threshold to obtain style loss information, and the feature similarity scores in the content feature similarity set are compared with a preset content feature similarity threshold to obtain content loss information. Finally, the style loss information and content loss information are fused to obtain the sample evaluation loss information.

[0092] The diffusion distance loss information refers to the loss information of the distance between different sample diffusion directions. The constraint condition for the diffusion distance loss information is to constrain the diffusion distance between different sample diffusion directions to exceed a preset distance threshold, which means that the diffusion distance loss information is used to constrain the distance of multiple diffusion directions to be as large as possible. There are various ways to determine the diffusion distance loss information. For example, the processor 620 can perform feature extraction on the noise in the noise set to obtain the noise features corresponding to the diffusion direction of each sample in multiple sample diffusion directions, obtain the feature distance between the noise features to obtain the diffusion distance between multiple sample diffusion directions, and compare the diffusion distance with the preset distance threshold to obtain the diffusion distance loss information.

[0093] The feature distance can be L1 / L2 distance, Euclidean distance, etc.

[0094] After determining the image-text similarity loss information, sample evaluation loss information, and diffusion distance loss information, the processor 620 can fuse these information to obtain the target diffusion loss information. There are several ways to fuse the image-text similarity loss information, sample evaluation loss information, and diffusion distance loss information. For example, the processor 620 can directly add the image-text loss information, sample evaluation loss information, and diffusion distance loss information to obtain the target diffusion loss information, as shown in formula (1).

[0095] Loss total1 =Loss sim +Loss style-content +Lossmse (1)

[0096] Among them, Loss total1 Loss information is diffused to the target. sim For image-text similarity loss information, Loss style-content Loss information is used to evaluate the sample. mse Information is lost due to diffusion distance.

[0097] In some embodiments, the processor 620 may also obtain diffusion weights and weight the image-text similarity loss information, sample evaluation loss information, and diffusion distance loss information based on the diffusion weights, and fuse the weighted image-text similarity loss information, the weighted sample evaluation loss information, and the weighted diffusion distance loss information to obtain target diffusion loss information.

[0098] After determining the target diffusion loss information, the processor 620 can converge the preset diffusion model based on the target diffusion loss information. The method of converging the preset diffusion model is similar to the method of converging the preset content detection model, as described above, and will not be repeated here.

[0099] Before training the preset diffusion model, the processor 620 can train a multi-expert model group and train the preset diffusion model based on the trained multi-expert model group to obtain the trained diffusion model. Then, the multi-expert model group guides the trained diffusion model to generate multiple fake content samples. The multiple fake content samples and multiple normal content samples are used as training samples for the preset content detection model. The preset content detection model is trained using the training samples, and the trained content detection model is used to detect the target content.

[0100] The training process of the multi-expert model group may include the following steps: the processor 620 can acquire content sample pairs and input the content sample pairs into the preset multi-model group to obtain the feature set and feature similarity set corresponding to the content sample pairs, and converge the preset multi-model group based on the feature set and feature similarity set to obtain the trained multi-expert model group.

[0101] The content sample pairs include image samples and text samples. The preset multi-model group includes multiple preset image-text matching models and a feature transfer network. The image-text matching model is configured to predict the matching degree between image samples and text samples. Each image-text matching model may further include at least one image encoder and at least one text encoder. The image encoder is configured to encode features from image samples to obtain image features, and the text encoder is configured to encode features from text samples to obtain text features. The input to the image-text matching model is a matching / non-matching image and text pair, and the output is the corresponding features and their similarity. The feature transfer network is configured to predict the features output by other image-text matching models using the features output by one image-text matching model. The input to the feature transfer network is the feature (image feature or text feature) output by one image-text matching model, and the output is the predicted feature (image feature or text feature) output by another (other) image-text matching model. There are several ways to input content sample pairs into a preset multi-model group to obtain the feature set and feature similarity set corresponding to the content sample pairs. For example, the processor 620 can input the content sample pairs into multiple preset image matching models to obtain the sample feature set and the feature similarity set. Each sample feature in the sample feature set is input into the feature transfer network to obtain the transfer feature set. The transfer feature set and the sample feature set are used as the feature set.

[0102] The sample feature set includes the sample image features of image samples and the sample text features of text samples output by each of the multiple preset image-text matching models. The feature similarity set includes the feature similarity between the sample image features and sample text features output by each preset image-text matching model. The transfer feature set includes the transfer of sample features corresponding to any one of the multiple preset image-text matching models to sample features corresponding to other preset image-text matching models. For example, taking three preset image-text matching models as an example, the sample image features output by preset image-text matching model A are input into the feature transfer network to obtain the sample image features corresponding to preset image-text matching model B and preset image-text matching model C, and so on, thus obtaining the transfer feature set after feature transfer between different preset image-text matching models.

[0103] After obtaining the feature set and feature similarity set corresponding to the content sample pairs, the processor 620 can converge the preset multi-model group based on the feature set and feature similarity set to obtain the trained multi-expert model group. There are several ways to converge the preset multi-model group. For example, the processor 620 can obtain the annotation matching results of the content sample pairs, and determine the feature matching loss information based on the annotation matching results, feature similarity set, and sample feature set; determine the feature complementarity loss information between multiple preset text and image models based on the sample feature set; determine the feature transfer loss information based on the sample feature set and transfer feature set; and converge the preset multi-model group based on the feature matching loss information, feature complementarity loss information, and feature transfer loss information to obtain the multi-expert model group.

[0104] The annotation matching result can be a label indicating whether the image sample and text sample in the content sample pair match. The feature matching loss information can be the loss information generated by the difference between the feature matching degree between the sample image features and the sample text features and the sample matching degree between the image sample and the text sample. There are multiple ways to determine the feature matching loss information. For example, the processor 620 can determine the first preset similarity threshold corresponding to the content sample pair based on the annotation matching result, select the target feature similarity corresponding to each preset image-text matching model in the feature similarity set, and compare the target feature similarity with the first preset similarity threshold to obtain the first feature matching loss information corresponding to each preset image-text matching model. Based on the sample feature set and the first preset similarity threshold, the processor can determine the second feature matching loss information corresponding to the preset multi-model group, and use the first feature matching loss information and the second feature matching loss information as the feature matching loss information.

[0105] The first feature matching loss information can be the loss information generated by the difference between the feature similarity between the image sample and the text sample output by each preset image-text matching model and the labeled matching result of the content sample pair. The constraint condition for the first feature matching loss information is: for matched content sample pairs, the feature similarity between the image sample and the text sample is constrained to be greater than a first preset similarity threshold; or, for unmatched content sample pairs, the feature similarity between the image sample and the text sample is constrained to be less than the first preset similarity threshold. In other words, for matched content sample pairs, the features between the image sample and the text sample should be as consistent as possible; otherwise, they should be as inconsistent as possible.

[0106] The second feature matching loss information can be the matching loss of the sum of features input from multiple image-text matching models, or it can be understood as the matching loss after adding image features and text features. The constraint condition for the second feature matching loss information can be that, for matched image samples and text samples, the similarity of the sum of features is greater than a first preset similarity threshold; or, for mismatched image samples and text samples, the similarity of the sum of features is less than the first preset similarity threshold. There are multiple ways to determine the second feature matching loss information. For example, the processor 620 can select sample image features from the sample feature set to obtain a sample image feature set, and select sample text features from the sample feature set to obtain a sample text feature set. It can then accumulate the sample image features in the sample image feature set to obtain the target sample image features, and accumulate the sample text features in the sample text feature set to obtain the target sample text features. Furthermore, it can obtain the current feature similarity between the target sample image features and the target sample text features, and compare the current feature similarity with the first preset similarity threshold to obtain the second feature matching loss information.

[0107] The feature complementarity loss information can be the loss information generated by the differences between sample image features or sample text features output by multiple image-text matching models for the same image sample or text sample. The constraint condition for the feature complementarity loss information is that the feature similarity between the same type of sample features (sample image features or sample text features) output by different image-text matching models is less than a second preset similarity threshold. For example, taking a second preset similarity threshold of 0 as an example, the constraint condition could be that the cosine similarity between sample image features or sample text features output by different image-text matching models should be as close to 0 as possible. There are several ways to determine the feature complementarity loss information. For example, the processor 620 can select the same type of sample features output by different image-text matching models from the sample feature set to obtain at least one sample feature pair, obtain the feature similarity between the features in each sample feature pair in the at least one sample feature pair, obtain a complementary feature similarity set, and compare the feature similarity in the complementary feature similarity set with the second preset similarity threshold to obtain the feature complementarity loss information.

[0108] The feature transfer loss information refers to the loss information generated by the feature transfer network when transferring sample features output by different image-text matching models. The constraint on the feature transfer loss information is that the feature similarity between sample features of different preset image-text matching models during transfer is less than a third preset similarity threshold. This can also be understood as ensuring that when transferring sample image feature A output by preset image-text matching model A to sample image feature B output by preset image-text matching model B, the similarity (cosine similarity) between the transferred image feature B and the original sample image feature B is as low as possible. There are several ways to determine the feature transfer loss information. For example, the processor 620 can select candidate sample features corresponding to each transfer feature in the transfer feature set from the sample feature set, obtain the feature similarity between the candidate sample features and the corresponding transfer features to obtain a transfer feature similarity set, and compare the feature similarity in the transfer feature similarity set with the third preset similarity threshold to obtain the feature transfer loss information.

[0109] After determining the feature matching loss information, the complementary loss information, and the feature transfer loss information, the processor 620 can converge the preset multi-model group based on these information to obtain a multi-expert model group. There are several ways to converge the preset multi-model group. For example, the processor 620 can fuse the feature matching loss information, the complementary loss information, and the feature transfer loss information to obtain target loss information, and then converge the preset multi-model group based on this target loss information to obtain the multi-expert model group.

[0110] There are various ways to fuse feature matching loss information, complementary loss information and feature transfer loss information. For example, the processor 620 can directly add the feature matching loss information, complementary loss information and feature transfer loss information to obtain the target loss information, as shown in formula (2).

[0111] Loss total2 =Loss matc h+Loss ensemble-matc h+Loss complementaty +Loss transfer (2)

[0112] Among them, Loss total2 Loss information for the target matc h represents the first feature matching loss information, and Loss ensemble-matc h represents the second feature matching loss information, and Loss complementaty For feature complementation loss information, Loss transfer This is for feature transfer loss information.

[0113] In some embodiments, the processor 620 may also obtain target loss weights, and based on the target loss weights, weight the feature matching loss information, feature complementation loss information, and feature transfer loss information respectively, and fuse the weighted feature matching loss information, weighted feature complementation loss information, and weighted feature transfer loss information to obtain target loss information.

[0114] After fusing the feature matching loss information, feature complementarity loss information, and feature transfer loss information, the processor 620 can converge the preset multi-model group based on the fused target loss information, thereby obtaining the trained multi-expert model group. The convergence method for the preset multi-model group and the convergence method for the content detection model are detailed above and will not be repeated here.

[0115] It should be noted that during the training of the preset multi-model group, multiple complementary large-scale pre-trained models (pre-set image-text matching models) are trained to cover a variety of text and content scenarios.

[0116] S130: Based on the probability of the forged content and the at least one feature of the forged content, determine the risk detection result of the target content and output the risk detection result.

[0117] The risk detection results may include either fake content or normal content, wherein the fake content is text-based image or video content with an artistic style.

[0118] There are several ways to determine the risk detection results of the target content, including the following:

[0119] For example, the processor 620 can obtain the feature mean of at least one feature of fake content. When the probability of fake content is greater than a preset probability threshold, or when the feature mean is greater than a preset feature mean threshold, the risk detection result of the target content is determined to be fake content; or when the probability of fake content is less than a preset probability threshold and the feature mean is less than a preset feature mean threshold, the risk detection result of the target content is determined to be normal content.

[0120] After determining the risk detection result of the target content, the processor 620 can output the risk detection result. There are several ways to output the risk detection result. For example, the processor 620 can directly send the risk detection result to at least one of the client 200, terminal, or server corresponding to the target user 100, so that the client 200, terminal, or server can respond to the target content or the request corresponding to the target content based on the risk detection result. Alternatively, the risk detection result can be directly visualized, and so on.

[0121] There are various ways to visualize risk detection results. For example, the processor 620 can directly display the risk detection result, or it can display the risk detection result through sound and light (for example, by broadcasting the risk detection result by voice, or by displaying different types of risk detection results by displaying different colored lights, or by displaying the risk detection result through sound and light linkage), or it can display the risk detection result for specific types of risk detection results (for example, only displaying the risk detection result for fake content, or only displaying the risk detection result for normal content, etc.), and so on.

[0122] In some embodiments, after determining or outputting the risk detection result of the target content, the processor 620 may also respond to the target content or the request corresponding to the target content based on the risk detection result. There may be various ways to respond. For example, the processor 620 may directly intercept the target content or the request corresponding to the target content, or the processor 620 may directly perform secondary or multiple verifications on the target content and, based on the secondary verification results, give a final response to the target content or the request corresponding to the target content, and so on.

[0123] Taking content generated using Disco Diffusion technology based on forged content as an example, the overall process of the content detection method can be as follows: Figure 4 As shown, the main components include multi-expert model training, fake content generation, content detection model training, model deployment, and inference, which can be detailed as follows:

[0124] (1) Multi-expert model group training: Train multiple complementary large-scale pre-trained models to cover a variety of text and content scenarios. The specific training process can be found above.

[0125] (2) Forged content generation: Existing forged content generation methods are usually based on the diffusion process (i.e. Markov process) known by a single pre-trained model, which makes the generated content patterns relatively simple and will affect the performance of subsequent content detection models. Therefore, in this scheme, a multi-expert model group trained is used for joint guidance to increase the randomness of the generated data, thereby improving the diversity and coverage of the content.

[0126] (3) Content detection model training: The generated fake content samples (fake images) and existing data are used as training data to train the preset content detection model, so as to obtain a content detection model with high coverage and high security performance. In addition, during the training phase, a two-way mixing (mixup) of image and feature levels is introduced to improve the performance of the content detection model.

[0127] (4) Model Deployment and Inference: The trained content detection model is deployed to the server (local or remote). The content image collected by the client 200 is uploaded to the server. The content image is input into the content detection model to obtain the probability of content being fake and the corresponding fake text features. If the probability of content being fake p is greater than threshold T1 or the mean of the fake content features is greater than threshold T2, it is determined to be fake content; otherwise, it is determined to be normal content. Specifically, it can be done as follows: Figure 5 As shown.

[0128] This solution trains multiple complementary large-scale pre-trained models to obtain a multi-expert model group covering various text and content scenarios. Then, under the guidance of the multi-expert model group, the reference content is diffused on a large scale to generate diverse and high-coverage fake content samples. The pre-set content detection model is trained using existing data from the fake content sample set, which effectively solves the problem of unevenly distributed training data for text, thereby obtaining a content security model with high coverage and high security capabilities, and thus improving the detection performance of content detection.

[0129] In summary, the content detection method P100 and system 001 provided in this specification, after acquiring at least one content image of the target content, input at least one content image into a content detection model to obtain the forgery probability of the target content and at least one forgery feature corresponding to the forgery probability. The training samples of the content detection model include multiple forgery samples generated after diffusion of reference content based on a multi-expert model group. The multi-expert model group includes multiple complementary image-text matching models, and the risk detection result of the target content is determined based on the forgery probability and at least one forgery feature, and the risk detection result is output. Since this scheme guides the diffusion of reference content through a multi-expert model group, and the multi-expert model includes multiple complementary image-text matching models, the randomness of the generated data is increased. In addition, no other additional factors need to be introduced, thereby improving the diversity and coverage of the generated forgery samples, and thus obtaining a content detection model with high coverage and high performance. Therefore, the detection performance of content detection can be improved.

[0130] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for performing content detection. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of the content detection method P100 described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 600, the program code causes the computing device 600 to perform the steps of the content detection method P100 described herein. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the computing device 600. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 600, partially on computing device 600, as a standalone software package, partially on computing device 600 and partially on a remote computing device, or entirely on a remote computing device.

[0131] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0132] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0133] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0134] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0135] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. All contents used for all purposes, except for any history of prosecution documents relating to it, that may be inconsistent with or conflict with this document, or any such history of prosecution documents that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0136] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A content detection method, comprising: Obtain at least one content image of the target content; The at least one content image is input into the content detection model to obtain the forgery probability of the target content and at least one forgery feature corresponding to the forgery probability. The training samples of the content detection model include multiple forgery samples generated after the reference content is diffused based on a multi-expert model group. The multi-expert model group includes multiple complementary image-text matching models. The reference content includes a text set and a reference image. as well as Based on the probability of the forged content and the at least one feature of the forged content, the risk detection result of the target content is determined and the risk detection result is output.

2. The content detection method according to claim 1, wherein, The target content includes at least one of image content or video content.

3. The content detection method according to claim 1, wherein, The training process of the multi-expert model group includes the following steps: Obtain content sample pairs and input them into a preset multi-model group to obtain the feature set and feature similarity set corresponding to the content sample pairs; and Based on the feature set and the feature similarity set, the preset multi-model group is converged to obtain the trained multi-expert model group.

4. The content detection method according to claim 3, wherein, The content sample pairs include image samples and text samples, and the preset multi-model group includes multiple preset image-text matching models and feature transfer networks; as well as The step of inputting the content sample pairs into a preset multi-model group to obtain the feature set and feature similarity set corresponding to the content sample pairs includes: The content sample pairs are respectively input into the plurality of preset image-text matching models to obtain a sample feature set and a feature similarity set. The sample feature set includes the sample image features of the image sample and the sample text features of the text sample output by each preset image-text matching model. The feature similarity set includes the feature similarity between the sample image features and the sample text features output by each preset image-text matching model. Each sample feature in the sample feature set is input into the feature transfer network to obtain a transferred feature set. The transferred feature set includes sample features corresponding to any one of the plurality of preset image-text matching models that have been transferred to sample features corresponding to other preset image-text matching models among the plurality of preset image-text matching models. The migration feature set and the sample feature set are used as the feature set.

5. The content detection method according to claim 4, wherein, The step of converging the preset multi-model group to obtain the trained multi-expert model group includes: Obtain the annotation matching results of the content sample pairs, and determine the feature matching loss information based on the annotation matching results, the feature similarity set, and the sample feature set; Based on the sample feature set, determine the feature complementarity loss information among the multiple preset image-text matching models; Based on the sample feature set and the transfer feature set, feature transfer loss information is determined; and Based on the feature matching loss information, feature complementation loss information, and feature transfer loss information, the preset multi-model group is converged to obtain the multi-expert model group.

6. The content detection method according to claim 5, wherein, The step of determining feature matching loss information based on the annotation matching result, the feature similarity set, and the sample feature set includes: Based on the annotation matching results, a first preset similarity threshold is determined for the content sample pair; Select the target feature similarity corresponding to each preset image-text matching model from the feature similarity set, and compare the target feature similarity with the first preset similarity threshold to obtain the first feature matching loss information corresponding to each preset image-text matching model; Based on the sample feature set and the first preset similarity threshold, determine the second feature matching loss information corresponding to the preset multi-model group; and The first feature matching loss information and the second feature matching loss information are used as the feature matching loss information.

7. The content detection method according to claim 6, wherein, The step of determining the second feature matching loss information corresponding to the preset multi-model group based on the sample feature set and the first preset similarity threshold includes: The sample image features are selected from the sample feature set to obtain the sample image feature set, and the sample text features are selected from the sample feature set to obtain the sample text feature set; The sample image features in the sample image feature set are summed to obtain the target sample image features, and the sample text features in the sample text feature set are summed to obtain the target sample text features; and The current feature similarity between the target sample image features and the target sample text features is obtained, and the current feature similarity is compared with the first preset similarity threshold to obtain the second feature matching loss information.

8. The content detection method according to claim 5, wherein, The step of determining the feature complementarity loss information among the multiple preset image-text matching models based on the sample feature set includes: Select sample features of the same type from the image-text matching models in the sample feature set to obtain at least one sample feature pair; Obtain the feature similarity between features in each of the at least one sample feature pair to obtain a complementary feature similarity set; and The feature similarity in the complementary feature similarity set is compared with a second preset similarity threshold to obtain the feature complementary loss information. The constraint condition of the feature complementary loss information is that the feature similarity between the same type of sample features output by different image-text matching models is less than the second preset similarity threshold.

9. The content detection method according to claim 5, wherein, The step of determining feature transfer loss information based on the sample feature set and the transfer feature set includes: Select candidate sample features corresponding to each migration feature in the migration feature set from the sample feature set; Obtain the feature similarity between the candidate sample features and the corresponding transfer features to obtain a transfer feature similarity set; and The feature similarity in the set of migration feature similarities is compared with a third preset similarity threshold to obtain the feature migration loss information. The constraint condition of the feature migration loss information is that the feature similarity between sample features of different preset image-text matching models during migration is less than the third preset similarity threshold.

10. The content detection method according to claim 1, wherein, The training process of the content detection model includes the following steps: Obtain the content image sample pair corresponding to the target content sample in the training sample; The content image sample pairs are input into a preset content detection model to obtain sample forged content features and predicted content categories; and Based on the sample forged content features and the predicted content category, the preset content detection model is converged to obtain the trained content detection model.

11. The content detection method according to claim 10, wherein, The step of inputting the content image sample pairs into a preset content detection model to obtain sample forged content features and predicted content categories includes: Linearly superimpose the content image samples in the content image sample pair to obtain a mixed content image sample; Feature extraction is performed on the content image sample pairs and the mixed content image samples respectively to obtain a candidate sample image feature set; and The forged content features of the sample are extracted from the set of candidate sample images, and the predicted content category of the target content sample is determined based on the forged content features.

12. The content detection method according to claim 11, wherein, The step of extracting the forged content features of the sample from the candidate sample image feature set includes: The sample image features in the candidate sample image feature set are linearly superimposed to obtain the mixed sample image features; Extract the initial forged content features corresponding to each sample image feature from the candidate sample image feature set and the mixed sample image features respectively; and The initial forged content features are used as the sample forged content features corresponding to the target content sample.

13. The content detection method according to claim 10, wherein, The step of converging the preset content detection model to obtain the trained content detection model includes: Obtain the labeled content category of the target content sample, and compare the labeled content category with the predicted content category to obtain content classification loss information; Based on the labeled content category and the characteristics of the sample forged content, determine the feature prediction loss information of the target content sample; and The feature prediction loss information and the content classification loss information are fused together, and the preset content detection model is converged based on the fused target content loss information to obtain the content detection model.

14. The content detection method according to claim 13, wherein, The labeled content categories include normal content samples and fake content samples; as well as The step of determining the feature prediction loss information of the target content sample based on the labeled content category and the sample forged content features includes: Based on the labeled content category, determine the preset feature category corresponding to the target content sample. Determine the current feature category corresponding to the forged content features of the sample, and The preset feature category is compared with the current feature category to obtain the feature prediction loss information. The constraint condition of the feature prediction loss information is that the sample fake content feature corresponding to the constrained normal content sample is all zero features.

15. The content detection method according to claim 10, wherein, Before obtaining the content image sample pair corresponding to the target content sample in the training sample, the method further includes: Obtain the aforementioned reference content; The reference content is input into the multi-expert model group and the diffusion model respectively to obtain multiple samples of forged content; and Multiple normal content samples are obtained, and the multiple normal content samples and the multiple fake content samples are used as the training samples.

16. The content detection method according to claim 15, wherein, The step involves inputting the reference content into the multi-expert model group and the diffusion model, respectively, to obtain multiple samples of forged content, including: The reference image is input into the diffusion model to obtain the initial diffusion content image. The initial diffused content image and the target text from the text set are input into the multi-expert model group to obtain noise increment images corresponding to multiple diffusion directions of the initial diffused content image, and The noise increment image is input into the diffusion model to obtain the multiple fake content samples.

17. The content detection method according to claim 15, wherein, The training process of the diffusion model includes the following steps: Obtain a reference image sample and the keyword sample corresponding to the reference image sample, and input the reference image sample into a preset diffusion model to obtain an initial diffusion content image sample; The keyword samples and the initial diffusion content image samples are input into the multi-expert model group to obtain the image-text similarity and multiple sample diffusion directions corresponding to the initial diffusion content image samples. The image-text similarity includes the similarity between the keyword samples and the initial diffusion content samples. The noise set corresponding to the multiple sample diffusion directions is input into the preset diffusion model to obtain the current diffusion content image sample set corresponding to the multiple sample diffusion directions; and Based on the image-text similarity, the noise set, and the current diffusion content image sample set, the preset diffusion model is converged to obtain the trained diffusion model.

18. The content detection method according to claim 17, wherein, The step of converging the preset diffusion model to obtain the trained diffusion model includes: The current set of diffused content image samples is input into a preset feature extraction model to obtain the evaluation features of each diffused content image sample in the current set of diffused content image samples; Based on the image-text similarity, the noise set, the current diffusion content image sample set, and the evaluation features, the target diffusion loss information is determined; and Based on the target diffusion loss information, the preset diffusion model is converged to obtain the diffusion model.

19. The content detection method according to claim 18, wherein, The determination of target diffusion loss information includes: Obtain the labeled image-text similarity between the reference image sample and the keyword sample, and compare the labeled image-text similarity with the image-text similarity to obtain image-text similarity loss information; Based on the evaluation features, the sample evaluation loss information between the diffusion content image samples in the current diffusion content image sample set is determined. The constraint condition for the sample evaluation loss information is that the feature difference between the evaluation features of different diffusion content image samples is less than a preset difference threshold. Based on the noise set, diffusion distance loss information between different diffusion directions of the plurality of samples is determined, wherein the constraint condition for the diffusion distance loss information is that the diffusion distance between different diffusion directions of the samples exceeds a preset distance threshold; and The target diffusion loss information is obtained by fusing the image-text similarity loss information, the sample evaluation loss information, and the diffusion distance loss information.

20. The content detection method according to claim 19, wherein, The evaluation features include style features and content features; as well as The step of determining the sample evaluation loss information among the diffused content image samples in the current diffused content image sample set based on the evaluation features includes: The feature similarity of style features and the feature similarity of content features among the diffused content image samples are obtained respectively to obtain a style feature similarity set and a content feature similarity set. The feature similarity in the style feature similarity set is compared with a preset style feature similarity threshold to obtain style loss information, and the feature similarity in the content feature similarity set is compared with a preset content feature similarity threshold to obtain content loss information. The style loss information and the content loss information are fused to obtain the sample evaluation loss information.

21. The content detection method according to claim 19, wherein, Determining the diffusion distance loss information between different sample diffusion directions among the multiple sample diffusion directions includes: The noise in the noise set is feature extracted to obtain the noise features corresponding to the diffusion direction of each sample in the multiple sample diffusion directions; Obtain the feature distance between the noise features to obtain the diffusion distance between the diffusion directions of the plurality of samples; and The diffusion distance is compared with a preset distance threshold to obtain the diffusion distance loss information.

22. The content detection method according to claim 1, wherein, The risk detection result includes either fake content or normal content, wherein the fake content is text-based image or video content with an artistic style; as well as The risk detection results for determining the target content include: Obtain the feature mean of the at least one forged content feature, and When the probability of the forged content is greater than a preset probability threshold, or when the mean value of the features is greater than a preset mean value of the features, the risk detection result of the target content is determined to be the forged content.

23. The content detection method according to claim 22, wherein, Also includes: When the forged content is less than the preset probability threshold and the average feature value is less than the preset average feature threshold, the risk detection result of the target content is determined to be normal content.

24. A content detection system, comprising: At least one storage medium storing at least one instruction set for content inspection; as well as At least one processor is communicatively connected to the at least one storage medium. When the content detection system is running, the at least one processor reads the at least one instruction set and executes the content detection method according to any one of claims 1-23 according to the instructions of the at least one instruction set.

Images