A content forgery attack detection method, device and equipment

Abstract

The embodiment of the specification discloses a content forgery attack detection method. Including: determining the end-to-cloud integrated model trained, the end-to-cloud integrated model including an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, a cloud-side forgery attack classifier, which is trained according to the corresponding classification loss and the end-side sparse constraint; determine the client and the cloud end where the end-to-cloud integrated model is split and deployed; obtain the to-be-detected content on the client, encode the to-be-detected content through the end-side feature encoder to obtain the end-side feature, input the end-side forgery attack classifier for classification to obtain the first classification result; if it is determined that the cloud end also needs to participate in detection, the end-side feature is converted and transmitted from the client to the cloud end, encoded through the cloud-side feature encoder to obtain the cloud-side feature, input the cloud-side forgery attack classifier for classification to obtain the second classification result; according to the classification result, it is judged whether the to-be-detected content belongs to the forgery attack.

Description

Technical Field

[0001] This specification relates to the field of machine learning technology, and in particular to a method, apparatus, and device for detecting content forgery attacks. Background Art

[0002] In recent years, content security has once again become a major concern in academia and industry. Deepfake attacks are currently one of the most threatening attacks in the content security field.

[0003] Specifically, deepfakes forge content by training deep learning models. Leveraging powerful deep learning technology, deepfakes can create forged content that is virtually indistinguishable from the real thing.

[0004] Based on this, how to effectively detect deepfakes attacks is a problem that must be solved in the field of content security. Summary of the Invention

[0005] One or more embodiments of this specification provide a content forgery attack detection method, apparatus, device, and storage medium to solve the following technical problem: how to effectively detect deepfakes attacks.

[0006] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0007] One or more embodiments of this specification provide a content forgery attack detection method, including:

[0008] Determine a trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier, and the end-cloud integrated model is trained based on corresponding classification losses and end-side sparsity constraints;

[0009] Determine to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud;

[0010] Obtaining content to be detected on the client, encoding the content to be detected using the end-side feature encoder to obtain end-side features, and inputting the features into the end-side forgery attack classifier for classification to obtain a first classification result;

[0011] If it is determined that the cloud side also needs to participate in the detection, the terminal side features are converted and transmitted from the client to the cloud side, encoded by the cloud side feature encoder to obtain cloud side features, and input into the cloud side forgery attack classifier for classification to obtain a second classification result;

[0012] According to the first classification result and / or the second classification result, it is determined whether the content to be detected belongs to a forgery attack.

[0013] One or more embodiments of this specification provide a content forgery attack detection device, including:

[0014] A model determination module determines a trained end-cloud integrated model, wherein the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier. The end-cloud integrated model is trained based on corresponding classification losses and end-side sparsity constraints.

[0015] A deployment determination module determines to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud;

[0016] a terminal-side detection module, which obtains content to be detected on the client, encodes the content to be detected using the terminal-side feature encoder to obtain terminal-side features, and inputs the features into the terminal-side forgery attack classifier for classification to obtain a first classification result;

[0017] The cloud detection module, if it is determined that the cloud needs to participate in the detection, converts the end-side features and transmits them from the client to the cloud, encodes them through the cloud feature encoder to obtain cloud features, and inputs them into the cloud forgery attack classifier for classification to obtain a second classification result;

[0018] The result judgment module judges whether the content to be detected belongs to a forgery attack according to the first classification result and / or the second classification result.

[0019] One or more embodiments of this specification provide a content forgery attack detection device, including:

[0020] at least one processor; and,

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0023] Determine a trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier, and the end-cloud integrated model is trained based on corresponding classifier losses and end-side sparsity constraints;

[0024] Determine to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud;

[0025] Obtaining content to be detected on the client, encoding the content to be detected using the end-side feature encoder to obtain end-side features, and inputting the features into the end-side forgery attack classifier for classification to obtain a first classification result;

[0026] If it is determined that the cloud side also needs to participate in the detection, the terminal side features are converted and transmitted from the client to the cloud side, encoded by the cloud side feature encoder to obtain cloud side features, and input into the cloud side forgery attack classifier for classification to obtain a second classification result;

[0027] According to the first classification result and / or the second classification result, it is determined whether the content to be detected belongs to a forgery attack.

[0028] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to:

[0029] Determine a trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier, and the end-cloud integrated model is trained based on corresponding classifier losses and end-side sparsity constraints;

[0030] Determine to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud;

[0031] Obtaining content to be detected on the client, encoding the content to be detected using the end-side feature encoder to obtain end-side features, and inputting the features into the end-side forgery attack classifier for classification to obtain a first classification result;

[0032] If it is determined that the cloud side also needs to participate in the detection, the terminal side features are converted and transmitted from the client to the cloud side, encoded by the cloud side feature encoder to obtain cloud side features, and input into the cloud side forgery attack classifier for classification to obtain a second classification result;

[0033] According to the first classification result and / or the second classification result, it is determined whether the content to be detected belongs to a forgery attack.

[0034] At least one of the above-mentioned technical solutions adopted in one or more embodiments of this specification can achieve the following beneficial effects: by training the end-side feature encoder, the end-side forgery attack classifier, the cloud-side feature encoder, and the cloud-side forgery attack classifier as a whole, that is, an end-cloud integrated model, and then splitting it into two parts, one part of which is deployed on the client and the other part is deployed on the cloud. Then, through the transmission of intermediate results, the entire inference process is completed collaboratively on both the end and the cloud. End-side sparse constraints are adopted during the overall training to effectively control the data scale of the end-side features and reduce the detection pressure on the client. The end-side features can also be converted accordingly and then transmitted to the cloud for processing, and the cloud will re-encode them according to its own capabilities and then detect them. Such processing fully utilizes the respective strengths of the end and cloud, avoids their respective weaknesses, takes into account the protection of data privacy on the end side, and can more effectively detect deepfakes attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] Figure 1 A schematic diagram of the overall concept of a content forgery attack detection method provided in one or more embodiments of this specification;

[0037] Figure 2 A flowchart of a content forgery attack detection method provided in one or more embodiments of this specification;

[0038] Figure 3 In an application scenario provided by one or more embodiments of this specification, Figure 2 A schematic diagram of an embodiment of the method;

[0039] Figure 4 A schematic diagram of the structure of a content forgery attack detection device provided in one or more embodiments of this specification;

[0040] Figure 5 A schematic diagram of the structure of a content forgery attack detection device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0041] The embodiments of this specification provide a content forgery attack detection method, apparatus, device, and storage medium.

[0042] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0043] Deepfakes: These technologies use text, audio, and image processing to generate false, factual content for distribution to content platforms or to impersonate individuals for authentication purposes. Multimodal deepfakes can significantly mislead public opinion and security.

[0044] Deepfakes attack detection: refers to the use of deep learning, machine learning and other technologies to detect whether the specified content contains deepfakes.

[0045] Deepfakes is a term currently used in the industry. For ease of understanding, this term will be used directly in the background introduction above and some of the following embodiments. However, to avoid the limitations of the literal meaning of the term, a more straightforward description is used in some of the following embodiments to replace the term, that is, "content forgery" to more broadly represent some forgery behaviors that include deepfakes in the common sense.

[0046] The applicant divides the deepfakes attack detection solutions he has tried into two types according to the deployment environment. The first type is a pure cloud-based deepfakes attack detection solution. This type of solution deploys the trained deepfakes attack detection model on a cloud server. In the inference stage, the client uploads the content data to the cloud, and then the cloud model outputs the corresponding deepfakes attack detection results based on the input. Due to the powerful computing power of the cloud, this type of solution can achieve better detection performance. However, this type of solution requires uploading the original content, is heavily dependent on the network conditions, and has the risk of privacy leakage. The second type is a purely local deepfakes attack detection solution. This type of solution deploys the trained deepfakes attack detection model to the client (for example, on terminal devices such as smartphones). After the client collects the content, it directly inputs it into the local model and then outputs the deepfakes attack detection results. This type of solution does not involve data transmission and therefore does not rely on the network. However, due to the limitation of the computing power on the end side, the detection performance is poor.

[0047] To solve the above problems, this application proposes an efficient deepfakes attack detection solution based on end-cloud integrated reasoning. Figure 1 . Figure 1 A schematic diagram of the overall concept of a content forgery attack detection method provided in one or more embodiments of this specification.

[0048] The overall approach consists of the following four parts:

[0049] End-to-end model training: Train an end-to-end deepfake attack detection model in terms of accuracy and efficiency.

[0050] Feature compression model training: Utilize the codec structure to further compress intermediate results, reduce transmission bandwidth requirements, and improve privacy and security.

[0051] Model deployment and update: The end-to-end cloud model is divided into two parts, one deployed on the client and the other on the cloud. Collaborative reasoning is performed on the content to be detected. This deployment solution allows for flexible updates and reduces update costs.

[0052] Deepfakes attack detection: Use the model obtained from the above training to detect deepfakes attacks.

[0053] It should be noted that the model training phase is performed in advance. When the trained model is actually used online, the process of deepfakes attack detection is performed. Figure 2 It mainly explains the process of online use, and then introduces the pre-executed model training process in a timely manner.

[0054] Figure 2 This is a flowchart of a method for detecting content forgery attacks, provided for one or more embodiments of this specification. This process can be executed on risk control-related devices, such as content review servers (e.g., servers for video sharing, live streaming, news, forums, etc.), facial recognition smart terminals (e.g., facial recognition vending machines, facial recognition cash registers, etc.), and electronic payment risk control servers. Certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0055] Figure 2 The process in includes the following steps:

[0056] S202: Determine the trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier. The end-cloud integrated model is trained based on corresponding classification losses and end-side sparse constraints.

[0057] In one or more embodiments of this specification, the "end" in "end-cloud" refers to the client, which is on the user side, also called the end side, and the "cloud" refers to the cloud, which is on the server side. The end-cloud integrated model is trained in an integrated manner and then split up and deployed on the client and cloud respectively. It is expected that the end-side forgery attack classifier and the cloud-side forgery attack classifier will also have the ability to detect deepfakes attacks. If necessary, these two classifiers can be trained separately, such as by conducting small sample fine-tuning training after the integrated training.

[0058] In one or more embodiments of this specification, it is mentioned above that the end-cloud integrated model is trained from the aspects of accuracy and efficiency.

[0059] The smaller the corresponding classification loss, the higher the accuracy of forged attack detection. The corresponding classification loss includes one or more of the following: the classification loss of the forged attack classifier on the end side for the training samples, the classification loss of the forged attack classifier on the cloud side for the training samples, and the classification loss of the forged attack classifier on the cloud side for the training samples.

[0060] The classification loss of the client-side forged attack classifier and the weighted joint classification loss of the training samples are 5.

[0061] It should be emphasized that the joint classification loss is the weighted sum of the classification results of the two classifiers, and the gap between the joint classification result and the corresponding training label is the joint classification loss. The more accurate the joint classification result, the smaller the joint classification loss. In this way, the two classifiers generate more joint classification results during the training process.

[0062] The dynamic changes and mutual constraints can more comprehensively tend to the common classification accuracy target, which helps to overcome the local optimal solution trap that each of them may fall into. In practical applications, due to the stronger computing power of the cloud, it is possible to consider using the joint classification loss function.

[0063] When the time is lost, a higher weight is given to the classification results of the cloud-side forged attack classifier.

[0064] The sparser the data on the client side, the more efficient the forgery attack detection will be. This can effectively reduce the computational burden on the client side. The corresponding sparse constraints on the client side include one or more of the following:

[0065] The weight sparsity constraints of multiple channels of the encoder and the sparsity constraints of the end-side features obtained by the end-side feature encoder from encoding the training samples.

[0066] For the sparse weight constraints of multiple channels of the end-side feature encoder, it is desirable that the channels with effective weights are as few as possible, so that the structure of the end-side feature encoder can be simplified as much as possible. For the sparse constraints of the end-side features obtained by the end-side feature encoder from encoding the training samples, it is desirable that the effective dimensions of the end-side features are as few as possible, so that the subsequent processing of the end-side features can be reduced.

[0067] These constraints can be combined with the classification loss as part of the total loss, allowing the cloud-end model to be trained with a zero total loss, allowing the model to be adjusted towards a balance between accuracy and efficiency until convergence.

[0068] S204: Determine to split the client and cloud where the end-cloud integrated model is deployed, the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud.

[0069] S206: Obtain the content to be detected on the client, encode the content to be detected 5 by the terminal-side feature encoder to obtain terminal-side features, input the features into the terminal-side forgery attack classifier for classification, and obtain a first classification result.

[0070] In one or more embodiments of this specification, the content to be detected is, for example, content to be published by a user on a client (e.g., an article provided by the user), or content to be used for identity authentication (e.g., a face video provided by the user). Before the content to be detected is actually used to achieve the user's purpose, it is detected whether the content to be detected is forged content, that is, whether the content to be detected is a forgery attack.

[0071] 0 The first classification result is a probability value (or it can be directly whether the content to be detected is forged content).

[0072] The conclusion itself) reflects the possibility that the content to be detected is a forged attack from the client's reasoning. By comparing with the corresponding set thresholds, the client's reasoning conclusion can be obtained.

[0073] If it is determined that the first classification result reflects that the possibility that the content to be detected is a forged attack is higher than the corresponding set threshold, then it can be determined that the content to be detected is a forged attack. In this case, if the security and efficiency requirements are high and the tolerance for misjudgment is high, then it can be considered to directly use the client's reasoning conclusion as the final conclusion without the cloud's cooperation in reasoning for further verification. The content to be detected can be directly intercepted on the client to reject the corresponding user intention, and there is no need to execute step S208. The final conclusion can be obtained directly based on the first classification result.

[0074] S208: If it is determined that the cloud side also needs to participate in the detection, the terminal side features are converted and transmitted from the client to the cloud side, encoded by the cloud side feature encoder to obtain cloud side features, and input into the cloud side forgery attack classifier for classification to obtain a second classification result.

[0075] If the first classification result reflects that the possibility that the content to be tested is a forged attack is not higher than the corresponding set threshold, then it means that the client believes that the content to be tested is safe. However, due to the limited capabilities of the client, it may be deceived. Therefore, in this case, for safety reasons, the cloud can further reason and verify (that is, determine whether the cloud needs to participate in the detection) before making a final conclusion.

[0076] In one or more embodiments of this specification, client privacy issues (primarily regarding the transmission process) and data transmission issues have been previously mentioned. By encoding the original content on the client, direct exposure of the original content to the cloud is avoided, thereby improving privacy and security. To improve transmission performance and reliability, client-side features are not transmitted directly to the cloud, but are converted before transmission. This conversion reduces the amount of transmitted data and further strengthens client privacy protection.

[0077] To achieve this goal, a feature compression model is pre-trained. In addition to compressing features, the feature compression model also has corresponding decoding capabilities. This decoding capability is used to reconstruct the compressed features after they are transmitted to the cloud. Based on this, if cloud-side detection is determined to be necessary, the trained feature compression model is selected. The client-side features are input into the feature compression model for compression. The resulting compressed features are then transmitted from the client to the cloud. Feature reconstruction is performed through the corresponding decoding process, and the cloud-based feature encoder then encodes the reconstructed features to produce cloud-based features.

[0078] In addition to compression processing, a highly robust feature compression model based on bilateral perturbation is provided to further improve the reliability of the transmission process. Some traditional feature compression models only consider the compression ratio of the feature and the task performance of the restored feature, but ignore the channel noise and channel dropout that the compressed feature may encounter during transmission. In view of this, the feature compression model provided in this application, on the one hand, improves overall performance by strengthening the perturbation of the input, and on the other hand, simulates the perturbation of channel noise and channel dropout on the compressed feature, which is collectively referred to as bilateral perturbation, to improve robustness in practical applications.

[0079] The feature compression model provided in this application includes an input disturber, a feature compressor, a channel disturbance simulator, and a feature decoder. The feature compression model is trained based on the feature reconstruction loss corresponding to the feature decoder (making the end-side features as consistent as possible with the reconstructed features), the input disturbance consistency loss corresponding to the input disturber (when the features before and after the input disturbance are used as input, the output is made as consistent as possible), and the channel disturbance consistency loss corresponding to the channel disturbance simulator (when the features before and after the channel disturbance are used as input, the output is made as consistent as possible). The use of these two consistency losses can effectively improve the anti-interference ability of the transmission process.

[0080] After the feature compression model is trained, there is no need to add interference yourself when it is actually used. Therefore, you can use a feature compressor and a feature decoder. The feature compressor is deployed on the client. When in use, the terminal-side features are input into the feature compressor for compression. The feature decoder is deployed on the cloud. When in use, the conversion features transmitted to the cloud after compression are input into the feature decoder in the cloud, and the features are reconstructed through corresponding decoding processing.

[0081] S210: Based on the first classification result and / or the second classification result, determine whether the content to be detected is a forgery attack. This step is the step of drawing a final conclusion.

[0082] In one or more embodiments of the present specification, similarly, the second classification result is, for example, a probability value, which reflects the possibility that the content to be detected is a forged attack from the perspective of cloud reasoning. By comparing with the corresponding set threshold, the cloud reasoning conclusion can be obtained. For example, if it is judged that the second classification result reflects that the possibility that the content to be detected is a forged attack is higher than the corresponding set threshold, it can be considered to overturn the previous positive conclusion of the client (that is, the cloud verification fails), and judge that the content to be detected is a forged attack, and use this as the final conclusion. If the second classification result is also a positive conclusion, then it is equivalent to passing the double detection of the client and the cloud, and it can be judged that the content to be detected does not belong to a forged attack, and use this as the final conclusion.

[0083] It should be noted that there are many ways to combine the first and second classification results to reach a final conclusion. Specific judgment logic can be set based on actual requirements for security, efficiency, etc. For example, if the first classification result is a negative conclusion, the cloud can also make appropriate decisions. If the second classification result is also a negative conclusion, the final judgment is that the content under test is a forgery attack.

[0084] pass Figure 2 The method trains the end-side feature encoder, end-side forgery attack classifier, cloud-side feature encoder, and cloud-side forgery attack classifier as a whole, namely, an end-cloud integrated model, and then splits it into two parts, one of which is deployed on the client and the other on the cloud. Then, through the transmission of intermediate results, the entire inference process is completed collaboratively on both the end and the cloud. End-side sparse constraints are adopted during the overall training to effectively control the data scale of the end-side features and reduce the detection pressure on the client. The end-side features can also be converted accordingly and then transmitted to the cloud for processing, which is then re-encoded by the cloud based on its own capabilities and then detected. This processing fully utilizes the respective strengths of the end and the cloud, avoids their respective weaknesses, takes into account the protection of end-side data privacy, and can more effectively detect deepfakes attacks.

[0085] based on Figure 2 This specification also provides some specific implementation plans and extension plans of the method, which will be described below.

[0086] In one or more embodiments of this specification, if a weight sparsity constraint is used for multiple channels of an end-side feature encoder during training, the weights of some channels will be reduced after training under this constraint. In this case, some channels can be pruned to simplify the end-side feature encoder. For example, for an end-side feature encoder trained according to the weight sparsity constraint, channels whose weights are not higher than a corresponding set threshold (for example, the threshold is set to 0 or a smaller value) among its multiple channels are pruned. The pruned end-side feature encoder can then be deployed on the client for subsequent inference.

[0087] In one or more embodiments of this specification, in actual applications, after the end-cloud integrated model is deployed, when encountering new scenarios or some detection cases where reasoning fails, targeted optimization may be required. In this case, considering that the cost of optimizing the entire model is high and it also involves deployment issues on both the end and the cloud, a simplified update solution that can be iterated efficiently is provided. Specifically, it includes: after the trained end-cloud integrated model is deployed, if it is necessary to optimize the end-cloud integrated model, a temporary simplified model (the temporary simplified model may also include a feature compressor) is constructed based on the end-side feature encoder, the cloud-side feature encoder, and the cloud-side forgery attack classifier, and the temporary simplified model is optimized and trained according to the corresponding classifier loss. During the optimization training process, only the weights of the modules in the cloud are updated, so there is no need to disturb the client.

[0088] According to the above description, in one application scenario provided by one or more embodiments of this specification, Figure 2 A schematic diagram of an embodiment of the method in Figure 3 shown.

[0089] exist Figure 3 It includes two models, namely the end-cloud integrated model and the feature compression model. Both models are deployed separately on the client and cloud sides.

[0090] For the end-cloud integrated model, the details are as follows:

[0091] Model structure: It consists of four parts: the first part is the on-device feature encoder, the second part is the on-device deepfakes classifier (which serves as the on-device forgery attack classifier mentioned above), the third part is the cloud-side feature encoder, and the fourth part is the cloud-side deepfakes classifier (which serves as the cloud-side forgery attack classifier mentioned above).

[0092] Input and output: The input of the on-device feature encoder is the original content (for example, images used as content training samples or content to be detected), and the output is the corresponding on-device features. The input of the on-device deepfakes classifier is the on-device features, and the output is the client-side deepfakes classification result (for example, the first classification result mentioned above). The input of the cloud-side feature encoder is the on-device features or reconstructed on-device features, and the output is the cloud-side features. The input of the cloud-side deepfakes classifier is the cloud-side features, and the output is the cloud-side deepfakes classification result (for example, the second classification result mentioned above).

[0093] Loss function: Loss function is recorded as Loss total =Loss cls +Loss weight-spa +Loss feat-spa +Loss ensemble-cls, including four parts. The first part is the classification loss, including the classification loss of the deepfakes classifier on the cloud and / or the client side. The second part is the weight sparsity constraint of the client-side feature encoder (for example, the L1 norm of the channel weight of the client-side feature encoder is used to make the L1 norm as small as possible). The third part is the sparsity constraint of the client-side features (for example, the L1 norm of the client-side features is used to make the L1 norm as small as possible). The fourth part is the joint classification loss (assuming a joint ratio of 0.9:0.1, then (0.9*cloud classification result + 0.1*client classification result) is made as accurate as possible).

[0094] Training method: Perform supervised training based on the above model structure and loss function until the model converges;

[0095] After training, the client-side feature encoder is pruned to remove channels with a weight of 0, resulting in a pruned client-side feature encoder for deployment.

[0096] For the feature compression model, the details are as follows:

[0097] Model structure: It consists of four parts: the first part is the input perturbator, the second part is the feature compressor, the third part is the channel perturbation simulator, and the fourth part is the feature decoder. The four parts are connected in series to form the entire model;

[0098] Input and output: The input perturbator takes the device-side features as input and outputs the perturbed device-side features. The feature compressor takes the device-side features before and after the perturbation as input and outputs the compressed features. The channel perturbation simulator takes the compressed features as input and outputs the compressed features after adding channel perturbations (including Gaussian noise and channel bit dropout, where the highest bit is set to zero. For example, the binary number 111 becomes the binary number 011 after bit dropout). The feature decoder takes the compressed features before and after the channel perturbation as input and outputs the reconstructed device-side features.

[0099] Loss function: Loss function is recorded as Loss total =Loss recon +Loss dis-cons +Loss channel-cons , including three parts, the first part is the feature reconstruction loss, the second part is the input perturbation consistency loss, and the third part is the channel perturbation consistency;

[0100] Training method: Supervised training is performed based on the above model structure and loss function (because the channel perturbation simulator is not differentiable on the cloud, its influence is ignored when calculating the gradient) until the model converges.

[0101] After the two models are trained, they are split and deployed. Specifically, the pruned end-side feature encoder, end-side deepfakes classifier, and feature compressor are deployed to the client; all trained modules can be deployed to the cloud (only a part of them is used in actual reasoning, and at least this part needs to be deployed ( Figure 3 Taking partial deployment on the cloud as an example), since model updates may require the use of other parts, full deployment on the cloud can also be considered.

[0102] After training and deployment, when the system is officially put into use, the content to be detected (assuming the user wants to publish the content) is generated on the client and input into the end-side feature encoder and end-side deepfakes classifier. The end-side features and the client-side inferred deepfake attack probability p1 are obtained.

[0103] T1 is the corresponding set threshold. If p1>T1, the content is directly judged to be a deepfake attack and blocked. Otherwise, the client-side features are input into the feature compressor to obtain compressed features, which are then transmitted to the cloud via the network. In the cloud, the compressed features are input into the feature decoder, and then processed by the cloud feature encoder and cloud deepfake classifier to obtain the cloud-based inferred deepfake attack probability p2.

[0104] T2 is the corresponding set threshold. If p2>T2, the content is directly judged as a deepfakes attack and blocked; otherwise, it is judged as normal content and published.

[0105] After deployment, if model optimization is required later, the pruned client-side feature encoder, feature compressor, feature decoder, cloud-side feature encoder, and cloud-side deepfakes classifier are connected in series to form a temporary simplified model. The temporary simplified model is retrained using classification loss. During the retraining phase, only the weights of the cloud-side encoder and cloud-side deepfakes classifier are updated, while the other modules remain unchanged. After the update is complete, only the cloud-side model needs to be updated, and no client update is required.

[0106] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 4 、 Figure 5 The apparatus and device can accordingly execute the above method and related optional solutions.

[0107] Figure 4 This is a schematic diagram of a content forgery attack detection device provided in one or more embodiments of this specification, the device comprising:

[0108] A model determination module 402 determines a trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier. The end-cloud integrated model is trained based on corresponding classification losses and end-side sparsity constraints.

[0109] A deployment determination module 404 determines to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud;

[0110] The end-side detection module 406 obtains content to be detected on the client, encodes the content to be detected using the end-side feature encoder to obtain end-side features, and inputs the features into the end-side forgery attack classifier for classification to obtain a first classification result.

[0111] If it is determined that the cloud needs to participate in the detection, the cloud detection module 408 converts the end-side features and transmits them from the client to the cloud. The features are encoded by the cloud feature encoder to obtain cloud features. The cloud features are input into the cloud forgery attack classifier for classification to obtain a second classification result.

[0112] The result judgment module 410 judges whether the content to be detected is a forgery attack according to the first classification result and / or the second classification result.

[0113] Optionally, the corresponding classification loss includes at least one of the following: classification losses of the device-side forged attack classifier and the cloud-side forged attack classifier on training samples respectively, and a weighted joint classification loss of the device-side forged attack classifier and the cloud-side forged attack classifier on training samples;

[0114] The end-side sparse constraint includes at least one of the following: a weight sparse constraint of multiple channels of the end-side feature encoder, and a sparse constraint of end-side features obtained by encoding training samples by the end-side feature encoder.

[0115] Optionally, the end-side sparse constraint includes at least weight sparse constraints of multiple channels of the end-side feature encoder;

[0116] The device further comprises:

[0117] The training and pruning module 412 prunes, before determining the trained end-cloud integrated model, the channels of the end-side feature encoder trained according to the weight sparsity constraint whose weights are not higher than a corresponding set threshold among the multiple channels.

[0118] Optionally, the result judgment module 410, after obtaining the first classification result, judges whether the first classification result reflects that the possibility that the content to be detected is a forgery attack is higher than a corresponding set threshold;

[0119] If so, it is determined that the content to be detected is a forged attack;

[0120] Otherwise, it is determined that the cloud also needs to participate in the detection.

[0121] Optionally, the result judgment module 410 judges whether the second classification result reflects that the possibility that the content to be detected is a forged attack is higher than a corresponding set threshold;

[0122] If so, it is determined that the content to be detected belongs to a forgery attack.

[0123] Optionally, the cloud detection module 408 determines a trained feature compression model;

[0124] Inputting the terminal-side features into the feature compression model for compression;

[0125] The compressed converted features are transmitted from the client to the cloud, and features are reconstructed through corresponding decoding processing. The reconstructed features are then encoded by the cloud feature encoder to obtain cloud features.

[0126] Optionally, the feature compression model includes an input perturbator, a feature compressor, a channel perturbation simulator, and a feature decoder, and the feature compression model is trained according to the feature reconstruction loss corresponding to the feature decoder, the input perturbation consistency loss corresponding to the input perturbator, and the channel perturbation consistency loss corresponding to the channel perturbation simulator;

[0127] The cloud detection module 408 inputs the terminal-side features into the feature compressor for compression;

[0128] The cloud detection module 408 inputs the feature decoder in the cloud and performs feature reconstruction through corresponding decoding processing.

[0129] Optionally, it also includes:

[0130] A model optimization module 414, after determining the trained end-cloud integrated model, if optimization of the end-cloud integrated model is required, constructs a temporary simplified model based on the end-side feature encoder, the end feature encoder, and the cloud-side forgery attack classifier;

[0131] The temporary simplified model is optimized and trained according to the corresponding classifier loss. During the optimization training, only the weights of the modules in the cloud are updated.

[0132] Figure 5 This is a schematic diagram of a content forgery attack detection device provided in one or more embodiments of this specification, the device comprising:

[0133] at least one processor; and,

[0134] a memory communicatively connected to the at least one processor; wherein,

[0135] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0136] Determine a trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier, and the end-cloud integrated model is trained based on corresponding classifier losses and end-side sparsity constraints;

[0137] Determine to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud;

[0138] Obtaining content to be detected on the client, encoding the content to be detected using the end-side feature encoder to obtain end-side features, and inputting the features into the end-side forgery attack classifier for classification to obtain a first classification result;

[0139] If it is determined that the cloud side also needs to participate in the detection, the terminal side features are converted and transmitted from the client to the cloud side, encoded by the cloud side feature encoder to obtain cloud side features, and input into the cloud side forgery attack classifier for classification to obtain a second classification result;

[0140] According to the first classification result and / or the second classification result, it is determined whether the content to be detected belongs to a forgery attack.

[0141] Based on the same idea, one or more embodiments of this specification further provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0142] Determine a trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier, and the end-cloud integrated model is trained based on corresponding classifier losses and end-side sparsity constraints;

[0143] Determine to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud;

[0144] Obtaining content to be detected on the client, encoding the content to be detected using the end-side feature encoder to obtain end-side features, and inputting the features into the end-side forgery attack classifier for classification to obtain a first classification result;

[0145] If it is determined that the cloud side also needs to participate in the detection, the terminal side features are converted and transmitted from the client to the cloud side, encoded by the cloud side feature encoder to obtain cloud side features, and input into the cloud side forgery attack classifier for classification to obtain a second classification result;

[0146] According to the first classification result and / or the second classification result, it is determined whether the content to be detected belongs to a forgery attack.

[0147] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0148] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the means for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the means for implementing various functions can be considered as both a software module implementing the method and a structure within the hardware component.

[0149] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0150] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0151] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0155] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0156] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0157] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0158] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0159] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0160] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0161] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0162] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A content forgery attack detection method, comprising: Determine a trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier, and the end-cloud integrated model is trained based on corresponding classification losses and end-side sparsity constraints; Determine to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud; Obtaining content to be detected on the client, encoding the content to be detected using the end-side feature encoder to obtain end-side features, and inputting the features into the end-side forgery attack classifier for classification to obtain a first classification result; If it is determined that the cloud side also needs to participate in the detection, the terminal side features are converted and transmitted from the client to the cloud side, encoded by the cloud side feature encoder to obtain cloud side features, and input into the cloud side forgery attack classifier for classification to obtain a second classification result; According to the first classification result and / or the second classification result, it is determined whether the content to be detected belongs to a forgery attack.

2. The method of claim 1, wherein the corresponding classification loss comprises at least one of the following: classification losses of the on-device forgery attack classifier and the cloud-based forgery attack classifier for training samples, respectively, and a weighted joint classification loss of the on-device forgery attack classifier and the cloud-based forgery attack classifier for training samples; The end-side sparse constraint includes at least one of the following: a weight sparse constraint of multiple channels of the end-side feature encoder, and a sparse constraint of end-side features obtained by encoding training samples by the end-side feature encoder.

3. The method according to claim 2, wherein the end-side sparse constraint comprises at least a weight sparse constraint of multiple channels of the end-side feature encoder; Before determining the trained end-cloud integrated model, the method further includes: For the end-side feature encoder trained according to the weight sparsity constraint, channels whose weights are not higher than a corresponding set threshold are pruned from the multiple channels.

4. The method according to claim 1, after obtaining the first classification result, the method further comprises: determining whether the first classification result indicates that the possibility that the to-be-detected content is a forged attack is higher than a corresponding set threshold; If so, it is determined that the content to be detected is a forged attack; Otherwise, it is determined that the cloud also needs to participate in the detection.

5. The method according to claim 1, wherein determining whether the content to be detected is a forgery attack based on the second classification result specifically comprises: determining whether the second classification result indicates that the possibility that the to-be-detected content is a forgery attack is higher than a corresponding set threshold; If so, it is determined that the content to be detected belongs to a forgery attack.

6. The method according to claim 1, wherein converting the terminal-side features and transmitting them from the client to the cloud, and encoding them by the cloud-side feature encoder to obtain cloud-side features, specifically comprises: Determine the trained feature compression model; Inputting the terminal-side features into the feature compression model for compression; The compressed converted features are transmitted from the client to the cloud, and features are reconstructed through corresponding decoding processing. The reconstructed features are then encoded by the cloud feature encoder to obtain cloud features.

7. The method of claim 6, wherein the feature compression model comprises an input perturbator, a feature compressor, a channel perturbation simulator, and a feature decoder, and the feature compression model is trained based on a feature reconstruction loss corresponding to the feature decoder, an input perturbation consistency loss corresponding to the input perturbator, and a channel perturbation consistency loss corresponding to the channel perturbation simulator. Inputting the terminal-side features into the feature compression model for compression specifically includes: Inputting the terminal-side features into the feature compressor for compression; The feature reconstruction through corresponding decoding processing specifically includes: The feature decoder is input into the cloud, and feature reconstruction is performed through corresponding decoding processing.

8. The method according to claim 1, after determining the trained end-cloud integrated model, the method further comprises: If optimization is required for the end-cloud integrated model, a temporary simplified model is constructed based on the end-side feature encoder, the cloud-side feature encoder, and the cloud-side forgery attack classifier; The temporary simplified model is optimized and trained according to the corresponding classifier loss. During the optimization training, only the weights of the modules in the cloud are updated.

9. A content forgery attack detection device, comprising: A model determination module determines a trained end-cloud integrated model, wherein the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier. The end-cloud integrated model is trained based on corresponding classification losses and end-side sparsity constraints. A deployment determination module determines to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud; a terminal-side detection module, which obtains content to be detected on the client, encodes the content to be detected using the terminal-side feature encoder to obtain terminal-side features, and inputs the features into the terminal-side forgery attack classifier for classification to obtain a first classification result; The cloud detection module, if it is determined that the cloud needs to participate in the detection, converts the end-side features and transmits them from the client to the cloud, encodes them through the cloud feature encoder to obtain cloud features, and inputs them into the cloud forgery attack classifier for classification to obtain a second classification result; The result judgment module judges whether the content to be detected belongs to a forgery attack according to the first classification result and / or the second classification result.

10. The apparatus of claim 9, wherein the corresponding classification loss comprises at least one of the following: classification losses of the on-device forgery attack classifier and the cloud-based forgery attack classifier on training samples, respectively, and a weighted joint classification loss of the on-device forgery attack classifier and the cloud-based forgery attack classifier on training samples; The end-side sparse constraint includes at least one of the following: a weight sparse constraint of multiple channels of the end-side feature encoder, and a sparse constraint of end-side features obtained by encoding training samples by the end-side feature encoder.

11. The apparatus according to claim 10, wherein the end-side sparse constraint comprises at least a weight sparse constraint of multiple channels of the end-side feature encoder; The device further comprises: The training and pruning module prunes, before determining the trained end-cloud integrated model, the channels of the end-side feature encoder trained according to the weight sparsity constraint whose weights are not higher than the corresponding set threshold among the multiple channels.

12. The apparatus according to claim 9, wherein the result judgment module, after obtaining the first classification result, judges whether the first classification result reflects that the possibility that the content to be detected is a forgery attack is higher than a corresponding set threshold; If so, it is determined that the content to be detected is a forged attack; Otherwise, it is determined that the cloud also needs to participate in the detection.

13. The apparatus according to claim 9, wherein the result judgment module judges whether the second classification result reflects that the possibility that the content to be detected is a forgery attack is higher than a corresponding set threshold; If so, it is determined that the content to be detected belongs to a forgery attack.

14. The apparatus of claim 9, wherein the cloud detection module determines a trained feature compression model; Inputting the terminal-side features into the feature compression model for compression; The compressed converted features are transmitted from the client to the cloud, and features are reconstructed through corresponding decoding processing. The reconstructed features are then encoded by the cloud feature encoder to obtain cloud features.

15. The apparatus of claim 14, wherein the feature compression model comprises an input perturbator, a feature compressor, a channel perturbation simulator, and a feature decoder, wherein the feature compression model is trained based on a feature reconstruction loss corresponding to the feature decoder, an input perturbation consistency loss corresponding to the input perturbator, and a channel perturbation consistency loss corresponding to the channel perturbation simulator. The cloud detection module inputs the terminal-side features into the feature compressor for compression; The cloud detection module inputs the feature decoder in the cloud and performs feature reconstruction through corresponding decoding processing.

16. The apparatus of claim 9, further comprising: a model optimization module, after determining the trained end-cloud integrated model, constructing a temporary simplified model based on the end-side feature encoder, the cloud-side feature encoder, and the cloud-side forgery attack classifier if optimization of the end-cloud integrated model is required; The temporary simplified model is optimized and trained according to the corresponding classifier loss. During the optimization training, only the weights of the modules in the cloud are updated.

17. A content forgery attack detection device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, wherein the instructions are executed by the at least one processor to enable the at least one processor to perform: Determine a trained end-cloud integrated model, where the end-cloud integrated model includes an end-side feature encoder, an end-side forgery attack classifier, a cloud-side feature encoder, and a cloud-side forgery attack classifier, and the end-cloud integrated model is trained based on corresponding classifier losses and end-side sparsity constraints; Determine to split the client and cloud where the end-cloud integrated model is deployed, wherein the end-side feature encoder and the end-side forgery attack classifier are deployed on the client, and the cloud-side feature encoder and the cloud-side forgery attack classifier are deployed on the cloud; Obtaining content to be detected on the client, encoding the content to be detected using the end-side feature encoder to obtain end-side features, and inputting the features into the end-side forgery attack classifier for classification to obtain a first classification result; If it is determined that the cloud side also needs to participate in the detection, the terminal side features are converted and transmitted from the client to the cloud side, encoded by the cloud side feature encoder to obtain cloud side features, and input into the cloud side forgery attack classifier for classification to obtain a second classification result; According to the first classification result and / or the second classification result, it is determined whether the content to be detected belongs to a forgery attack.

Images