Image tampering detection method, device, storage medium and electronic device

Abstract

This specification discloses an image tampering detection method, device, storage medium and electronic device, wherein the method includes: the electronic device determines a first model gradient matrix corresponding to a local image tampering detection model, then performs local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix, and then uploads the second model gradient matrix to a service platform. The second model gradient matrix is ​​used by the service platform to determine target model parameters based on each second model gradient matrix. Thereafter, the electronic device receives the target model parameters sent by the service platform, updates the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model, and then performs image detection processing on the target privacy image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target privacy image, and then performs tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target privacy image.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to an image tampering detection method, device, storage medium, and electronic device. Background Art

[0002] In consumer finance scenarios, transactions between consumers and financial institutions involve numerous financial documents containing user identity information, such as salary certificates, professional credentials, and IDs. Typically, these documents are uploaded by users to the financial institution's platform in the form of images. After the platform verifies the documents and reviews the information, transactions can be processed. Summary of the Invention

[0003] This specification provides an image tampering detection method, device, storage medium, and electronic device. The technical solution is as follows:

[0004] In a first aspect, this specification provides an image tampering detection method, applied to an electronic device, the method comprising:

[0005] Determine a first model gradient matrix corresponding to the local image tampering detection model, and perform local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix;

[0006] Uploading the second model gradient matrix to the service platform, where the second model gradient matrix is ​​used to instruct the service platform to determine target model parameters based on the second model gradient matrix uploaded by the at least one electronic device;

[0007] receiving the target model parameters sent by the service platform, and updating the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model;

[0008] Based on the target image tampering detection model, image detection processing is performed on the target private image to obtain a target thermal detection map corresponding to the target private image, and tampering identification processing is performed on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0009] In a second aspect, this specification provides an image tampering detection method, which is applied to a service platform. The method includes:

[0010] Receive a second model gradient matrix uploaded by at least one electronic device, where the second model gradient matrix is ​​obtained by the electronic device performing local gradient differential privacy calculation based on the first model gradient matrix corresponding to the local image tampering detection model;

[0011] determining target model parameters based on each of the second model gradient matrices;

[0012] The target model parameters are sent to each of the electronic devices, where the target model parameters are used to instruct the electronic device to update a local image tampering detection model based on the target model parameters to obtain a target image tampering detection model, perform image detection processing on a target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0013] In a third aspect, this specification provides an image tampering detection device, applied to an electronic device, comprising:

[0014] A gradient processing module, configured to determine a first model gradient matrix corresponding to a local image tampering detection model, and perform local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix;

[0015] a gradient sending module, configured to upload the second model gradient matrix to a service platform, wherein the second model gradient matrix is ​​used to instruct the service platform to determine target model parameters based on the second model gradient matrix uploaded by the at least one electronic device;

[0016] a data receiving module, configured to receive the target model parameters sent by the service platform, and update the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model;

[0017] An image processing module is configured to perform image detection processing on a target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0018] In a fourth aspect, this specification provides an image tampering detection device, which is applied to a service platform, and the device includes:

[0019] A data receiving module, configured to receive a second model gradient matrix uploaded by at least one electronic device, where the second model gradient matrix is ​​obtained by performing a local gradient differential privacy calculation on the first model gradient matrix corresponding to the local image tampering detection model by the electronic device;

[0020] a data processing module, configured to determine target model parameters based on each of the second model gradient matrices;

[0021] a data sending module, configured to send the target model parameters to each of the electronic devices, wherein the target model parameters are used to instruct the electronic devices to update a local image tampering detection model based on the target model parameters to obtain a target image tampering detection model, perform image detection processing on a target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0022] In a fifth aspect, this specification provides a computer storage medium having a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0023] In a sixth aspect, this specification provides a computer program product, wherein the computer program product stores at least one instruction, and the at least one instruction is loaded by a processor to execute the above-mentioned method steps.

[0024] In a seventh aspect, this specification provides an electronic device, which may include: a memory and a processor; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the memory and executing the above-mentioned method steps.

[0025] The beneficial effects of the technical solutions provided in this specification include at least:

[0026] In an embodiment of the present specification, an electronic device determines a first model gradient matrix corresponding to a local image tampering detection model, and then performs local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix. The electronic device then uploads the second model gradient matrix to a service platform. The second model gradient matrix is ​​used by the service platform to determine target model parameters based on the second model gradient matrix uploaded by at least one electronic device. Thereafter, the electronic device receives the target model parameters sent by the service platform, and the electronic device updates the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model. The electronic device then performs image detection processing on the target privacy image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target privacy image, and then performs tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target privacy image. Using the above method steps, the electronic device pre-processes the gradient data to be uploaded to the service platform to improve the uploaded gradient data's ability to resist reverse attacks. Since the target image tampering detection model used by the electronic device to perform image detection processing on the private image is a model obtained by processing the model parameters obtained by the service platform based on the gradient data of multiple electronic devices, the model combines multi-party data during training, so the model's recognition accuracy for private images is improved. Therefore, the recognition results of the model are then tampered with, thereby improving the accuracy of the tampering recognition results of the private image. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 This is a schematic diagram of the system architecture of an image tampering detection method provided by an embodiment of this specification;

[0029] Figure 2 This is a flow chart of an image tampering detection method provided in an embodiment of this specification;

[0030] Figure 3 This is a flowchart of another image tampering detection method provided by an embodiment of this specification;

[0031] Figure 4 is a schematic diagram of a target thermal detection map provided in an embodiment of this specification;

[0032] Figure 5 This is a flow chart of an image tampering detection method provided in an embodiment of this specification;

[0033] Figure 6 This is a flowchart of another image tampering detection method provided by an embodiment of this specification;

[0034] Figure 7 This is a structural diagram of an image tampering detection device provided in an embodiment of this specification;

[0035] Figure 8 This is a structural diagram of an image tampering detection device provided in an embodiment of this specification;

[0036] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0037] Figure 10 It is a structural diagram of a service platform provided in an embodiment of this specification. DETAILED DESCRIPTION

[0038] In order to make the invention objectives, features, and advantages of the embodiments of this specification more obvious and easy to understand, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this specification.

[0039] In the description of this specification, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in this specification can be understood according to the specific circumstances. In addition, in the description of this specification, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0040] In consumer finance scenarios, financial institutions are required to review user-uploaded materials, such as financial credentials containing user identity information, and process related transactions for users upon approval. Tamper detection of user-uploaded materials by financial institutions is crucial for financial credential review. Related technologies rely on self-trained tamper detection models to detect tampering in user-uploaded materials. However, individual financial institutions possess limited data, which is not permitted to be shared externally. Consequently, the amount of training data available to these models is limited, resulting in poor tamper detection performance.

[0041] In order to solve the above technical problems, this specification is described in detail below in conjunction with specific embodiments.

[0042] See Figure 1 , is a system architecture diagram of an image tampering detection method provided in an embodiment of this specification.

[0043] like Figure 1 As shown, Figure 1 Including service platforms and electronic equipment, Figure 1 The number of electronic devices and the number of nodes in the service platform shown are only exemplary, and the embodiments of this specification do not limit their numbers.

[0044] In the embodiments of this specification, the service platform may refer to a server consisting of one node or a server cluster consisting of multiple nodes.

[0045] When the service platform is a server cluster composed of multiple nodes, each node in the service platform can be a separate server device, such as: rack-mounted, blade, tower, or cabinet-type server equipment, or a workstation, mainframe computer and other hardware devices with strong computing capabilities; it can also be a server cluster composed of multiple servers. The servers in the service cluster can be composed in a symmetrical manner, where each server has equivalent functions and status in the transaction link, and each server can provide services to the outside world independently. Providing services to the outside world independently can be understood as not requiring the assistance of other servers.

[0046] In the embodiments of this specification, the electronic device may be a computer device that has functions such as image detection and tamper identification processing for private images. Different electronic devices are devices of different consumer finance institutions.

[0047] It should be noted that the electronic device and at least one node in the service platform establish a communication connection through a network for interactive communication. The network can be a wireless network or a wired network. Wireless networks include, but are not limited to, cellular networks, wireless local area networks, infrared networks, or Bluetooth networks. Wired networks include, but are not limited to, Ethernet, universal serial bus (USB), or controller area network. In one or more embodiments of the specification, technologies and / or formats including Hypertext Markup Language (HTML) and Extensible Markup Language (XML) are used to represent data (such as target compressed packages) exchanged over the network. Conventional encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the above-mentioned data communication technologies.

[0048] In an embodiment of the present specification, an electronic device determines a first model gradient matrix corresponding to a local image tampering detection model, and then performs local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix. The electronic device then uploads the second model gradient matrix to a service platform. The second model gradient matrix is ​​used by the service platform to determine target model parameters based on the second model gradient matrix uploaded by at least one electronic device. Thereafter, the electronic device receives the target model parameters sent by the service platform, and the electronic device updates the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model. The electronic device then performs image detection processing on the target privacy image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target privacy image, and then performs tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target privacy image. Using the above method steps, the electronic device pre-processes the gradient data to be uploaded to the service platform to improve the uploaded gradient data's ability to resist reverse attacks. Since the target image tampering detection model used by the electronic device to perform image detection processing on the private image is a model obtained by processing the model parameters obtained by the service platform based on the gradient data of multiple electronic devices, the model combines multi-party data during training, so the model's recognition accuracy for private images is improved. Therefore, the recognition results of the model are then tampered with, thereby improving the accuracy of the tampering recognition results of the private image.

[0049] See Figure 2 , is a flow chart of an image tampering detection method provided in an embodiment of this specification. The execution subject of the method described in this embodiment is an electronic device, such as Figure 2 As shown, the method of the embodiment of this specification may include the following steps:

[0050] S202: Determine a first model gradient matrix corresponding to a local image tampering detection model, and perform local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix.

[0051] It is easy to understand that the local image tampering detection model refers to a model with image detection capabilities obtained based on a machine learning model. The image detection capability of the local image tampering detection model can be understood as an ability to identify tampered areas that may exist in an image. The tampered area in an image refers to an area obtained by modifying the original image area in the image through one or more image operations; image operations may include image synthesis, copy and paste, deletion, modification and other operations. Specifically, the image recognized by the local image tampering detection model of the embodiment of this specification may be a private image such as an image of a personal identification document, an image of a credential material with user identity information, etc.; the credential material may be a salary certificate material, an occupation certificate material, a medical certificate material and other credentials.

[0052] The first model gradient matrix refers to a matrix composed of the gradients of the model parameters of the local image tampering detection model. The gradients in the first model gradient matrix can be obtained by training the local image tampering detection model using training data. Model parameters are variables used to describe the data within the model. These variables are usually learned during model training to minimize the error between the model prediction and the real data. In electronic devices, the model parameters and model structure of the local image tampering detection model can be obtained from the service platform. Specifically, the model parameters can be weights, biases, or other adjustable variables. These parameters are continuously adjusted by iterating on the training data to enable the model to more accurately predict new data. For example, in a neural network, the model parameters are the weights and biases of each neuron. In linear regression, the model parameters are the regression coefficients and intercept. In a support vector machine, the model parameters are the decision boundary and support vector.

[0053] The second model gradient matrix is ​​used to instruct the service platform to determine the target model parameters based on the second model gradient matrix uploaded by at least one electronic device. The second model gradient matrix is ​​a gradient matrix obtained by performing differential privacy calculation on the local gradients in the first model gradient matrix.

[0054] In one or more embodiments of the present specification, the step of determining the first model gradient matrix corresponding to the local image tampering detection model may specifically be: performing model training processing on the local image tampering detection model based on the sample private images to obtain the first model gradient matrix. The sample private images may include first private images authorized by the user and collected within a historical time period, and may also include second private images obtained by performing data augmentation processing on the first private images authorized by the user. For a single electronic device, the number and types of first private images authorized by the user collected are limited. In this case, the second private image may be obtained by performing data augmentation processing such as splicing, deletion, and scaling on the first private image to achieve the effect of expanding the sample data. When the local image tampering detection model is trained using the sample private images, the local image tampering detection model may be trained multiple times using a loss function and a back-propagation algorithm to obtain a first model gradient matrix of the model parameters of the local image tampering detection model.

[0055] The step of performing local gradient differential privacy calculation based on the first model gradient matrix to obtain the second model gradient matrix can specifically be: determining a first gradient subset in the first model gradient matrix, performing noise processing on all gradients in the first gradient subset to obtain a second gradient subset, and updating the first model gradient matrix based on the second gradient subset to obtain the second model gradient matrix.

[0056] When determining the first gradient subset in the first model gradient matrix, the gradient of at least one specified position in the first model gradient matrix can be used as the gradient constituting the first gradient subset. The specified position can be determined according to the position of the specified parameter in the network structure, the network structure refers to the structure of the network adopted by the local image tampering detection model, and the specified parameter refers to the network parameter in the network adopted by the local image tampering detection model. The specified parameter can be obtained by querying the network parameter mapping relationship. The correspondence between the reference model and the reference parameter can be stored in the network parameter mapping relationship. When the local image tampering detection model is used as the reference model, its corresponding reference parameter is the specified parameter. The correspondence between the reference model and the reference parameter stored in the network parameter mapping relationship can be configured by a technician based on prior experience.

[0057] When performing noise processing on all gradients in the first gradient subset to obtain the second gradient subset, random noise such as Gaussian noise may be used to perform noise processing on all gradients in the first gradient subset to obtain the second gradient subset.

[0058] When the first model gradient matrix is ​​updated based on the second gradient subset to obtain the second model gradient matrix, the corresponding position of each second gradient in the second gradient subset in the first model gradient matrix can be first determined, and then the first gradient at each corresponding position in the first model gradient matrix is ​​replaced with the second gradient corresponding to the corresponding position, so as to obtain the second model gradient matrix.

[0059] S204: Upload the second model gradient matrix to the service platform.

[0060] It is easy to understand that the service platform refers to the server that forms the federated learning architecture with the electronic devices. Figure 1 The system architecture shown, i.e., the federated learning architecture, includes a service platform and multiple electronic devices. In the federated learning architecture, the electronic device sends the second model gradient matrix to the service platform, the service platform aggregates the second model gradient matrix sent by each electronic device, and the service platform then sends the model parameters obtained by the aggregation processing to each electronic device. During the communication process between the service platform and each electronic device, the image data used by each electronic device to calculate the second model gradient matrix will not flow out to the outside of the electronic device, that is, the image data used by each electronic device is only inside each electronic device. Therefore, the federated learning architecture is used to achieve joint training of the model while ensuring that the data does not go out of the domain.

[0061] In one or more embodiments of the present specification, executing step S204 may specifically include: encrypting the second model gradient matrix using a preset encryption key to obtain encrypted gradient data, and uploading the encrypted gradient data to the service platform. The preset encryption key may be an encryption key pre-agreed between the electronic device and the service platform in a trusted environment; the preset encryption key may also be the public key of the service platform. Since the electronic device uploads the encrypted second model gradient matrix, the service platform may decrypt the encrypted gradient data using a preset decryption key corresponding to the preset encryption key to obtain the second model gradient matrix. When the preset encryption key is an encryption key pre-agreed between the electronic device and the service platform in a trusted environment, the preset decryption key used by the service platform is the decryption key corresponding to the preset encryption key. When the preset encryption key is the public key of the service platform, the preset decryption key used by the service platform may be the private key of the service platform. In this manner, by sending the encrypted second model gradient matrix to the service platform, the second model gradient matrix is ​​prevented from being leaked and thus insecure.

[0062] Optionally, in addition to sending the encrypted second model gradient matrix to the service platform, the encrypted second model gradient matrix can also be sent to the service platform when the current time reaches a preset sending time. The preset sending time can be a sending time agreed upon with the service platform, such as a preset sending time of 8:00 PM or 10:00 PM daily. In this way, by sending the second model gradient matrix to the service platform at a specific time, the efficiency of the joint training of the model is guaranteed.

[0063] S206: Receive target model parameters sent by the service platform, and update the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model.

[0064] It is easy to understand that the target model parameters refer to the latest model parameters of the local image tampering detection model obtained by the service platform through aggregation processing based on the second model gradient matrix sent by each electronic device.

[0065] In one or more embodiments of the present specification, receiving the target model parameters sent by the service platform may be that the service platform actively sends the target model parameters to the electronic device so that the electronic device can receive the target model parameters; or the electronic device sends a parameter query request to the service platform, and the service platform sends the target model parameters to the electronic device based on the parameter query request, so that the electronic device can receive the target model parameters.

[0066] Specifically, the electronic device receives the target model parameters sent by the service platform, which may include: the electronic device receives encrypted parameter data sent by the service platform, the encrypted parameter data being encrypted data obtained by the service platform encrypting the target model parameters, and the electronic device decrypting the encrypted parameter data to obtain the target model parameters. When the service platform encrypts the target model parameters to obtain the encrypted parameter data, the encryption key used may be a preset encryption key agreed upon in advance by the electronic device and the service platform in a trusted environment, or may be the public key of the electronic device. Accordingly, the service platform encrypts the target model parameters, and the electronic device needs to decrypt the encrypted parameter data. When the encryption key is the above-mentioned preset encryption key, the decryption key used by the electronic device is the preset decryption key corresponding to the preset encryption key. When the encryption key is the public key of the electronic device, the decryption key used by the electronic device is the private key of the electronic device. In this way, by receiving the data obtained by encrypting the target model parameters sent by the service platform, the target model parameters are avoided from being leaked and causing insecurity.

[0067] The local image tampering detection model is updated based on the target model parameters to obtain the target image tampering detection model. Specifically, the model parameters of the local image tampering detection model are updated to the target model parameters to obtain the target image tampering detection model.

[0068] S208 , performing image detection processing on the target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and performing tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0069] For easy understanding, the target private image refers to a private image authorized by the user to be detected. This can include images of personal identification documents, images of credential materials containing the user's identity information, and other private images. Credential materials can include documents such as salary certificates, occupation certificates, and medical certificates.

[0070] A target heat map may refer to image data that uses color coding to display the distribution of tampered areas. Specifically, in the target heat map, different colors may be used to display tampered areas with different probabilities, where the probability refers to the probability of tampering in the tampered area detected by the target image tampering detection model.

[0071] The tampering identification result may include a result of whether the target private image is a tampered image.

[0072] In one or more embodiments of the present specification, the target privacy image is input into the target image tampering detection model, and the target thermal detection map is output. Furthermore, a device-specific classifier can be used to perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target privacy image. Among them, the device-specific classifier can be determined based on the data characteristics of the privacy images owned by different electronic devices, and the device-specific classifiers in different electronic devices may not be exactly the same. For example, different electronic devices can use a classifier with the same network structure, and different electronic devices can configure different discrimination thresholds in the classifier according to the data characteristics of their own privacy images, so as to achieve targeted discrimination of the results identified by the public model by different electronic devices. The tampering identification result can be that the target privacy image is a tampered image, and the tampering identification result can also be that the target privacy image is not a tampered image. In this way, by configuring different device-specific classifiers for different electronic devices, the tampering identification results of the privacy images to be detected by each electronic device can be accurately determined.

[0073] Optionally, the tampering identification result may include whether the target private image is a tampered image, and may also include the tampered areas present in the target private image. In this case, the step of performing tampering identification processing on the target thermal detection image to obtain a tampering identification result corresponding to the target private image may specifically include: performing verification processing on the reference tampering areas in the target thermal detection image based on a preset tampering verification method to obtain a true tampering result corresponding to each reference tampering area; and determining a tampering identification result corresponding to the target private image based on the true tampering results.

[0074] The preset tampering verification method may be a tampering verification method determined by the electronic device based on data features of the private image it possesses, and different electronic devices may have different preset tampering verification methods.

[0075] Determining the tampering identification result corresponding to the target private image based on the actual tampering results can be understood as follows: when at least one actual tampering result is a first tampering result indicating the presence of tampering, the reference tampering area corresponding to the first tampering result is used as the tampering area in the target private image, generating a tampering identification result that includes the tampered area; when all actual tampering results are second tampering results indicating the absence of tampering, a tampering identification result is generated indicating that the target private image is not a tampered image. In this way, different electronic devices use their own preset tampering verification methods to verify the tampering areas that may exist in the target thermal detection map, accurately determining the tampering identification results for the private images that each electronic device needs to detect.

[0076] The embodiments of this specification adopt the above-mentioned method steps. The electronic device pre-processes the gradient data to be uploaded to the service platform to improve the uploaded gradient data's resistance to reverse attacks. Since the target image tampering detection model used by the electronic device to perform image detection processing on private images is a model obtained by processing the gradient data of multiple electronic devices based on the service platform, and the model incorporates multi-party data during training, the model's recognition accuracy for private images is improved. Therefore, the recognition results of the model are then subjected to tampering identification processing, thereby improving the accuracy of the tampering identification results of private images.

[0077] See Figure 3 , is a flow chart of an image tampering detection method provided in an embodiment of this specification. The execution subject of the method described in this embodiment is an electronic device, such as Figure 3 As shown, the method of the embodiment of this specification may include the following steps:

[0078] S302: Determine a first model gradient matrix corresponding to a local image tampering detection model.

[0079] In one or more embodiments of the present specification, executing S302 may specifically include: obtaining a historical private image; and performing model training processing on a local image tampering detection model based on the historical private image to obtain a first model gradient matrix.

[0080] Historical private images can include images of identity information or credential materials containing the user's identity information, collected with user authorization in financial anti-fraud detection scenarios. Identity information can refer to personal identification documents. Credential materials can include documents such as salary certificates, employment certificates, and medical certificates.

[0081] Obtaining historical private images may include determining a current training transaction scenario and obtaining historical private images corresponding to the current training transaction scenario from a preset database. The private images stored in the preset database may be private images corresponding to multiple training transaction scenarios. The private images corresponding to each training transaction scenario may be private images uploaded by the user terminal within a historical time period, which may include the past 24 hours, the past 7 days, or other time periods.

[0082] The current training transaction scenario refers to the transaction scenario of this joint training agreed upon by each electronic device and the service platform. The scenario involved in the embodiment of this specification is a financial anti-fraud identification scenario. The financial anti-fraud identification scenario can specifically include a variety of different transaction scenarios, such as medical certificate identification scenarios, loan certificate identification scenarios, etc. The current training transaction scenario can be any scenario under the financial anti-fraud identification scenario. In the process of each electronic device and the service platform adopting the federated learning architecture to jointly train the model, the model can be jointly trained by scenario. For example, the model training of a transaction scenario can be completed when the number of joint training reaches the preset rounds. The embodiment of this specification obtains privacy images through the current training transaction scenario, which ensures the consistency of the data content used by each electronic device when jointly training the model, and is conducive to improving the recognition accuracy of the model obtained by subsequent joint modeling.

[0083] The local image tampering detection model is trained based on the historical private images to obtain a first model gradient matrix. This can be understood as performing data augmentation on the historical private images to obtain a reference private image, annotating the reference private image to obtain training data, and using the training data to train the local image tampering detection model to obtain the first model gradient matrix.

[0084] Specifically, data augmentation processing is performed on historical private images to obtain a reference private image. This can be done by processing the historical private images using a data augmentation method. Data augmentation methods may include, but are not limited to, image copying and pasting, image cutting and splicing, image generation, and image resampling and scaling. Image copying and pasting refers to copying and pasting data from a certain location in a historical private image to another location in the same historical private image. Image cutting and splicing refers to cutting out portions of images from multiple historical private images and splicing the resulting portions. Image generation refers to using digital image processing techniques to replace data in a historical private image with similar data. Alternatively, a historical private image may be input into a generative model to obtain a reference private image, where the data at a certain location in the reference private image is similar to the initial data at that location in the historical private image. Image resampling and scaling refers to adjusting the clarity, scale, or visual quality of the historical private image by changing its resolution, size, or sampling rate. The embodiments of this specification increase the number of model samples through the aforementioned data augmentation methods, thereby improving the model's generalization capabilities.

[0085] Specifically, the training data is obtained by annotating the reference private image, which may include annotating the tampered area in the reference private image, annotating the tampered label of the reference private image, and annotating the private area in the reference private image, so as to obtain training data containing annotated information such as the reference private image, the tampered area corresponding to the reference private image, the tampered label corresponding to the reference private image, and the private area corresponding to the reference private image. The tampered label annotated with the reference private image may be represented by a preset character, such as 1 and 0. When the tampered label is 1, it indicates that a tampered area exists in the reference private image; when the tampered label is 0, it indicates that no tampered area exists in the reference private image.

[0086] Specifically, the training data is used to perform model training on the local image tampering detection model to obtain a first model gradient matrix. This can be understood as inputting the training data as batch data into the local image tampering detection model for forward propagation calculation to obtain the output of the local image tampering detection model. Then, based on the output of the local image tampering detection model, the error back propagation algorithm is used to calculate the gradients of the model parameters of the local image tampering detection model. These gradients constitute a gradient matrix to obtain the first model gradient matrix.

[0087] S304: Determine an initial local gradient subset in the first model gradient matrix.

[0088] In one or more embodiments of the present specification, executing step S304 may specifically include: A2, determining model features for local image tampering detection; A4, determining candidate network parameters corresponding to the local image tampering detection model based on the model features; A6, determining an initial local gradient subset corresponding to the candidate network parameters in the first model gradient matrix.

[0089] The model features refer to the features of the network structure used by the model. For example, the model features may include the features of the network layers in the network structure, such as the number of network layers, the functions of the network layers, etc.

[0090] The candidate network parameters refer to the network parameters selected from the network structure adopted by the local image tampering detection model according to the model characteristics.

[0091] The initial local gradient subset refers to a gradient set consisting of partial gradients of the first model gradient matrix.

[0092] Specifically, when executing step A2, the model features of the local image tampering detection model may be obtained from the model configuration file. The model configuration file may store model features, model parameters and other model configuration information.

[0093] Specifically, when executing step A4, it can be: a2, determining the candidate network layer in the local image tampering detection model based on the model features; a4, determining the sample mask image corresponding to the sample privacy image, and inputting the sample mask image into the local image tampering detection model to obtain the learnable network parameters in the candidate network layer; a6, determining the candidate network parameters corresponding to the local image tampering detection model based on the learnable network parameters.

[0094] When executing step a2, a network layer mapping relationship is obtained. The network layer mapping relationship is used to store the correspondence between the reference model features and the reference candidate network layer. The target candidate network layer corresponding to the model features is searched in the network layer mapping relationship. The target candidate network layer is the candidate network layer of the local image tampering detection model in the embodiment of this description. The network layer mapping relationship can be generated by the model network expert configuration, or it can be generated by the technician based on prior experience. For example, if the network used by the local image tampering detection model is CATNet, the reference candidate network layer corresponding to the model features of CATNet in the network layer mapping relationship can be the first convolution layer in the RGB branch, the last convolution layer in the RGB branch, and the first convolution layer in the DCT branch. When the network layer mapping relationship is queried based on the model features of CATNet, the candidate network layers can be obtained as the first convolution layer in the RGB branch, the last convolution layer in the RGB branch, and the first convolution layer in the DCT branch.

[0095] When executing step a4, determining the sample mask image corresponding to the sample private image can be understood as selecting a preset number of sample private images from the full set of private images of the electronic device. The preset number can be set according to actual needs, and then at least one privacy area in the sample private image is used as a mask area. The mask area is masked to obtain a sample mask image containing only the mask area. Inputting the sample mask image into the local image tampering detection model to obtain the learnable network parameters in the candidate network layer can be understood as inputting the sample mask image into the local image tampering detection model to obtain the network parameters in the candidate network layer, and determining the learnable network parameters from these network parameters. The learnable network parameters here include the names of the learnable network parameters and the parameter values ​​of the learnable network parameters. Learnable network parameters refer to parameters whose parameter values ​​are learned during the training process, that is, parameters whose parameter values ​​are variable values. The parameter values ​​of the learnable parameters are variable values. It can be understood that their parameter values ​​usually start from a set of random values ​​and then are updated in an iterative manner as the network learns. For example, the learnable parameters may be parameters of the convolution layer, including the weights of the convolution kernel, the offsets of each channel, and the like.

[0096] During step a6, candidate network parameters corresponding to the local image tampering detection model are determined based on the learnable network parameters. This can be understood as selecting the learnable network parameters with the largest absolute value as the candidate network parameters corresponding to the local image tampering detection model. This is because the candidate network parameters have a strong response output to input in the private region of the private image and can be used to locate the gradient corresponding to the private region.

[0097] Specifically, when executing step A4, it can be: b2, determining at least one gradient corresponding to each candidate network parameter in the first model gradient matrix; b4, filtering at least one gradient to obtain a target gradient corresponding to the candidate network parameter; b6, determining an initial local gradient subset corresponding to the candidate network parameter based on the target gradient.

[0098] When executing step b2, after obtaining the candidate network parameters, it is easy to determine the position information of the candidate network parameters in the network adopted by the local image tampering detection model. Since the gradients at different positions in the first model gradient matrix correspond to the parameters at different positions in the network, at least one gradient corresponding to each candidate network parameter can be selected from the first model gradient matrix according to the position information of the candidate network parameters in the network.

[0099] During step b4, for each candidate network parameter's at least one gradient, a gradient threshold corresponding to the candidate network parameter is obtained from the gradient filtering mapping table. The at least one gradient is filtered using the gradient threshold, i.e., gradients smaller than the gradient threshold are filtered out from the at least one gradient to obtain a target gradient corresponding to the candidate network parameter. The gradient filtering mapping table is preconfigured with gradient thresholds corresponding to different candidate network parameters. The gradient threshold is used to filter out smaller gradients from the gradients corresponding to the candidate network parameters.

[0100] When executing step b6, for each candidate network parameter, which corresponds to a target gradient, the target gradients corresponding to all candidate network parameters are stored in a gradient set to obtain an initial local gradient subset.

[0101] S306 , performing noise processing on the initial local gradient subset to obtain a target local gradient subset, and updating the first model gradient matrix based on the target local gradient subset to obtain a second model gradient matrix.

[0102] In one or more embodiments of the present specification, performing noise processing on the initial local gradient subset to obtain a target local gradient subset may specifically include: B2, performing gradient clipping processing on the initial local gradient subset to obtain a reference local gradient subset; and B4, performing noise processing on the reference gradient subset to obtain a target local gradient subset.

[0103] When executing step B2, the initial local gradient subset can be gradient clipped by the gradient boundary to obtain a reference local gradient subset. The gradient clipping formula can be:

[0104]

[0105] Where C represents the gradient boundary, represents the initial local gradient subset, represents the reference local gradient subset,

[0106] Represents the second normal form. The value of C can be set according to the actual application. The gradient calculated by the above gradient clipping formula will not be greater than C, that is, the gradient in the reference local gradient subset will not be greater than C. The embodiments of this specification ensure the stability of the model during the training process by clipping the gradient subset.

[0107] When executing step B4, each gradient in the reference gradient subset is subjected to noise processing, and all the noisy gradients constitute the target local gradient subset. For example, the noise addition formula can be:

[0108]

[0109] Where m is the number of sample private images selected in step a4, is the variance of Gaussian noise, C is the above gradient boundary, g represents the reference gradient subset, Represents the target local gradient subset. Set the privacy budget and the relaxation term Afterwards, you can

[0110] Know when When the local gradient subset is located by the privacy region achieve Differential privacy protection level

[0111] No. Privacy Budget It is a hyperparameter that controls the effect of differential privacy protection. The smaller its value is, the better the privacy protection effect is, but the greater the noise introduced is.

[0112] When performing an update process on the first model gradient matrix based on the target local gradient subset to obtain the second model gradient matrix, specifically, the corresponding positions of each gradient in the target local gradient subset in the first model gradient matrix are determined, and the gradients at these corresponding positions in the first model gradient matrix are replaced by the gradients corresponding to the corresponding positions in the target local gradient subset to obtain the second model gradient matrix. For example, a gradient in the target local gradient subset is g1, and the corresponding position of g1 in the first model gradient matrix is ​​the 3rd row and 4th column. The gradient of the 3rd row and 4th column in the first model gradient matrix is ​​G1, and G1 in the first model gradient matrix is ​​replaced by g1. By analogy, the gradients at the corresponding positions in the first model gradient matrix are replaced by the corresponding gradients in the target local gradient subset to obtain the second model gradient matrix.

[0113] S308: Upload the second model gradient matrix to the service platform.

[0114] S310: Receive target model parameters sent by the service platform, and update the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model.

[0115] Specifically, the implementation of S308 and S310 can be found in Figure 2 The explanation of S204 and S206 in the illustrated embodiment will not be repeated here.

[0116] S312: Perform image detection processing on the target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image.

[0117] The definition of target privacy image and target thermal detection map can be found in Figure 2 The explanation of S208 in the illustrated embodiment will not be repeated here.

[0118] In one or more embodiments of this specification, the target privacy image is input into the target image tampering detection model to obtain the target thermal detection map output by the target image tampering detection model. For example, see Figure 4 Schematic diagram of the target thermal detection map shown. Figure 4 The “red area”, “orange area” and “yellow area” marked in the figure are the tampered areas detected by the target image tampering detection model as possible tampering. The tampered areas distinguished by different colors represent different tampering probabilities. For example, the tampering probability of the “red area” is greater than that of the “orange area”, and the tampering probability of the “orange area” is greater than that of the “yellow area”.

[0119] S314: Obtain a device-specific classifier, and perform tampering identification processing on the target thermal detection image based on the device-specific classifier to obtain a tampering identification result corresponding to the target private image.

[0120] It is easy to understand that a device-specific classifier refers to a classifier that is set specifically for the data characteristics of private images stored in an electronic device. The device-specific classifiers of different electronic devices are not all the same.

[0121] Because the data characteristics of private images held by different electronic devices are not entirely identical, and different electronic devices have different preferences for the recall and accuracy of local image tampering detection models, different electronic devices can set device-specific classifiers with different discrimination thresholds to adapt to the tampering identification needs of different electronic devices for private images. For example, the target thermal detection map indicates that a certain character may have been tampered with, but for a certain electronic device, tampering is considered to have occurred only when consecutive characters are tampered with, and tampering of a single character is not considered to have occurred. Therefore, different electronic devices need to be configured with classifiers that adapt to their own identification needs to determine the final tampering identification results.

[0122] In one or more embodiments of this specification, a device-specific classifier may be selected from a model library locally configured on the electronic device to obtain a device-specific classifier. Furthermore, the target thermal detection image is input into the device-specific classifier to obtain a tampering identification result. The tampering identification result may be a binary classification result, i.e., the tampering identification result may include two results: tampering-positive and tamper-negative. Tampering-positive indicates that the target private image has been tampered with, and tampering-negative indicates that the target private image has not been tampered with.

[0123] Optionally, the device-specific classifier of the embodiment of this specification may be composed of a 3-layer fully connected network, and the classifier uses a sigmoid activation function to obtain the final classification output. The electronic device trains the classifier using a binary cross entropy loss and an error back propagation algorithm. The distribution range of the model output value using the sigmoid activation function is 0-1, and the model output value can be used to indicate the probability that the image has been tampered with. Different electronic devices can select an appropriate discrimination threshold based on the transaction status and the importance of data verification to obtain tampering identification results.

[0124] For example, when the discrimination threshold of the device-specific classifier is set to 0.7, the target private image with a model output value equal to or greater than 0.7 will be identified as a tampered image. Figure 4 As shown in the schematic diagram of the target thermal detection map, if the model output values ​​of the device classifier for the "red area", "orange area" and "yellow area" in the embodiment of this specification are 0.81, 0.67 and 0.32 respectively, then for the device classifier, the tampering identification result of the "red area" is tampered, the tampering identification result of the "orange area" is not tampered, and the tampering identification result of the "yellow area" is not tampered.

[0125] In the embodiment of this specification, the larger the discrimination threshold of the device-specific classifier is set, the higher the recognition accuracy of the device-specific classifier is and the lower the recall rate is.

[0126] In an embodiment of the present specification, the electronic device first determines the first model gradient matrix of the local image tampering detection model locally, and then performs noise processing on the gradients in the first model gradient matrix that are significantly perceived in the privacy region to obtain a second model gradient matrix. In this way, by adding noise perturbations to the gradient matrix uploaded to the service platform for joint training, external attackers are prevented from stealing the uploaded gradients and conducting reverse attacks, thereby improving the security of private data. In addition, upon receiving the target model parameters sent by the service platform, the model is updated to obtain the latest target image tampering detection model. The target image tampering detection model also performs secondary tampering identification processing on the target privacy image using a device-specific classifier for the electronic device. While adapting to the tampering detection needs of different electronic devices, it also improves the accuracy of the tampering identification results of the privacy image by each electronic device.

[0127] See Figure 5 , is a flow chart of an image tampering detection method provided in an embodiment of this specification. The execution subject of the method described in this embodiment is a service platform, such as Figure 5 As shown, the method of the embodiment of this specification may include the following steps:

[0128] S502: Receive a second model gradient matrix uploaded by at least one electronic device.

[0129] The second model gradient matrix is ​​obtained by performing a local gradient differential privacy calculation based on the first model gradient matrix corresponding to the local image tampering detection model. The steps for each electronic device to calculate the second model gradient matrix from the first model gradient matrix can be found in the following. Figure 2 The description of S202 in the illustrated embodiment will not be repeated here.

[0130] In some embodiments, executing step S502 may specifically include: receiving encrypted gradient data uploaded by at least one electronic device, decrypting each encrypted gradient data using a preset decryption key to obtain a second model gradient matrix corresponding to each electronic device. The preset decryption keys used to decrypt the encrypted gradient data sent by different electronic devices may be different. The preset decryption key may be a decryption key pre-agreed between the electronic device and the service platform in a trusted environment; the preset decryption key may also be the public key of the electronic device. The service platform decrypts the encrypted gradient data, and accordingly, the electronic device encrypts the second model gradient matrix. When the preset decryption key is the decryption key pre-agreed between the electronic device and the service platform in the trusted environment, the preset encryption key used by the electronic platform is the encryption key corresponding to the preset decryption key. When the preset decryption key is the public key of the electronic device, the encryption key used by the electronic device may be the private key of the electronic device. In this manner, by sending the encrypted second model gradient matrix to the service platform, the second model gradient matrix is ​​prevented from being leaked and potentially insecure.

[0131] Optionally, before executing S502, the service platform may share the local image tampering detection model with each electronic device. Specifically, the service platform may send the model architecture and initial model parameters of the local image tampering detection model to the electronic device. The local image tampering detection model may be a model obtained by the service platform through model training based on an open source privacy data training set.

[0132] S504: Determine target model parameters based on each second model gradient matrix.

[0133] It is easy to understand that the target model parameters refer to the latest model parameters of the local image tampering detection model obtained by the service platform through aggregation processing based on the second model gradient matrix sent by each electronic device.

[0134] In one or more embodiments of the present specification, the service platform may perform gradient aggregation processing on each second model gradient matrix of the same batch to obtain an aggregated gradient, and update the model parameters of the local image tampering detection model based on the aggregated gradient to obtain the target model parameters. The second model gradient matrices of the same batch may refer to the second model gradient matrices received within the same time period. The service platform and each electronic device may agree in advance on the time period for uploading the second model gradient matrix in each round. In the process of calculating the target model parameters in each round, the second model gradient matrix within the time period can be used for calculation to obtain the target model parameters.

[0135] Optionally, the service platform may use a federated average algorithm (FedAverage, abbreviated as FedAvg) to obtain the aggregated gradient.

[0136] S506: Send the target model parameters to each electronic device.

[0137] The target model parameters are used to instruct the electronic device to update the local image tampering detection model based on the target model parameters to obtain the target image tampering detection model, perform image detection processing on the target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image. Each electronic device updates the local image tampering detection model based on the target model parameters to obtain the target image tampering detection model, and performs subsequent processing based on the target image tampering detection model. For details, see [ tampering detection results ]. Figure 2 The description of S206-S208 in the illustrated embodiment will not be repeated here.

[0138] In some embodiments, executing step S506 may specifically include: encrypting the target model parameters using a preset encryption key to obtain encrypted parameter data, and sending the encrypted parameter data to each electronic device. The encrypted parameter data sent to different electronic devices may use different encryption keys. The preset encryption key used by the service platform may be an encryption key pre-agreed between the electronic device and the service platform in a trusted environment; the preset encryption key may also be the public key of the electronic device. The service platform encrypts the target model parameters, and the electronic device is then required to decrypt the encrypted parameter data to obtain the target model parameters. When the preset encryption key used by the service platform is an encryption key pre-agreed between the electronic device and the service platform in a trusted environment, the decryption key used by the electronic device may be a preset decryption key corresponding to the preset encryption key; when the preset encryption key used by the service platform is the public key of the electronic device, the decryption key used by the electronic device may be the private key of the electronic device. In this way, the service platform sends the encrypted target model parameters to the electronic device, preventing the target model parameters from being leaked and potentially insecure.

[0139] It should be noted that the service platform can calculate multiple rounds of target model parameters using multiple batches of the second model gradient matrix to continuously update the local image tampering detection model shared with each electronic device until the local image tampering detection model converges. Each time the service platform updates the local image tampering detection model, it can send the latest model parameters of the local image tampering detection model, i.e., the target model parameters, to each electronic device.

[0140] In an embodiment of the present specification, the service platform receives a second model gradient matrix uploaded by each electronic device. The second model gradient matrix is ​​a gradient matrix obtained by the electronic device through local differential privacy calculation, and then determines the target model parameters based on the second model gradient matrix, thereby achieving the effect of integrating and training the model using multi-party data, thereby improving the recognition ability of the model. Since the privacy level of the second model gradient matrix uploaded by the electronic device is higher, it effectively resists the reverse attack of external attackers and ensures the security of the data during the joint training model. The service platform then sends the target model parameters to each electronic device, allowing the electronic device to update the local image tampering detection model shared by the service platform according to the target model parameters to obtain a target image tampering detection model with better recognition ability, thereby improving the detection effect of the electronic device on the private image using the target image tampering detection model, and thereby improving the accuracy of the recognition result obtained by the electronic device through secondary recognition of the detection result using the target image tampering detection model.

[0141] See Figure 6 , is a flow chart of an image tampering detection method provided in an embodiment of this specification. The execution subject of the method described in this embodiment is a service platform, such as Figure 6 As shown, the method of the embodiment of this specification may include the following steps:

[0142] S602: Receive a second model gradient matrix uploaded by at least one electronic device.

[0143] The second model gradient matrix is ​​obtained by performing a local gradient differential privacy calculation based on the first model gradient matrix corresponding to the local image tampering detection model. The steps for each electronic device to calculate the second model gradient matrix from the first model gradient matrix can be found in the following. Figure 3 The description of S302-S306 in the illustrated embodiment will not be repeated here.

[0144] S604: Calculate target gradients based on the second model gradient matrices.

[0145] In some embodiments, when executing S604, it may specifically include: C2, obtaining the amount of matrix training data uploaded by each electronic device, where the matrix training data amount is the training data consumption value when generating the second model gradient matrix in the model training phase; C4, determining the total amount of data based on all matrix training data amounts; C6, determining the ratio of the matrix training data amount to the total amount of data, and using the ratio as the gradient weight corresponding to the electronic device; C8, performing weighted summation based on the gradient weight and the second model gradient matrix to obtain the target gradient.

[0146] Among them, the matrix training data volume is the training data consumption value of the electronic device when obtaining the second model gradient matrix. The training data consumption value refers to the number of sample privacy images used by the electronic device when obtaining the second model gradient matrix. Since the second model gradient matrix is ​​derived from the first model gradient matrix, the training data consumption value can also be understood as the number of sample privacy images used by the electronic device when training to obtain the first model gradient matrix.

[0147] When executing step C4, the sum of the training data amounts of all matrices may be obtained to obtain the total data amount.

[0148] When executing step C6, for each matrix training data amount, the ratio of the amount to the total amount of data can be calculated to obtain the gradient weight corresponding to the electronic device.

[0149] When executing step C8, for each electronic device, the product of its corresponding gradient weight and the uploaded second model gradient matrix can be calculated, and then these products are summed to obtain the target gradient.

[0150] In the embodiment of the present application, the gradient mean is calculated by weighted summation to comprehensively evaluate each second model gradient matrix, thereby ensuring the accuracy of the gradient used to update the model parameters.

[0151] S606: Obtain a learning rate corresponding to the local image tampering detection model, and calculate the product of the learning rate and the target gradient.

[0152] S608: Obtain initial model parameters corresponding to the local image tampering model, calculate the sum of the product and the initial model parameters, and use the sum as the target model parameters.

[0153] It is easy to understand that the initial model parameters refer to the model parameters obtained by the service platform and electronic devices in the previous round of aggregate training.

[0154] In the embodiment of this specification, the formula for calculating the target model parameters may be:

[0155]

[0156] in, represents the initial model parameters, represents the target model parameters, represents the learning rate, K represents the total number of electronic devices, represents the amount of matrix training data uploaded by the kth electronic device, n represents the total amount of data, represents the second model gradient matrix of the i-th electronic device.

[0157] S610: Send the target model parameters to each electronic device.

[0158] The target model parameters are used to instruct the electronic device to update the local image tampering detection model based on the target model parameters to obtain the target image tampering detection model, perform image detection processing on the target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image. Each electronic device updates the local image tampering detection model based on the target model parameters to obtain the target image tampering detection model, and performs subsequent processing based on the target image tampering detection model. For details, see [ tampering detection results ]. Figure 3 The description of S310 - S314 in the illustrated embodiment will not be repeated here.

[0159] S612: Determine the current training round of the local image tampering detection model.

[0160] S614: If the current training round is greater than or equal to the preset round, a training end instruction is sent to each electronic device.

[0161] S612 and S614 are explained below.

[0162] It is easy to understand that the current training round refers to the number of rounds that the local image tampering detection model has been trained.

[0163] The preset rounds may refer to a preset maximum number of iteration rounds of the model.

[0164] The training end instruction is used to instruct each electronic device to stop uploading the second model gradient matrix.

[0165] In one or more embodiments of the present description, determining the current training round of the local image tampering detection model can be understood as determining the current training round by the current number of calculations of the target model parameters, and the current training round can also be determined by the number of receptions of the second model gradient matrix corresponding to each electronic device. Further, it can be determined whether the current training round is greater than or equal to the preset round. When the current training round is greater than or equal to the preset round, it means that the number of iterations of the model parameters has reached the maximum iteration round, and the service platform and the electronic device can stop the joint training of the local image tampering detection model, then the service platform can send a training end instruction to each electronic device to inform the electronic device to stop the joint training model through the training end instruction. When the current training round is less than the preset round, it means that the number of iterations of the model parameters has not reached the maximum iteration round, and the service platform and the electronic device still need to continue to jointly train the model, so the service platform does not need to send a training end instruction to the electronic device.

[0166] In an embodiment of the present specification, the service platform receives the second model gradient matrix uploaded by each electronic device, determines the target gradient based on each second model gradient matrix, and then determines the target model parameters based on the target gradient. While achieving the effect of integrating and training the model using multi-party data, it also improves the accuracy of the gradient by calculating the mean of the target model gradient. Then, the more accurate gradient is used to calculate the more accurate target model parameters, and the model is updated using the more accurate target model parameters, thereby further improving the model's recognition ability. Because noise is added to the second model gradient matrix uploaded by each electronic device, the second model gradient matrix has good privacy, and the target model parameters calculated by the service platform based on the second model gradient matrix also have good privacy. The service platform then sends the more accurate target model parameters to each electronic device, allowing the electronic device to update the local image tampering detection model shared by the service platform based on the more accurate target model parameters to obtain a target image tampering detection model with better recognition ability. This improves the electronic device's detection effect on private images using the target image tampering detection model, and further improves the accuracy of the tampering recognition results obtained by the electronic device through secondary recognition of the detection results using the target image tampering detection model.

[0167] The following will be combined Figure 7 , the image tampering detection device provided by the embodiment of the present application is introduced in detail. It should be noted that, Figure 7 The image tampering detection device shown is used to implement the present application Figure 2 and Figure 3 For the convenience of explanation, only the part related to the embodiment of this specification is shown. For the specific technical details not disclosed, please refer to the application. Figure 2 and Figure 3 The embodiment shown.

[0168] See Figure 7 , which shows a schematic diagram of the structure of an image tampering detection device according to an embodiment of the present specification. The image tampering detection device 1 can be implemented as all or part of the device through software, hardware, or a combination of both. According to some embodiments, the image tampering detection device 1 includes a gradient processing module 11, a gradient sending module 12, a data receiving module 13, and an image processing module 14, which are specifically used to:

[0169] A gradient processing module 11 is configured to determine a first model gradient matrix corresponding to a local image tampering detection model, and perform local gradient differential privacy calculation based on the first model gradient matrix to obtain a second model gradient matrix;

[0170] a gradient sending module 12, configured to upload the second model gradient matrix to a service platform, wherein the second model gradient matrix is ​​used to instruct the service platform to determine target model parameters based on the second model gradient matrix uploaded by the at least one electronic device;

[0171] A data receiving module 13 is configured to receive the target model parameters sent by the service platform, and update the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model;

[0172] The image processing module 14 is configured to perform image detection processing on the target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0173] Optionally, the gradient processing module 11 includes:

[0174] a first processing unit, configured to determine an initial local gradient subset in the first model gradient matrix;

[0175] The second processing unit is configured to perform noise addition processing on the initial local gradient subset to obtain a target local gradient subset, and update the first model gradient matrix based on the target local gradient subset to obtain a second model gradient matrix.

[0176] Optionally, the first processing unit includes:

[0177] a parameter determination unit, configured to determine model features of the local image tampering detection, and determine candidate network parameters corresponding to the local image tampering detection model based on the model features;

[0178] A gradient determination unit is used to determine an initial local gradient subset corresponding to the candidate network parameters in the first model gradient matrix.

[0179] Optionally, the parameter determination unit is specifically configured to:

[0180] Determining a candidate network layer in the local image tampering detection model based on the model features;

[0181] Determine a sample mask image corresponding to the sample private image, and input the sample mask image into the local image tampering detection model to obtain learnable network parameters in the candidate network layer;

[0182] Determine candidate network parameters corresponding to the local image tampering detection model based on the learnable network parameters.

[0183] Optionally, the gradient determination unit is specifically configured to:

[0184] Determining at least one gradient corresponding to each of the candidate network parameters in the first model gradient matrix;

[0185] Filtering the at least one gradient to obtain a target gradient corresponding to the candidate network parameter;

[0186] An initial local gradient subset corresponding to the candidate network parameters is determined based on the target gradient.

[0187] Optionally, the second processing unit is specifically configured to:

[0188] Performing gradient clipping on the initial local gradient subset to obtain a reference local gradient subset;

[0189] Noise processing is performed on the reference gradient subset to obtain a target local gradient subset.

[0190] Optionally, the gradient processing module 11 is specifically configured to: obtain a historical private image;

[0191] A local image tampering detection model is trained based on the historical privacy image to obtain a first model gradient matrix.

[0192] Optionally, the image processing module 14 is specifically configured to:

[0193] Obtain a device-specific classifier, and perform tampering identification processing on the target thermal detection image based on the device-specific classifier to obtain a tampering identification result corresponding to the target privacy image; wherein the device-specific classifiers of different electronic devices are not all the same.

[0194] The following will be combined Figure 8 , the image tampering detection device provided by the embodiment of the present application is introduced in detail. It should be noted that, Figure 8 The image tampering detection device shown is used to implement the present application Figure 5 and Figure 6 For the convenience of explanation, only the part related to the embodiment of this specification is shown. For the specific technical details not disclosed, please refer to the application. Figure 5 and Figure 6 The embodiment shown.

[0195] See Figure 8 , which shows a schematic diagram of the structure of an image tampering detection device according to an embodiment of the present specification. The image tampering detection device 2 can be implemented as all or part of the device through software, hardware, or a combination of both. According to some embodiments, the image tampering detection device 2 includes a data receiving module 21, a data processing module 22, and a data sending module 23, which are specifically used to:

[0196] A data receiving module 21 is configured to receive a second model gradient matrix uploaded by at least one electronic device, where the second model gradient matrix is ​​obtained by performing a local gradient differential privacy calculation on the first model gradient matrix corresponding to the local image tampering detection model by the electronic device;

[0197] A data processing module 22 is configured to determine target model parameters based on each of the second model gradient matrices;

[0198] The data sending module 23 is used to send the target model parameters to each of the electronic devices, where the target model parameters are used to instruct the electronic devices to update the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model, perform image detection processing on the target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0199] Optionally, the data processing module 22 includes:

[0200] A first calculation unit, configured to calculate a target gradient based on each of the second model gradient matrices;

[0201] a second calculation unit, configured to obtain a learning rate corresponding to the local image tampering detection model, and calculate a product of the learning rate and the target gradient;

[0202] The third calculation unit is used to obtain initial model parameters corresponding to the local image tampering model, calculate the sum of the product and the initial model parameters, and use the sum as the target model parameters.

[0203] Optionally, the first computing unit is specifically configured to:

[0204] Acquire the amount of matrix training data uploaded by each of the electronic devices, where the amount of matrix training data is a training data consumption value when generating the second model gradient matrix in the model training phase;

[0205] Determine the total amount of data based on all the matrix training data amounts;

[0206] Determining a ratio of the matrix training data volume to the total data volume, and using the ratio as a gradient weight corresponding to the electronic device;

[0207] A weighted sum is performed based on the gradient weight and the second model gradient matrix to obtain a target gradient.

[0208] Optionally, the image tampering detection device 2 is further configured to:

[0209] Determining a current training round of the local image tampering detection model;

[0210] If the current training round is greater than or equal to the preset round, a training end instruction is sent to each of the electronic devices, where the training end instruction is used to instruct each of the electronic devices to stop uploading the second model gradient matrix.

[0211] Please refer to Figure 9 , which shows a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of this specification. The electronic device described in this specification may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0212] Processor 110 may include one or more processing cores. Using various interfaces and circuits, processor 110 connects various components within the terminal. It executes instructions, programs, code sets, or instruction sets stored in memory 120, as well as accesses data stored in memory 120, to perform various functions and process data for terminal 100. Optionally, processor 110 may be implemented in hardware using at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). Processor 110 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may also be implemented independently of the processor 110 via a separate communications chip.

[0213] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The operating system may be an Android system, including systems deeply developed based on the Android system, an iOS system developed by Apple, including systems deeply developed based on the iOS system, or other systems.

[0214] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0215] The input device 130 is used to receive input commands or data and includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch-sensitive device. The output device 140 is used to output commands or data and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 may be a touch-sensitive display.

[0216] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of this specification.

[0217] In addition, those skilled in the art will understand that the structures of the electronic devices shown in the above figures do not limit the electronic devices. The electronic devices may include more or fewer components than shown, or may combine certain components or arrange the components differently. For example, the electronic devices may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WiFi) modules, power supplies, Bluetooth modules, and other components, which will not be described in detail here.

[0218] exist Figure 9In the electronic device shown, the processor 110 may be configured to call a program of the image tampering detection method stored in the memory 120 and specifically perform the following operations:

[0219] Determine a first model gradient matrix corresponding to the local image tampering detection model, and perform local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix;

[0220] Uploading the second model gradient matrix to the service platform, where the second model gradient matrix is ​​used to instruct the service platform to determine target model parameters based on the second model gradient matrix uploaded by the at least one electronic device;

[0221] receiving the target model parameters sent by the service platform, and updating the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model;

[0222] Based on the target image tampering detection model, image detection processing is performed on the target private image to obtain a target thermal detection map corresponding to the target private image, and tampering identification processing is performed on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0223] In one embodiment, when the processor 110 performs the step of performing a local gradient differential privacy calculation based on the first model gradient matrix to obtain a second model gradient matrix, the processor 110 specifically performs the following operations:

[0224] determining an initial local gradient subset in the first model gradient matrix;

[0225] Noise processing is performed on the initial local gradient subset to obtain a target local gradient subset, and the first model gradient matrix is ​​updated based on the target local gradient subset to obtain a second model gradient matrix.

[0226] In one embodiment, when executing the step of determining the initial local gradient subset in the first model gradient matrix, the processor 110 specifically performs the following operations:

[0227] Determining model features of the local image tampering detection, and determining candidate network parameters corresponding to the local image tampering detection model based on the model features;

[0228] An initial local gradient subset corresponding to the candidate network parameters is determined in the first model gradient matrix.

[0229] In one embodiment, when the processor 110 performs the step of determining the candidate network parameters corresponding to the local image tampering detection model based on the model features, the processor 110 specifically performs the following operations:

[0230] Determining a candidate network layer in the local image tampering detection model based on the model features;

[0231] Determine a sample mask image corresponding to the sample private image, and input the sample mask image into the local image tampering detection model to obtain learnable network parameters in the candidate network layer;

[0232] Determine candidate network parameters corresponding to the local image tampering detection model based on the learnable network parameters.

[0233] In one embodiment, when the processor 110 performs the step of determining the initial local gradient subset corresponding to the candidate network parameters in the first model gradient matrix, the following operations are specifically performed:

[0234] Determining at least one gradient corresponding to each of the candidate network parameters in the first model gradient matrix;

[0235] Filtering the at least one gradient to obtain a target gradient corresponding to the candidate network parameter;

[0236] An initial local gradient subset corresponding to the candidate network parameters is determined based on the target gradient.

[0237] In one embodiment, the processor 110 performs the following operations when performing the noise addition process on the initial local gradient subset to obtain the target local gradient subset:

[0238] Performing gradient clipping on the initial local gradient subset to obtain a reference local gradient subset;

[0239] Noise processing is performed on the reference gradient subset to obtain a target local gradient subset.

[0240] In one embodiment, when executing the step of determining the first model gradient matrix corresponding to the local image tampering detection model, the processor 110 specifically performs the following operations:

[0241] Get historical privacy images;

[0242] A local image tampering detection model is trained based on the historical privacy image to obtain a first model gradient matrix.

[0243] In one embodiment, when the processor 110 performs the step of performing tampering identification processing on the target thermal detection image to obtain a tampering identification result corresponding to the target private image, the processor 110 specifically performs the following operations:

[0244] Obtain a device-specific classifier, and perform tampering identification processing on the target thermal detection image based on the device-specific classifier to obtain a tampering identification result corresponding to the target privacy image; wherein the device-specific classifiers of different electronic devices are not all the same.

[0245] Please refer to Figure 10 , which shows a schematic diagram of the structure of a service platform provided by an exemplary embodiment of this specification. The service platform described in this specification may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0246] Processor 110 may include one or more processing cores. Using various interfaces and circuits, processor 110 connects various components within the terminal. It executes instructions, programs, code sets, or instruction sets stored in memory 120, as well as accesses data stored in memory 120, to perform various functions and process data for terminal 100. Optionally, processor 110 may be implemented in hardware using at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). Processor 110 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may also be implemented independently of the processor 110 via a separate communications chip.

[0247] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The operating system may be an Android system, including systems deeply developed based on the Android system, an iOS system developed by Apple, including systems deeply developed based on the iOS system, or other systems.

[0248] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0249] The input device 130 is used to receive input commands or data and includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch-sensitive device. The output device 140 is used to output commands or data and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 may be a touch-sensitive display.

[0250] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of this specification.

[0251] Furthermore, those skilled in the art will appreciate that the structures of the electronic devices illustrated in the above figures do not limit the service platform. The service platform may include more or fewer components than illustrated, or may combine certain components or arrange them differently. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WiFi) modules, power supplies, Bluetooth modules, and other components, which will not be detailed here.

[0252] exist Figure 10In the service platform shown, the processor 110 may be configured to call the image tampering detection method program stored in the memory 120 and specifically perform the following operations:

[0253] Receive a second model gradient matrix uploaded by at least one electronic device, where the second model gradient matrix is ​​obtained by the electronic device performing local gradient differential privacy calculation based on the first model gradient matrix corresponding to the local image tampering detection model;

[0254] determining target model parameters based on each of the second model gradient matrices;

[0255] The target model parameters are sent to each of the electronic devices, where the target model parameters are used to instruct the electronic device to update a local image tampering detection model based on the target model parameters to obtain a target image tampering detection model, perform image detection processing on a target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image.

[0256] In one embodiment, when executing the step of determining the target model parameters based on each of the second model gradient matrices, the processor 110 specifically performs the following operations:

[0257] Calculating a target gradient based on each of the second model gradient matrices;

[0258] Obtaining a learning rate corresponding to a local image tampering detection model, and calculating a product of the learning rate and the target gradient;

[0259] Initial model parameters corresponding to the local image tampering model are obtained, a sum of the product and the initial model parameters is calculated, and the sum is used as a target model parameter.

[0260] In one embodiment, when executing the step of calculating the target gradient based on each of the second model gradient matrices, the processor 110 specifically performs the following operations:

[0261] Acquire the amount of matrix training data uploaded by each of the electronic devices, where the amount of matrix training data is a training data consumption value when generating the second model gradient matrix in the model training phase;

[0262] Determine the total amount of data based on all the matrix training data amounts;

[0263] Determining a ratio of the matrix training data volume to the total data volume, and using the ratio as a gradient weight corresponding to the electronic device;

[0264] A weighted sum is performed based on the gradient weight and the second model gradient matrix to obtain a target gradient.

[0265] In one embodiment, the processor 110 further performs the following operations:

[0266] Determining a current training round of the local image tampering detection model;

[0267] If the current training round is greater than or equal to the preset round, a training end instruction is sent to each of the electronic devices, where the training end instruction is used to instruct each of the electronic devices to stop uploading the second model gradient matrix.

[0268] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, and display, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties. The collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the historical private images and target private images mentioned in this specification were obtained with full authorization.

[0269] An embodiment of this specification further provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the image tampering detection method described in the above embodiments.

[0270] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of this specification can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0271] The above description is only an optional embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0272] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for detecting image tampering, applied to an electronic device, comprising: Determine a first model gradient matrix corresponding to the local image tampering detection model, and perform local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix; Uploading the second model gradient matrix to the service platform, where the second model gradient matrix is ​​used to instruct the service platform to determine target model parameters based on the second model gradient matrix uploaded by at least one electronic device; receiving the target model parameters sent by the service platform, and updating the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model; performing image detection processing on the target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and performing tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image; The performing of local gradient differential privacy calculation based on the first model gradient matrix to obtain the second model gradient matrix includes: determining the model features of the local image tampering detection model, determining the candidate network parameters corresponding to the local image tampering detection model based on the model features, determining the initial local gradient subset corresponding to the candidate network parameters in the first model gradient matrix, performing noise processing on the initial local gradient subset to obtain a target local gradient subset, and updating the first model gradient matrix based on the target local gradient subset to obtain the second model gradient matrix; The determining, based on the model features, candidate network parameters corresponding to the local image tampering detection model includes: obtaining a network layer mapping relationship, the network layer mapping relationship being used to store a correspondence between a reference model feature and a reference candidate network layer, searching the network layer mapping relationship for a candidate network layer corresponding to the model feature, determining a sample mask image corresponding to a sample private image, inputting the sample mask image into the local image tampering detection model to obtain learnable network parameters in the candidate network layer, and using the parameter with the largest absolute value among the learnable network parameters as the candidate network parameter corresponding to the local image tampering detection model; The determining, in the first model gradient matrix, an initial local gradient subset corresponding to the candidate network parameter includes: selecting at least one gradient corresponding to the candidate network parameter from the first model gradient matrix according to position information of the candidate network parameter in the network; obtaining a gradient threshold corresponding to the candidate network parameter from a gradient filtering mapping table, wherein the gradient filtering mapping table is configured with gradient thresholds corresponding to different candidate network parameters, filtering out gradients smaller than the gradient threshold from the at least one gradient to obtain a target gradient corresponding to the candidate network parameter; and determining an initial local gradient subset corresponding to the candidate network parameter based on the target gradient; The performing noise processing on the initial local gradient subset to obtain a target local gradient subset includes: performing gradient clipping processing on the initial local gradient subset according to a gradient boundary to obtain a reference gradient subset, obtaining a selected number of the sample private images, obtaining a variance of Gaussian noise, and performing noise processing based on the reference gradient subset, the selected number, the variance, and the gradient boundary using a preset formula to obtain a target local gradient subset, wherein the preset formula is as follows: Wherein, m represents the number of selected sample private images, is the variance of the Gaussian noise, C is the gradient boundary, g represents the reference gradient subset, represents the target local gradient subset; The performing of tampering identification processing on the target thermal detection image to obtain a tampering identification result corresponding to the target privacy image includes: obtaining a device-specific classifier, and performing tampering identification processing on the target thermal detection image based on the device-specific classifier to obtain a tampering identification result corresponding to the target privacy image; wherein the device-specific classifiers of different electronic devices are not all the same.

2. The method according to claim 1, wherein determining a first model gradient matrix corresponding to the local image tampering detection model comprises: Get historical privacy images; A local image tampering detection model is trained based on the historical privacy image to obtain a first model gradient matrix.

3. A method for detecting image tampering, applied to a service platform, comprising: Receive a second model gradient matrix uploaded by at least one electronic device, where the second model gradient matrix is ​​obtained by the electronic device performing local gradient differential privacy calculation based on the first model gradient matrix corresponding to the local image tampering detection model; determining target model parameters based on each of the second model gradient matrices; Sending the target model parameters to each of the electronic devices, the target model parameters being used to instruct the electronic devices to update a local image tampering detection model based on the target model parameters to obtain a target image tampering detection model, performing image detection processing on a target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and performing tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image; The step of the electronic device determining the second model gradient matrix includes: determining a model feature of the local image tampering detection model, determining candidate network parameters corresponding to the local image tampering detection model based on the model feature, determining an initial local gradient subset corresponding to the candidate network parameter in the first model gradient matrix, performing noise processing on the initial local gradient subset to obtain a target local gradient subset, and updating the first model gradient matrix based on the target local gradient subset to obtain a second model gradient matrix; The electronic device determines, based on the model features, candidate network parameters corresponding to the local image tampering detection model, including: obtaining a network layer mapping relationship, the network layer mapping relationship being used to store a correspondence between a reference model feature and a reference candidate network layer, searching the network layer mapping relationship for a candidate network layer corresponding to the model feature, determining a sample mask image corresponding to a sample private image, inputting the sample mask image into the local image tampering detection model to obtain learnable network parameters in the candidate network layer, and using the parameter with the largest absolute value among the learnable network parameters as the candidate network parameter corresponding to the local image tampering detection model; The determining, in the first model gradient matrix, an initial local gradient subset corresponding to the candidate network parameter includes: selecting at least one gradient corresponding to the candidate network parameter from the first model gradient matrix according to position information of the candidate network parameter in the network; obtaining a gradient threshold corresponding to the candidate network parameter from a gradient filtering mapping table, wherein the gradient filtering mapping table is configured with gradient thresholds corresponding to different candidate network parameters, filtering out gradients smaller than the gradient threshold from the at least one gradient to obtain a target gradient corresponding to the candidate network parameter; and determining an initial local gradient subset corresponding to the candidate network parameter based on the target gradient; The performing noise processing on the initial local gradient subset to obtain a target local gradient subset includes: performing gradient clipping processing on the initial local gradient subset according to a gradient boundary to obtain a reference gradient subset, obtaining a selected number of the sample private images, obtaining a variance of Gaussian noise, and performing noise processing based on the reference gradient subset, the selected number, the variance, and the gradient boundary using a preset formula to obtain a target local gradient subset, wherein the preset formula is as follows: Wherein, m represents the number of selected sample private images, is the variance of the Gaussian noise, C is the gradient boundary, g represents the reference gradient subset, represents the target local gradient subset; The electronic device performs tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target privacy image, including: obtaining a device-specific classifier, and performing tampering identification processing on the target thermal detection map based on the device-specific classifier to obtain a tampering identification result corresponding to the target privacy image; wherein the device-specific classifiers of different electronic devices are not all the same.

4. The method according to claim 3, wherein determining the target model parameters based on each of the second model gradient matrices comprises: Calculating a target gradient based on each of the second model gradient matrices; Obtaining a learning rate corresponding to a local image tampering detection model, and calculating a product of the learning rate and the target gradient; Initial model parameters corresponding to the local image tampering model are obtained, a sum of the product and the initial model parameters is calculated, and the sum is used as a target model parameter.

5. The method according to claim 4, wherein the step of calculating the target gradient based on each of the second model gradient matrices comprises: Acquire the amount of matrix training data uploaded by each of the electronic devices, where the amount of matrix training data is a training data consumption value when generating the second model gradient matrix in the model training phase; Determine the total amount of data based on all the matrix training data amounts; Determining a ratio of the matrix training data volume to the total data volume, and using the ratio as a gradient weight corresponding to the electronic device; A weighted sum is performed based on the gradient weight and the second model gradient matrix to obtain a target gradient.

6. The method according to claim 3, further comprising: Determining a current training round of the local image tampering detection model; If the current training round is greater than or equal to the preset round, a training end instruction is sent to each of the electronic devices, where the training end instruction is used to instruct each of the electronic devices to stop uploading the second model gradient matrix.

7. An image tampering detection device, applied to an electronic device, comprising: A gradient processing module, configured to determine a first model gradient matrix corresponding to a local image tampering detection model, and perform local gradient differential privacy calculation processing based on the first model gradient matrix to obtain a second model gradient matrix; a gradient sending module, configured to upload the second model gradient matrix to a service platform, wherein the second model gradient matrix is ​​used to instruct the service platform to determine target model parameters based on the second model gradient matrix uploaded by at least one electronic device; a data receiving module, configured to receive the target model parameters sent by the service platform, and update the local image tampering detection model based on the target model parameters to obtain a target image tampering detection model; an image processing module, configured to perform image detection processing on a target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image; The gradient processing module is specifically configured to: determine the model features of the local image tampering detection, determine the candidate network parameters corresponding to the local image tampering detection model based on the model features, determine the initial local gradient subset corresponding to the candidate network parameters in the first model gradient matrix, perform noise processing on the initial local gradient subset to obtain a target local gradient subset, and update the first model gradient matrix based on the target local gradient subset to obtain a second model gradient matrix; The determining, based on the model features, candidate network parameters corresponding to the local image tampering detection model includes: obtaining a network layer mapping relationship, the network layer mapping relationship being used to store a correspondence between a reference model feature and a reference candidate network layer, searching the network layer mapping relationship for a candidate network layer corresponding to the model feature, determining a sample mask image corresponding to a sample private image, inputting the sample mask image into the local image tampering detection model to obtain learnable network parameters in the candidate network layer, and using the parameter with the largest absolute value among the learnable network parameters as the candidate network parameter corresponding to the local image tampering detection model; The determining, in the first model gradient matrix, an initial local gradient subset corresponding to the candidate network parameter includes: selecting at least one gradient corresponding to the candidate network parameter from the first model gradient matrix according to position information of the candidate network parameter in the network; obtaining a gradient threshold corresponding to the candidate network parameter from a gradient filtering mapping table, wherein the gradient filtering mapping table is configured with gradient thresholds corresponding to different candidate network parameters, filtering out gradients smaller than the gradient threshold from the at least one gradient to obtain a target gradient corresponding to the candidate network parameter; and determining an initial local gradient subset corresponding to the candidate network parameter based on the target gradient; The performing noise processing on the initial local gradient subset to obtain a target local gradient subset includes: performing gradient clipping processing on the initial local gradient subset according to a gradient boundary to obtain a reference gradient subset, obtaining a selected number of the sample private images, obtaining a variance of Gaussian noise, and performing noise processing based on the reference gradient subset, the selected number, the variance, and the gradient boundary using a preset formula to obtain a target local gradient subset, wherein the preset formula is as follows: Wherein, m represents the number of selected sample private images, is the variance of the Gaussian noise, C is the gradient boundary, g represents the reference gradient subset, represents the target local gradient subset; Among them, the image processing module is specifically used to: obtain a device-specific classifier, and perform tampering identification processing on the target thermal detection image based on the device-specific classifier to obtain a tampering identification result corresponding to the target privacy image; wherein, the device-specific classifiers of different electronic devices are not all the same.

8. An image tampering detection device, applied to a service platform, comprising: A data receiving module, configured to receive a second model gradient matrix uploaded by at least one electronic device, where the second model gradient matrix is ​​obtained by performing a local gradient differential privacy calculation on the first model gradient matrix corresponding to the local image tampering detection model by the electronic device; a data processing module, configured to determine target model parameters based on each of the second model gradient matrices; a data sending module, configured to send the target model parameters to each of the electronic devices, wherein the target model parameters are used to instruct the electronic devices to update a local image tampering detection model based on the target model parameters to obtain a target image tampering detection model, perform image detection processing on a target private image based on the target image tampering detection model to obtain a target thermal detection map corresponding to the target private image, and perform tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target private image; The step of the electronic device determining the second model gradient matrix includes: determining the model features of the local image tampering detection, determining the candidate network parameters corresponding to the local image tampering detection model based on the model features, determining the initial local gradient subset corresponding to the candidate network parameters in the first model gradient matrix, performing noise processing on the initial local gradient subset to obtain a target local gradient subset, and updating the first model gradient matrix based on the target local gradient subset to obtain a second model gradient matrix; The electronic device determines, based on the model features, candidate network parameters corresponding to the local image tampering detection model, including: obtaining a network layer mapping relationship, the network layer mapping relationship being used to store a correspondence between a reference model feature and a reference candidate network layer, searching the network layer mapping relationship for a candidate network layer corresponding to the model feature, determining a sample mask image corresponding to a sample private image, inputting the sample mask image into the local image tampering detection model to obtain learnable network parameters in the candidate network layer, and using the parameter with the largest absolute value among the learnable network parameters as the candidate network parameter corresponding to the local image tampering detection model; The determining, in the first model gradient matrix, an initial local gradient subset corresponding to the candidate network parameter includes: selecting at least one gradient corresponding to the candidate network parameter from the first model gradient matrix according to position information of the candidate network parameter in the network; obtaining a gradient threshold corresponding to the candidate network parameter from a gradient filtering mapping table, wherein the gradient filtering mapping table is configured with gradient thresholds corresponding to different candidate network parameters, filtering out gradients smaller than the gradient threshold from the at least one gradient to obtain a target gradient corresponding to the candidate network parameter; and determining an initial local gradient subset corresponding to the candidate network parameter based on the target gradient; The performing noise processing on the initial local gradient subset to obtain a target local gradient subset includes: performing gradient clipping processing on the initial local gradient subset according to a gradient boundary to obtain a reference gradient subset, obtaining a selected number of the sample private images, obtaining a variance of Gaussian noise, and performing noise processing based on the reference gradient subset, the selected number, the variance, and the gradient boundary using a preset formula to obtain a target local gradient subset, wherein the preset formula is as follows: Wherein, m represents the number of selected sample private images, is the variance of the Gaussian noise, C is the gradient boundary, g represents the reference gradient subset, represents the target local gradient subset; The electronic device performs tampering identification processing on the target thermal detection map to obtain a tampering identification result corresponding to the target privacy image, including: obtaining a device-specific classifier, and performing tampering identification processing on the target thermal detection map based on the device-specific classifier to obtain a tampering identification result corresponding to the target privacy image; wherein the device-specific classifiers of different electronic devices are not all the same.

9. A computer storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 2 or 3 to 6.

10. A computer program product, wherein the computer program product stores at least one instruction, wherein the at least one instruction is loaded by a processor and executes the method steps according to any one of claims 1 to 2 or 3 to 6.

11. An electronic device comprising: A processor and a memory; wherein the memory stores a computer program, the computer program being suitable for being loaded by the processor and executing the method steps according to any one of claims 1 to 2 or 3 to 6.

Images