Picture origin identification method

Abstract

Embodiments of the present specification provide a picture source identification method, comprising: in response to a picture source identification request, extracting a to-be-identified picture in the picture source identification request, and determining a device template library; determining, according to a dual-view feature encoder, a device source feature of each source device in the device template library and a picture source feature of the to-be-identified picture; calculating a similarity between the picture source feature of the to-be-identified picture and the device source feature of each source device, and determining a target source device of the to-be-identified picture from the device template library according to the similarity. The method extracts the picture source feature of the to-be-identified picture in the RGB view and the noise view through the dual-view feature encoder, and matches the picture source feature with the device source feature of the device in the device template library, so as to realize the identification of the target source device (such as a mobile phone or a camera) corresponding to the to-be-identified picture, and the identification efficiency and accuracy are high.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of picture processing, in particular to two picture source identification methods. BACKGROUND

[0002] In today's digital age, with the help of various hardware or software tools, image creation and editing have become very simple, so the authenticity and authority of images have been greatly challenged. Image forensics technology can help us evaluate the authenticity and integrity of a given image, and one of the most important areas is the image source identification problem; for example, image source identification can be applied in image leakage tracing scenarios, platform stolen image identification scenarios, real person authentication device authentication scenarios, black and gray production group tracing and identification scenarios, etc. However, there is no particularly good solution for image source identification at present. SUMMARY

[0003] Therefore, the embodiments of the present specification provide two picture source identification methods. One or more embodiments of the present specification simultaneously relate to two picture source identification devices, a computing device, a computer readable storage medium and a computer program to solve the technical defects in the prior art.

[0004] According to a first aspect of the embodiments of the present specification, a picture source identification method is provided, comprising:

[0005] In response to a picture source identification request, extracting a to-be-identified picture in the picture source identification request, and determining a device template library;

[0006] According to a dual-view feature encoder, determining a device source feature of each source device in the device template library and a picture source feature of the to-be-identified picture, wherein the dual-view feature encoder is obtained by training through a picture view and a noise view;

[0007] Calculating the similarity of the picture source feature of the to-be-identified picture and the device source feature of each source device, and determining a target source device of the to-be-identified picture from the device template library according to the similarity.

[0008] According to a second aspect of the embodiments of the present specification, a picture source identification device is provided, comprising:

[0009] The request receiving module is configured to, in response to a picture source identification request, extract a to-be-identified picture in the picture source identification request, and determine a device template library;

[0010] a feature determination module configured to determine a device source feature of each source device in the device template library and a picture source feature of the picture to be identified based on a dual-view feature encoder, wherein the dual-view feature encoder is obtained by training with picture view and noise view;

[0011] The source determination module is configured to calculate the similarity between the image source feature of the image to be identified and the device source feature of each source device, and determine the target source device of the image to be identified from the device template library based on the similarity.

[0012] According to a third aspect of the embodiments of this specification, a method for identifying the source of an image is provided, comprising:

[0013] In response to a picture source identification request, extracting the picture to be identified and the initial source device in the picture source identification request;

[0014] Determining, according to a dual-view feature encoder, a picture source feature of the picture to be identified and a device source feature of the initial source device, wherein the dual-view feature encoder is obtained by training through picture view and noise view;

[0015] Calculate the similarity between the image source feature of the image to be identified and the device source feature of the initial source device, and determine whether the initial source device is the target source device of the image to be identified based on the similarity.

[0016] According to a fourth aspect of the embodiments of this specification, a device for identifying the source of an image is provided, comprising:

[0017] an information extraction module configured to extract, in response to a picture source identification request, a picture to be identified and an initial source device in the picture source identification request;

[0018] a source feature determination module configured to determine the image source feature of the image to be identified and the device source feature of the initial source device based on a dual-view feature encoder, wherein the dual-view feature encoder is obtained by training the image view and the noise view;

[0019] The source device determination module is configured to calculate the similarity between the image source feature of the image to be identified and the device source feature of the initial source device, and determine whether the initial source device is the target source device of the image to be identified based on the similarity.

[0020] According to a fifth aspect of the embodiments of this specification, there is provided a computing device, including:

[0021] memory and processor;

[0022] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned image source identification method are implemented.

[0023] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned image source identification method are implemented.

[0024] According to a seventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned image source identification method.

[0025] One embodiment of the present specification implements two methods for identifying image sources, one of which includes: in response to an image source identification request, extracting the image to be identified in the image source identification request, and determining a device template library; determining the device source characteristics of each source device in the device template library and the image source characteristics of the image to be identified based on a dual-view feature encoder, wherein the dual-view feature encoder is obtained through image perspective and noise perspective training; calculating the image source characteristics of the image to be identified and the similarity with the device source characteristics of each source device, and determining the target source device of the image to be identified from the device template library based on the similarity.

[0026] Specifically, the image source identification method uses a dual-view feature encoder to jointly extract the image source features of the image to be identified in the RGB view and the noise view, and matches the image source features with the device source features of the devices in the device template library to achieve the identification of the target source device (such as a mobile phone or camera, etc.) corresponding to the image to be identified, with high recognition efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of a specific scenario of a method for identifying the source of an image provided by an embodiment of this specification;

[0028] Figure 2 This is a flow chart of a method for identifying the source of an image provided by one embodiment of this specification;

[0029] Figure 3 This is a flow chart for obtaining source identification features in a method for identifying the source of an image provided by one embodiment of this specification;

[0030] Figure 4 This is a flowchart of another method for identifying the source of an image provided by an embodiment of this specification;

[0031] Figure 5 This is a flowchart of a processing process of a method for identifying the source of an image provided by an embodiment of this specification;

[0032] Figure 6 This is a schematic diagram of the structure of a device for identifying the source of an image provided by one embodiment of this specification;

[0033] Figure 7 This is a schematic structural diagram of another image source identification device provided by an embodiment of this specification;

[0034] Figure 8 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0035] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0036] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0037] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0038] First, the terms involved in one or more embodiments of this specification are explained.

[0039] Contrastive Learning: is a method for describing similar and different things to deep learning models; using this method, machine learning models can be trained to distinguish between similar and different images.

[0040] Dual-View Feature Learning: The process of converting image data of the source to be identified into source identification features, including hand-designed camera sensor fingerprint noise based on traditional digital signal processing methods and deep learning feature encoder based on contrastive learning framework.

[0041] Image Source Identification: Identification of the camera model and individual camera used to capture an image.

[0042] In today's digital age, with the help of various hardware or software tools, the creation and editing of images have become very simple, so the authenticity and authority of images have been greatly challenged. Image forensics technology can help us evaluate the authenticity and integrity of a given image. One of the most important areas is the image source identification problem (ISI). The purpose of the ISI problem is mainly to identify the fingerprint of the digital image acquisition device, including the identification of a specific shooting device model and the identification of a specific shooting device individual. In layman's terms, it is the identification of the camera model that captured the image and the identification of the individual camera. Image source identification technology is mainly used in the field of data security, and can also be used as a supplementary certificate chain for other forensic technologies such as original image identification. The specific application scenarios are described as follows:

[0043] Image Leakage Tracing: Numerous internal corporate information leaks demonstrate that high-risk data leaks often occur via image transmission. For example, taking a screen shot can quickly generate a high-fidelity copy of sensitive data. Image source identification technology can track the device that captured the leaked image, facilitating image leak tracing.

[0044] Platform image theft identification scenario: Black and gray industries steal merchant images to file malicious appeals and complaints, which will cause legal and public opinion risks. Image source identification technology can effectively detect image theft.

[0045] Real-person authentication and device authentication scenarios: In real-person authentication scenarios, the authenticator's device can be verified through image source recognition technology, effectively preventing black and gray industries from injecting face-changing images and videos.

[0046] Tracing and identifying illegal and gray market gangs: Usually, images produced in batches by illegal and gray market gangs are generated using a fixed set of equipment. Source identification technology can effectively track commonly used equipment and achieve targeted strikes against these gangs.

[0047] In addition, the image source identification technology can also be used for screening and assisting identification of suspicious images, which is of great significance.

[0048] To achieve image source identification, the present specification provides camera model source identification based on sensor noise and deep learning-based camera model classification, wherein the camera model source identification based on sensor noise is a method using sensor noise as a feature source, that is, extracting essential features in the picture to be tested that are irrelevant to semantics but related to the phone model. For example, PRNU (Photo response non-uniformity) is a source of digital camera pattern noise, which is related to the camera sensor model selected by the phone, and is a pattern noise irrelevant to semantic features. However, the method of camera model source identification based on sensor noise has good effect on pictures with original imaging data, but is not robust to post-processing, and sensor noise is easily destroyed in the enterprise commonly used picture post-processing chain such as size scaling and JPEG compression. In addition, this method is commonly used for camera sensor source identification of one-time imaging and high-quality imaging. Currently, after one-time imaging, the phone manufacturers will use their own algorithms to optimize the picture (such as night scene enhancement, image super-resolution, etc.), and the traces of the corresponding picture quality optimization algorithm will further cover the sensor noise, affecting the feature extraction effect.

[0049] The deep learning-based camera model classification can learn and extract picture features based on a deep learning model; for example, training an efficient-net model to do 27 classifications of 27 phone models, or extracting PRNU and Noiseprint two noises, inputting a twin neural network, and outputting a two-classification result to predict whether the image pair is from the same device; but the above methods are all for closed set classification, and the first method can only do multi-classification for the existing 27 classes, and cannot handle unseen classes and subsequently added classes; the second method outputs a two-classification matching result, which cannot realize 1vN multi-model source identification.

[0050] Therefore, in the present specification, two other picture source identification methods are provided. One or more embodiments of the present specification simultaneously relate to two picture source identification devices, a computing device, a computer-readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

[0051] Referring to Figure 1 , Figure 1 A specific scene schematic diagram of a picture source identification method according to an embodiment of the present specification is shown.

[0052] In actual applications, the image source identification method provided in the embodiments of this specification can be applied to a terminal or to a server (an ordinary server or a cloud server, etc.). When the image source identification method is applied to a terminal, the dual-view feature encoder and the subsequent feature similarity comparison can be implemented in the terminal; and when the image source identification method is applied to a server, the dual-view feature encoder and the subsequent feature similarity comparison can be implemented on the server, or the dual-view feature encoder can be trained and applied on the server, and the subsequent feature similarity comparison can be implemented on the terminal, etc.

[0053] For ease of understanding, the embodiments of this specification are described in detail by taking the application of the image source identification method to the server as an example.

[0054] Figure 1 The system includes a terminal 102 and a server 104, wherein the server 104 is deployed with a pre-trained dual-view feature encoder.

[0055] During specific implementation, the server 104 receives the image source identification request sent by the terminal 102, and parses the image source identification request. If the image source identification request is determined to be: when judging whether the target image is a image taken by the target mobile phone, the target image carried in the image source identification request and the historical images taken by the target mobile phone are respectively input into the dual-view feature encoder to obtain the image source features of the target image and the device source features of the target mobile phone; calculate the similarity of the image source features of the target image and the device source features of the target mobile phone, and when the similarity is greater than the preset similarity threshold, determine that the target image is a image taken by the target mobile phone, that is, the source device of the target image is the target mobile phone.

[0056] If the image source identification request is determined according to the analysis results as follows: when the source mobile phone of the target image is determined, the target image carried in the image source identification request is input into the dual-view feature encoder to obtain the image source characteristics of the target image; at the same time, the historical images taken by each mobile phone in the mobile phone template library are input into the dual-view feature encoder to obtain the device source characteristics of each mobile phone; the image source characteristics of the target image and the feature distance from the device source characteristics of each mobile phone are calculated, and the mobile phone with the smaller feature distance is used as the source mobile phone of the target image.

[0057] See also Figure 2 , Figure 2 A flowchart of a method for identifying the source of an image provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0058] Step 202: In response to the picture source identification request, extract the picture to be identified in the picture source identification request, and determine the device template library.

[0059] Among them, the picture to be identified can be understood as a picture of any type, any size, and any content to be identified, such as a picture taken by a mobile phone or camera; the device can be understood as any model of mobile phone, camera and other devices that can take pictures; then when the device is understood as a mobile phone or camera, the device template library includes pictures taken by any mobile phone or camera.

[0060] Specifically, taking the device as a mobile phone as an example, in response to the image source identification request, the device template library corresponding to the image to be identified carried in the image source identification request is determined; it can be understood as responding to the image source identification request, and determining the corresponding mobile phone template library according to the image to be identified carried in the image source identification request.

[0061] In addition, the image to be identified can also be understood as being extracted from three-dimensional video data captured by a VR (virtual reality) device or an AR (augmented reality) device. The specific implementation method is as follows:

[0062] The step of extracting, in response to the image source identification request, the image to be identified in the image source identification request, includes:

[0063] In response to the image source identification request, determining video data carried in the image source identification request, wherein the video data includes video data obtained through a VR device or an AR device;

[0064] A target video frame is extracted from the video data, and the target video frame is used as a picture to be identified.

[0065] In specific implementation, the sources of the images to be identified are different, and their corresponding source devices are also different. For example, when the image to be identified is a target video frame extracted from video data obtained from a VR device or an AR device, the image source identification request in the embodiment of this specification can be parsed as a request to determine the target VR device or AR device of the image to be identified. Then, the technical problem to be solved by the embodiment of this specification is to determine which VR device or AR device is the source device of the image to be identified.

[0066] Step 204: Determine the device source feature of each source device in the device template library and the image source feature of the image to be identified based on the dual-view feature encoder.

[0067] The dual-view feature encoder is obtained by training the image view and the noise view.

[0068] In specific implementation, before the dual-view feature encoder is applied, it is necessary to train it based on the picture training set through the picture perspective and the noise perspective, so that the picture source characteristics of the picture to be identified or the device source characteristics of the source device can be quickly and accurately obtained through the picture perspective and the noise perspective.

[0069] Specifically, the training steps of the dual-view feature encoder include:

[0070] Obtain sample images taken by multiple source devices and determine the source device corresponding to each sample image;

[0071] Obtaining a third imaging fingerprint noise of each sample image according to a preset noise extraction method;

[0072] Performing feature fusion on each sample image and the third imaging fingerprint noise to obtain a fusion feature of each sample image;

[0073] A dual-view feature encoder is trained based on the fusion features of each sample image and the device source features of the source device corresponding to each sample image.

[0074] Specifically, multiple source devices can be understood as multiple models of mobile phones, cameras, etc.; sample images taken by multiple source devices can be understood as sample images taken by different models of mobile phones or cameras, and each model of mobile phone or camera can take one, two, or more sample images. In addition, to eliminate the influence of semantic information on sample images, multiple source devices can take sample images with the same or similar subject matter, such as all images of human faces, all images of animals, or all images of commodities, etc.; and the device source characteristics of the source device corresponding to each sample image can be understood as the characteristics of the model, device parameters, etc. of the mobile phone or camera corresponding to each sample image.

[0075] In addition, the preset noise extraction methods include but are not limited to PRNU, CFA (color filter array), Noiseprint (a camera model fingerprint based on deep convolutional neural network), BayarConv (a deep learning high-pass convolution kernel with a size of 5*5 and a fixed middle value of -1); and the fusion method of feature fusion of each sample image with its corresponding third imaging fingerprint noise also includes but is not limited to splicing, weighted summation, or attention mechanism module for processing and implementation.

[0076] The backbone network of the dual-view feature encoder includes, but is not limited to, MobileNet (a lightweight deep learning convolutional neural network structure), EfficientNet (a deep learning convolutional neural network structure), and ResNet (a deep learning convolutional neural network structure with skip connection and bottleneck structure).

[0077] In the above example, if the device is a mobile phone, a plurality of sample pictures of the same or similar subject content photographed by the mobile phone are obtained, and the mobile phone corresponding to each sample picture is determined; third imaging fingerprint noise (for example, mobile phone imaging fingerprint noise) of each sample picture is obtained according to a preset noise extraction method; then the feature fusion of each sample picture and the corresponding third imaging fingerprint noise is performed to obtain the fusion feature of each sample picture; finally, the dual-view feature encoder is trained according to the fusion feature of each sample picture and the device source feature of the source device corresponding to each sample picture.

[0078] In the embodiments of the present specification, the dual-view feature learning (picture view and noise view) based on the deep learning method is adopted to train the dual-view feature encoder, so that the dual-view feature encoder can quickly and accurately identify the features of the source device corresponding to any picture (i.e., picture source features) in the future.

[0079] In actual application, the picture for which the picture source feature identification is performed in the future may be a post-processed picture. In order to ensure that the dual-view feature encoder can also accurately identify the picture source features of the picture damaged in the post-processing link, the dual-view feature encoder can be trained according to the post-processed sample pictures, so as to improve the robustness of the dual-view feature encoder in processing the post-processed pictures to be identified. The specific implementation mode is as follows:

[0080] Before the third imaging fingerprint noise of each sample picture is obtained according to the preset noise extraction method, the following steps are further included:

[0081] The post-processing of each sample picture is performed to obtain the processed each sample picture, wherein the post-processing includes enlargement, reduction and / or compression of the each sample picture.

[0082] Specifically, before the dual-view feature encoder is trained according to the sample pictures, the post-processing link of the picture is added in the data preprocessing, and various post-processing operations are performed on each sample picture to increase the robustness of the dual-view feature encoder after training to the post-processed pictures. If the post-processing of the sample picture is not performed during the training process, in order to ensure the accuracy of the dual-view feature encoder in the future, the post-processing operation can be performed on the picture before the feature extraction of the picture source feature identification.

[0083] In the embodiments of the present specification, a dual-view feature learning (picture view and noise view) based on a deep learning method is adopted to extract the mode features (i.e. post-processing) of the image quality optimization algorithms of each mobile phone or camera manufacturer, and a dual-view feature encoder is trained by the mode features and the device source features (such as model features) of the mobile phone or camera device, so that the dual-view feature encoder can quickly and accurately identify the source device features (i.e. picture source features) corresponding to the picture of any mode feature in the future.

[0084] In addition, when training the dual-view feature encoder, multiple sample pictures taken by each source device will be obtained, so that the training set formed by these sample pictures will include sample pictures of the same class (sample pictures taken by mobile phones or cameras of the same category) and sample pictures of different classes (sample pictures taken by mobile phones or cameras of different categories); in order to ensure that the dual-view feature encoder can better distinguish the picture source features of the same class of pictures and give more accurate classification results, a contrast learning framework is used to train the dual-view feature encoder, and the distance constraints are made on the source recognition features (picture source features) of the same class and different class sample pictures, the distances of the same class sample pictures are shortened, the distances of different classes are increased, and the weights of the sample pairs that are difficult to distinguish are increased. The specific implementation mode is as follows:

[0085] The dual-view feature encoder is trained according to the fusion features of each sample picture and the device source features of the source device corresponding to each sample picture, including:

[0086] The fusion features of each sample picture are used as training samples, and the device source features of the source device corresponding to each sample picture are used as training labels of the training samples;

[0087] According to a preset feature distance constraint method, a first spatial distance between training samples of the same device model and a second spatial distance between training samples of different device models are trained;

[0088] The dual-view feature encoder is trained according to the training samples, the training labels, the first spatial distance and the second spatial distance.

[0089] Among them, the preset feature distance constraint methods include Euclidean space constraint methods (such as MarginLoss in Euclidean space constraints), spherical space constraint methods (such as CosineLoss in spherical space constraints), and methods that use Euclidean space constraint loss, spherical space constraint loss and hard-pair mining to increase the weight of difficult pairs (such as TripletLoss, ternary loss function). Euclidean space constraint, spherical space constraint and TripletLoss are all loss functions used to train dual-view feature encoders. During the gradient descent process, the features of devices of the same category are brought closer, and the features of devices of different categories are kept away.

[0090] Specifically, when training the dual-view feature encoder, the fusion features of each sample image are used as training samples, and the device source features of the source device corresponding to each sample image are used as training labels corresponding to the training samples; then, training samples of the same class are determined based on the training labels, and according to the preset feature distance constraint method, the first spatial distance between these training samples of the same class and the second spatial distance between training samples of different classes are trained; finally, based on the above training samples, the training labels, first spatial distances and second spatial distances corresponding to each training sample, the dual-view feature encoder is trained.

[0091] In practical applications, the source identification features of the extracted sample images are constrained at different distance spaces, and different strategies are used to assign weights to individual sample pairs in order to better train the feature distances of devices of the same model or different models. That is, the role of using different distance space constraint losses is to distinguish the features of sample images of devices from different sources as much as possible; because in practical applications, the images to be identified may be of unknown devices or images that have been attacked by device replacement. At this time, the dual-view feature encoder trained using the embodiment of this specification can more accurately distinguish between different models or the same model but different device individuals.

[0092] The dual-view feature encoder provided in the embodiment of this specification first uses a closed-set multi-classification method to pre-train the model during training to accelerate convergence, and then uses a contrastive learning framework to impose distance constraints on the source identification features of sample images of the same class and different classes, shortening the distance between sample images of the same class and the source device, and increasing the distance between sample images of different classes and the source device, and increasing the weights of sample pairs that are more difficult to distinguish, so that the source identification features of the template image and the source identification distance of the image to be tested can be used to accurately obtain the multi-classification results of a certain image relative to multiple source devices, as well as the binary classification results of the matching of a certain image relative to a certain source device.

[0093] In specific implementation, after the dual-view feature encoder is trained, the device source feature of each source device in the device template library and the picture source feature of the picture to be identified can be determined quickly and accurately according to the trained dual-view feature encoder. The specific implementation is as follows:

[0094] The determining of the device source feature of each source device in the device template library and the picture source feature of the picture to be identified according to the dual-view feature encoder includes:

[0095] The determining of the device source feature of each source device in the device template library according to the dual-view feature encoder includes:

[0096] The determining of the picture source feature of the picture to be identified according to the dual-view feature encoder.

[0097] The dual-view feature encoder is the dual-view feature encoder trained above.

[0098] Specifically, the device source feature of each source device in the device template library can be obtained by the dual-view feature encoder, and the picture source feature of the picture to be identified can also be determined.

[0099] The determining of the device source feature of each source device in the device template library according to the dual-view feature encoder and the determining of the picture source feature of the picture to be identified according to the dual-view feature encoder are described in detail below.

[0100] First, the determining of the device source feature of each source device in the device template library according to the dual-view feature encoder is explained. The specific implementation is as follows:

[0101] The determining of the device source feature of each source device in the device template library according to the dual-view feature encoder includes:

[0102] The at least one device photographed picture corresponding to each source device in the device template library is determined.

[0103] The picture source feature of each device photographed picture is obtained according to the dual-view feature encoder.

[0104] The picture source features of all the device photographed pictures corresponding to each source device are clustered to obtain the device source feature of each source device.

[0105] The device template library includes a plurality of device photographed pictures photographed by a plurality of source devices.

[0106] Still using the above example, when the device is a mobile phone, first determine the multiple device-taken pictures corresponding to each mobile phone in the mobile phone template library; then, based on the dual-view feature encoder, obtain the picture source features of the pictures taken by each device; finally, cluster the source features of all device-taken pictures corresponding to each mobile phone to obtain the device source features of each mobile phone.

[0107] In actual applications, the device source characteristics of each source device can also be stored in the device template library, so that it can be accessed at any time during application without the need to process the device source characteristics of each source device in real time according to the dual-view feature encoder.

[0108] In the embodiment of this specification, the image source characteristics of the device-taken pictures corresponding to each source device in the device template library can be obtained based on the dual-view feature encoder, and finally clustering is performed based on the image source characteristics of the device-taken pictures corresponding to each source device to accurately obtain the device source characteristics of each source device.

[0109] The step of obtaining the image source features of the images taken by each device according to the dual-view feature encoder includes:

[0110] Obtaining, according to a preset noise extraction method, a second imaging fingerprint noise of the picture taken by each device;

[0111] Performing feature fusion on the picture taken by each device and the second imaging fingerprint noise to obtain fusion features of the picture taken by each device;

[0112] The fusion features of the pictures taken by each device are input into the dual-view feature encoder to obtain the picture source features of the pictures taken by each device.

[0113] Among them, the specific implementation method of obtaining the second imaging fingerprint noise of the picture taken by each device according to the preset noise extraction method is consistent with the method of obtaining the third imaging fingerprint noise of each sample picture according to the preset noise extraction method during the training of the above-mentioned dual-view feature encoder, and will not be repeated here.

[0114] In the specific implementation, first, according to the preset noise extraction method, the second imaging fingerprint noise of the picture taken by each device is obtained, and then the picture taken by each device and its corresponding second imaging fingerprint noise are feature fused to obtain the fusion feature of the picture taken by each device; finally, the fusion feature of the picture taken by each device is input into the dual-view feature encoder to obtain the picture source feature of the picture taken by each device.

[0115] The method of determining the image source characteristics of the image to be identified based on the dual-view feature encoder is the same as the method of obtaining the image source characteristics of the image taken by each device based on the dual-view feature encoder. The specific implementation method is as follows:

[0116] The step of determining the image source feature of the image to be identified according to the dual-view feature encoder includes:

[0117] Obtaining a first imaging fingerprint noise of the image to be identified according to a preset noise extraction method;

[0118] Performing feature fusion on the image to be identified and the first imaging fingerprint noise to obtain a fusion feature of the image to be identified;

[0119] The fusion features of the image to be identified are input into a dual-view feature encoder to obtain the image source features of the image to be identified.

[0120] In the specific implementation, the first imaging fingerprint noise of the image to be identified is first obtained according to the preset noise extraction method, and then the image to be identified and its corresponding first imaging fingerprint noise are feature fused to obtain the fusion features of each image to be identified; finally, the fusion features of the image to be identified are input into the dual-view feature encoder to obtain the image source features of the image to be identified.

[0121] See also Figure 3 , Figure 3 A flowchart of obtaining source identification features in a method for identifying the source of an image provided by an embodiment of this specification is shown.

[0122] by Figure 3 The input image in is an example of a picture to be identified.

[0123] Step 302: Determine the image to be identified.

[0124] Step 304: performing noise extraction on the image to be identified by a noise extraction module to obtain the camera imaging fingerprint noise of the image to be identified.

[0125] Step 306: performing feature fusion on the image to be identified and the camera imaging fingerprint noise of the image to be identified to obtain a fusion feature.

[0126] Step 308: Input the fused features into the dual-view feature encoder.

[0127] Step 310: Obtain the traceability features of the image to be identified output by the dual-view feature encoder.

[0128] Among them, the traceability feature can be understood as the source feature of the image.

[0129] In actual application, the sample pictures can or can not be post-processed during the training of the dual-view feature encoder. In order to ensure that the trained dual-view feature encoder can still accurately identify the picture source feature of the device photographed picture without post-processing of the sample pictures, i.e. in the case of training of the dual-view feature encoder, the device photographed picture needs to be post-processed and then the picture source feature is obtained through the dual-view feature encoder. The specific implementation is as follows:

[0130] After determining the at least one device photographed picture corresponding to each source device in the device template library, the method further includes:

[0131] Post-processing each device photographed picture to obtain a processed device photographed picture, wherein the post-processing includes enlarging, reducing and / or compressing the device photographed picture.

[0132] Similarly, in order to ensure that the trained dual-view feature encoder can still accurately identify the picture source feature of the to-be-identified picture without post-processing of the sample pictures, i.e. in the case of training of the dual-view feature encoder, the to-be-identified picture needs to be post-processed and then the picture source feature is obtained through the dual-view feature encoder. The specific implementation is as follows:

[0133] Before determining the device source feature of each source device in the device template library and the picture source feature of the to-be-identified picture according to the dual-view feature encoder, the method further includes:

[0134] Post-processing the to-be-identified picture to obtain a processed to-be-identified picture, wherein the post-processing includes enlarging, reducing and / or compressing the to-be-identified picture.

[0135] Step 206: calculating the similarity between the picture source feature of the to-be-identified picture and the device source feature of each source device, and determining the target source device of the to-be-identified picture from the device template library according to the similarity.

[0136] Specifically, in the case of determining the picture source feature of the to-be-identified picture and the device source feature of each source device according to the above embodiment, the similarity between the picture source feature of the to-be-identified picture and the device source feature of each source device is calculated, and the target source device of the to-be-identified picture is determined from the device template library according to the similarity.

[0137] For example, the source devices in the device template library are sorted in descending order according to the similarity, and the source device with the highest similarity ranked first is taken as the target source device of the to-be-identified picture.

[0138] The image source identification method provided in the embodiments of this specification uses a dual-view feature encoder to jointly extract the image source features of the image to be identified in the RGB view and the noise view, and matches the image source features with the device source features of the devices in the device template library to achieve identification of the target source device (such as a mobile phone or camera, etc.) corresponding to the image to be identified, with high recognition efficiency and accuracy.

[0139] See also Figure 4 , Figure 4 A flowchart of another method for identifying the source of an image provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0140] Step 402: In response to the picture source identification request, extract the picture to be identified and the initial source device in the picture source identification request.

[0141] Similar to the above embodiment, the image to be identified can also be understood as being extracted from three-dimensional video data captured by a VR (virtual reality) device or an AR (augmented reality) device. The specific implementation method is as follows:

[0142] The step of extracting, in response to the image source identification request, the image to be identified in the image source identification request, includes:

[0143] In response to the image source identification request, determining video data carried in the image source identification request, wherein the video data includes video data obtained through a VR device or an AR device;

[0144] A target video frame is extracted from the video data, and the target video frame is used as a picture to be identified.

[0145] In specific implementation, the sources of the images to be identified are different, and their corresponding source devices are also different. For example, when the image to be identified is a target video frame extracted from video data obtained from a VR device or an AR device, the image source identification request in the embodiment of this specification can be parsed as a request to determine the target VR device or AR device of the image to be identified. Then, the technical problem to be solved by the embodiment of this specification is to determine which VR device or AR device is the source device of the image to be identified.

[0146] Step 404: Determine the image source feature of the image to be identified and the device source feature of the initial source device according to the dual-view feature encoder.

[0147] The dual-view feature encoder is obtained by training the image view and the noise view.

[0148] Specifically, the dual-view feature encoder in the embodiment of this specification and the specific implementation method of obtaining the image source feature of the image to be identified based on the dual-view feature encoder are the same as those in the above embodiment. The details are as follows:

[0149] The determining, based on the dual-view feature encoder, the picture source feature of the picture to be identified and the device source feature of the initial source device includes:

[0150] Determining the image source feature of the image to be identified according to the dual-view feature encoder; and

[0151] A device source feature of the initial source device is determined according to the dual-view feature encoder.

[0152] The method for obtaining the image source feature of the image to be identified and the device source feature of the original source device according to the dual-view feature encoder is also the same as that in the above embodiment. The details are as follows:

[0153] The step of determining the image source feature of the image to be identified based on the dual-view feature encoder includes:

[0154] Obtaining a first imaging fingerprint noise of the image to be identified according to a preset noise extraction method;

[0155] Performing feature fusion on the image to be identified and the first imaging fingerprint noise to obtain a fusion feature of the image to be identified;

[0156] The fusion features of the image to be identified are input into a dual-view feature encoder to obtain the image source features of the image to be identified.

[0157] At the same time, determining the device source feature of the initial source device according to the dual-view feature encoder includes:

[0158] Obtaining a device-photographed picture corresponding to the initial source device;

[0159] Obtaining, according to the dual-view feature encoder, a picture source feature of a picture taken by a device corresponding to the initial source device;

[0160] The picture source features of the pictures taken by the device corresponding to the initial source device are clustered to obtain the device source features of the initial source device.

[0161] Step 406: Calculate the similarity between the image source feature of the image to be identified and the device source feature of the initial source device, and determine whether the initial source device is the target source device of the image to be identified based on the similarity.

[0162] Specifically, after determining whether the initial source device is the target source device of the image to be identified according to the similarity, the method further includes:

[0163] If the similarity is greater than or equal to a preset similarity threshold, determining the initial source device as the target source device of the image to be identified; or

[0164] When the similarity is less than the preset similarity threshold, it is determined that the initial source device is not the target source device of the image to be identified.

[0165] The preset similarity threshold can be set according to actual application, and this specification does not impose any limitation on this. For example, the preset similarity threshold can be set to 80% or 90%.

[0166] Specifically, after obtaining the picture source characteristics of the picture to be identified and the device source characteristics of the initial source device, the similarity between the picture source characteristics of the picture to be identified and the device source characteristics of the initial source device is calculated. When the similarity is greater than or equal to a preset similarity threshold, the initial source device is determined as the target source device of the picture to be identified; conversely, when the similarity is less than the preset similarity threshold, it is determined that there is no corresponding relationship between the initial source device and the picture to be identified; for example, it can be understood that the picture to be identified is not a picture taken by a mobile phone or camera.

[0167] The image source identification method provided in the embodiments of this specification uses a dual-view feature encoder to jointly extract the image source features of the image to be identified and the device source features of the initial source device in the RGB view and the noise view, and calculates the similarity between the image source features and the device source features to determine whether the image to be identified comes from the initial source device, thereby achieving high recognition efficiency and accuracy.

[0168] The following combined Figure 5 , taking the application of the image source identification method provided in this specification in binary classification and multi-classification scenarios as an example, the image source identification method is further explained. Figure 5 A flowchart of a processing process of a method for identifying the source of an image provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0169] Step 502: Receive a picture source identification request, and determine the mobile phone image carried in the picture source identification request.

[0170] In actual applications, the image source identification request may include two situations. One is to determine whether the source device of the mobile phone image is the target mobile phone. In this case, the target mobile phone is also carried in the image source identification request; the other is to obtain the source device of the mobile phone image when the target mobile phone does not exist. In this case, the mobile phone template library can be determined based on the image source identification request.

[0171] Step 504: Parse the image source identification request to determine the template image of the target mobile phone, or the template image of each mobile phone in the mobile phone template library.

[0172] Step 506: The mobile phone image is post-processed through a post-processing link.

[0173] Step 508: Input the post-processed mobile phone imaging picture into the dual-view feature encoder.

[0174] Step 510: Input the template image of the target mobile phone or the template image of each mobile phone in the mobile phone template library into the dual-view feature encoder.

[0175] Step 512: Obtain the traceability features of the mobile phone imaging image output by the dual-view feature encoder.

[0176] Step 514: Obtain the template features of the target mobile phone output by the dual-view feature encoder or the template features of each mobile phone in the mobile phone template library.

[0177] Step 516: perform similarity matching on the tracing feature and the template feature of the target mobile phone, or perform feature distance matching on the tracing feature and the template feature of each mobile phone.

[0178] Step 518: When the similarity between the traceability feature and the template feature of the target mobile phone is greater than or equal to a preset similarity threshold, it is determined that the source device of the mobile phone imaging image is the target mobile phone.

[0179] That is, it can be understood that the image imaged by the mobile phone is determined to be taken by the target mobile phone, such as the 1V1 matching in the figure.

[0180] Step 520: The device with the closest characteristic distance is used as the source device of the mobile phone imaging image.

[0181] This can be understood as the 1VN traceability in the figure.

[0182] The image source identification method provided in the embodiments of this specification adopts dual-view feature learning and camera source identification based on a multi-classification pre-training and comparative learning framework. In the feature learning part, traditional methods are used to extract sensor noise features and deep learning methods are used to extract the post-photography processing algorithm pattern features of various mobile phone manufacturers. The two are combined for dual-view feature learning to enhance the model's ability to extract source identification features. At the same time, during training, the training data is processed using a post-processing link commonly deployed online. Compared with the existing classification scheme based on the PRNU feature distance of camera sensor noise, this scheme can extract the pattern features of the image quality optimization algorithms developed by various mobile phone manufacturers. It does not rely solely on the easily destructive camera sensor noise, is robust to the existing post-processing link, and the data required for source identification does not impose additional load on the online link. For camera source identification based on multi-classification pre-training and contrastive learning framework, a closed-set multi-classification framework is used to pre-train the dual-view feature encoder, enhancing the model's ability to extract source identification features and accelerating network convergence. The contrastive learning framework is used to constrain the distance of source identification features, and the distance constraints are imposed on source identification features of the same class and different classes, shortening the distance of the same class and increasing the distance of different classes. The weights of sample pairs that are more difficult to distinguish are also increased. After deployment, the distance and clustering of the source identification features of the template image and the source identification features of the test image are used to give the following: multi-classification results of 1vN source identification and binary classification results of 1v1 matching. Compared with traditional multi-classification-based deep learning methods, the extracted features are more concentrated in space by category, with larger inter-class distances, better generalization for unseen classes, and easier to fine-tune using pre-trained models when new camera types are added later.

[0183] Corresponding to the above method embodiment, this specification also provides an embodiment of a device for identifying the source of an image. Figure 6 FIG1 shows a schematic diagram of a structure of a device for identifying the source of an image provided by an embodiment of this specification. Figure 6 As shown, the device includes:

[0184] The request receiving module 602 is configured to, in response to a picture source identification request, extract a picture to be identified in the picture source identification request and determine a device template library;

[0185] a feature determination module 604 configured to determine a device source feature of each source device in the device template library and an image source feature of the image to be identified based on a dual-view feature encoder, wherein the dual-view feature encoder is obtained by training with image view and noise view;

[0186] The source determination module 606 is configured to calculate the similarity between the image source feature of the image to be identified and the device source feature of each source device, and determine the target source device of the image to be identified from the device template library based on the similarity.

[0187] Optionally, the device further includes:

[0188] The first processing module is configured to:

[0189] Post-processing is performed on the image to be identified to obtain a processed image to be identified, wherein the post-processing includes enlarging, reducing and / or compressing the image to be identified.

[0190] Optionally, the feature determination module 604 is further configured to:

[0191] determining, according to the dual-view feature encoder, a device source feature of each source device in the device template library; and

[0192] According to the dual-view feature encoder, the picture source feature of the picture to be identified is determined.

[0193] Optionally, the feature determination module 604 is further configured to:

[0194] Determining at least one device-captured picture corresponding to each source device in the device template library;

[0195] Obtaining, according to the dual-view feature encoder, picture source features of pictures taken by each device;

[0196] Cluster the picture source features of all the pictures taken by the devices corresponding to each source device to obtain the device source feature of each source device.

[0197] Optionally, the feature determination module 604 is further configured to:

[0198] Obtaining, according to a preset noise extraction method, a second imaging fingerprint noise of the picture taken by each device;

[0199] Performing feature fusion on the picture taken by each device and the second imaging fingerprint noise to obtain fusion features of the picture taken by each device;

[0200] The fusion features of the pictures taken by each device are input into the dual-view feature encoder to obtain the picture source features of the pictures taken by each device.

[0201] Optionally, the feature determination module 604 is further configured to:

[0202] Obtaining a first imaging fingerprint noise of the image to be identified according to a preset noise extraction method;

[0203] Performing feature fusion on the image to be identified and the first imaging fingerprint noise to obtain a fusion feature of the image to be identified;

[0204] input the fusion feature of the picture to be identified into the dual-view feature encoder to obtain a picture source feature of the picture to be identified.

[0205] Optionally, the feature determination module 604 is further configured to:

[0206] The training step of the dual-view feature encoder includes:

[0207] obtain sample pictures taken by a plurality of source devices, and determine a source device corresponding to each sample picture;

[0208] obtain a third imaging fingerprint noise of each sample picture according to a preset noise extraction method;

[0209] fuse the third imaging fingerprint noise with each sample picture to obtain a fusion feature of each sample picture;

[0210] train a dual-view feature encoder according to the fusion feature of each sample picture and a device source feature of the source device corresponding to each sample picture.

[0211] Optionally, the feature determination module 604 is further configured to:

[0212] use the fusion feature of each sample picture as a training sample, and use the device source feature of the source device corresponding to each sample picture as a training label of the training sample;

[0213] train a first spatial distance between training samples of the same device model and a second spatial distance between training samples of different device models according to a preset feature distance constraint method;

[0214] train a dual-view feature encoder according to the training sample, the training label, the first spatial distance, and the second spatial distance.

[0215] Optionally, the apparatus further includes:

[0216] a second processing module configured to:

[0217] perform post-processing on each device-taken picture to obtain a processed device-taken picture, wherein the post-processing includes enlarging, reducing, and / or compressing each device-taken picture.

[0218] Optionally, the apparatus further includes:

[0219] a third processing module configured to:

[0220] Post-processing is performed on each sample image to obtain each processed sample image, wherein the post-processing includes enlarging, reducing and / or compressing each sample image.

[0221] Optionally, the request receiving module 602 is further configured to:

[0222] In response to the image source identification request, determining video data carried in the image source identification request, wherein the video data includes video data obtained through a VR device or an AR device;

[0223] A target video frame is extracted from the video data, and the target video frame is used as a picture to be identified.

[0224] The image source identification device provided in the embodiment of this specification uses a dual-view feature encoder to jointly extract the image source features of the image to be identified in the RGB view and the noise view, and matches the image source features with the device source features of the devices in the device template library to achieve identification of the target source device (such as a mobile phone or camera, etc.) corresponding to the image to be identified, with high recognition efficiency and accuracy.

[0225] The above is a schematic diagram of an apparatus for identifying the source of an image according to this embodiment. It should be noted that the technical solution of this apparatus for identifying the source of an image is based on the same concept as the technical solution of the aforementioned method for identifying the source of an image. For details not described in detail in the technical solution of the apparatus for identifying the source of an image, please refer to the description of the technical solution of the aforementioned method for identifying the source of an image.

[0226] Corresponding to the above method embodiment, this specification also provides an embodiment of a device for identifying the source of an image. Figure 7 FIG. 1 shows a schematic diagram of another image source identification device provided by an embodiment of this specification. Figure 7 As shown, the device includes:

[0227] The information extraction module 702 is configured to extract the image to be identified and the initial source device in response to the image source identification request;

[0228] a source feature determination module 704 configured to determine the image source feature of the image to be identified and the device source feature of the initial source device based on a dual-view feature encoder, wherein the dual-view feature encoder is obtained by training the image view and the noise view;

[0229] The source device determination module 706 is configured to calculate the similarity between the image source feature of the image to be identified and the device source feature of the initial source device, and determine whether the initial source device is the target source device of the image to be identified based on the similarity.

[0230] Optionally, the source feature determination module 704 is further configured to:

[0231] Determining the image source feature of the image to be identified according to the dual-view feature encoder; and

[0232] A device source feature of the initial source device is determined according to the dual-view feature encoder.

[0233] Optionally, the source feature determination module 704 is further configured to:

[0234] Obtaining a first imaging fingerprint noise of the image to be identified according to a preset noise extraction method;

[0235] Performing feature fusion on the image to be identified and the first imaging fingerprint noise to obtain a fusion feature of the image to be identified;

[0236] The fusion features of the image to be identified are input into a dual-view feature encoder to obtain the image source features of the image to be identified.

[0237] Optionally, the source feature determination module 704 is further configured to:

[0238] Obtaining a device-photographed picture corresponding to the initial source device;

[0239] Obtaining, according to the dual-view feature encoder, a picture source feature of a picture taken by a device corresponding to the initial source device;

[0240] The picture source features of the pictures taken by the device corresponding to the initial source device are clustered to obtain the device source features of the initial source device.

[0241] The image source identification device provided in the embodiments of this specification uses a dual-view feature encoder to jointly extract the image source features of the image to be identified and the device source features of the initial source device in the RGB view and the noise view, and calculates the similarity between the image source features and the device source features to determine whether the image to be identified comes from the initial source device, thereby achieving high recognition efficiency and accuracy.

[0242] The above is a schematic diagram of an apparatus for identifying the source of an image according to this embodiment. It should be noted that the technical solution of this apparatus for identifying the source of an image is based on the same concept as the technical solution of the aforementioned method for identifying the source of an image. For details not described in detail in the technical solution of the apparatus for identifying the source of an image, please refer to the description of the technical solution of the aforementioned method for identifying the source of an image.

[0243] Figure 8 8 shows a block diagram of a computing device 800 according to one embodiment of the present disclosure. Components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0244] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)), whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0245] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 8 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0246] The computing device 800 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 800 can also be a mobile or stationary server.

[0247] The processor 820 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned image source identification method.

[0248] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the aforementioned image source identification method are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the aforementioned image source identification method.

[0249] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned image source identification method.

[0250] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solution of the aforementioned image source identification method. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the aforementioned image source identification method.

[0251] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned image source identification method.

[0252] The above is an illustrative embodiment of a computer program. It should be noted that the technical solution of this computer program and the technical solution of the above-mentioned image source identification method are based on the same concept. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solution of the above-mentioned image source identification method.

[0253] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0254] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0255] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0256] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0257] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for identifying the source of an image, comprising: In response to a request for identifying a source of an image, extracting a to-be-identified image from the request and determining a device template library; Determining, according to a dual-view feature encoder, a device source feature of each source device in the device template library and a picture source feature of the picture to be identified, wherein the input of the dual-view feature encoder is a picture fusion feature obtained by fusing the picture with the sensor noise of the source device, and the dual-view feature encoder is obtained by training the picture perspective and the noise perspective, wherein the training includes using the fusion feature of each sample picture as a training sample and using the device source feature of the source device corresponding to each sample picture as a training label for the training sample; The image source feature of the image to be identified and the similarity between the feature of the device source of each source device are calculated, and the target source device of the image to be identified is determined from the device template library according to the similarity.

2. The image source identification method according to claim 1, before determining the device source feature of each source device in the device template library and the image source feature of the image to be identified based on the dual-view feature encoder, further comprising: Post-processing is performed on the image to be identified to obtain a processed image to be identified, wherein the post-processing includes enlarging, reducing and / or compressing the image to be identified.

3. The image source identification method according to claim 1 or 2, wherein determining the device source feature of each source device in the device template library and the image source feature of the image to be identified based on the dual-view feature encoder comprises: determining, according to a dual-view feature encoder, a device source feature of each source device in the device template library; as well as According to the dual-view feature encoder, the picture source feature of the picture to be identified is determined.

4. The image source identification method according to claim 3, wherein determining the device source feature of each source device in the device template library based on the dual-view feature encoder comprises: Determining at least one device-captured picture corresponding to each source device in the device template library; Obtaining, according to the dual-view feature encoder, picture source features of pictures taken by each device; Cluster the picture source features of all the pictures taken by the devices corresponding to each source device to obtain the device source feature of each source device.

5. The image source identification method according to claim 4, wherein obtaining the image source features of each image taken by the device according to the dual-view feature encoder comprises: Obtaining, according to a preset noise extraction method, a second imaging fingerprint noise of the picture taken by each device; Performing feature fusion on the picture taken by each device and the second imaging fingerprint noise to obtain fusion features of the picture taken by each device; The fusion features of the pictures taken by each device are input into the dual-view feature encoder to obtain the picture source features of the pictures taken by each device.

6. The image source identification method according to claim 3, wherein determining the image source feature of the image to be identified based on the dual-view feature encoder comprises: Obtaining a first imaging fingerprint noise of the image to be identified according to a preset noise extraction method; Performing feature fusion on the image to be identified and the first imaging fingerprint noise to obtain a fusion feature of the image to be identified; The fusion features of the image to be identified are input into a dual-view feature encoder to obtain the image source features of the image to be identified.

7. The image source identification method according to claim 1, wherein the step of training the dual-view feature encoder comprises: Obtain sample images taken by multiple source devices and determine the source device corresponding to each sample image; Obtaining a third imaging fingerprint noise of each sample image according to a preset noise extraction method; Performing feature fusion on each sample image and the third imaging fingerprint noise to obtain a fusion feature of each sample image; A dual-view feature encoder is trained based on the fusion features of each sample image and the device source features of the source device corresponding to each sample image.

8. The image source identification method according to claim 7, wherein the step of training a dual-view feature encoder based on the fusion features of each sample image and the device source features of the source device corresponding to each sample image comprises: According to a preset feature distance constraint method, training the first spatial distance between training samples of the same device model and the second spatial distance between training samples of different device models; A dual-view feature encoder is trained according to the training samples, the training labels, the first spatial distance, and the second spatial distance.

9. The image source identification method according to claim 4, after determining that at least one device corresponding to each source device in the device template library has taken the image, further comprising: Post-processing is performed on the pictures taken by each device to obtain processed pictures taken by each device, wherein the post-processing includes enlarging, reducing and / or compressing the pictures taken by each device.

10. The image source identification method according to claim 1, wherein, in response to an image source identification request, extracting the image to be identified in the image source identification request comprises: In response to the image source identification request, determining video data carried in the image source identification request, wherein the video data includes video data obtained through a VR device or an AR device; A target video frame is extracted from the video data, and the target video frame is used as a picture to be identified.

11. A method for identifying the source of an image, comprising: In response to a picture source identification request, extracting the picture to be identified and the initial source device in the picture source identification request; Determining, according to a dual-view feature encoder, a picture source feature of the picture to be identified and a device source feature of the initial source device, wherein the input of the dual-view feature encoder is a picture fusion feature obtained by fusing the picture with sensor noise of the source device, and the dual-view feature encoder is obtained by training the picture perspective and the noise perspective, wherein the training includes using the fusion feature of each sample picture as a training sample and using the device source feature of the source device corresponding to each sample picture as a training label for the training sample; Calculate the similarity between the image source feature of the image to be identified and the device source feature of the initial source device, and determine whether the initial source device is the target source device of the image to be identified based on the similarity.

12. The image source identification method according to claim 11, wherein determining the image source feature of the image to be identified and the device source feature of the initial source device based on the dual-view feature encoder comprises: Determining the image source features of the image to be identified according to the dual-view feature encoder; as well as A device source feature of the initial source device is determined according to the dual-view feature encoder.

13. The image source identification method according to claim 12, wherein determining the image source feature of the image to be identified based on the dual-view feature encoder comprises: Obtaining a first imaging fingerprint noise of the image to be identified according to a preset noise extraction method; Performing feature fusion on the image to be identified and the first imaging fingerprint noise to obtain a fusion feature of the image to be identified; The fusion features of the image to be identified are input into a dual-view feature encoder to obtain the image source features of the image to be identified.

14. The image source identification method according to claim 12, wherein determining the device source feature of the initial source device based on the dual-view feature encoder comprises: Obtaining a device-photographed picture corresponding to the initial source device; Obtaining, according to the dual-view feature encoder, a picture source feature of a picture taken by a device corresponding to the initial source device; The picture source features of the pictures taken by the device corresponding to the initial source device are clustered to obtain the device source features of the initial source device.

Images