Image processing method, device and computer equipment

Abstract

The application discloses an image processing method and device and computer equipment. The method comprises the following steps: obtaining a target image; using an image recognition model to identify the source of the target image to obtain source information of the target image, wherein the image recognition model is trained based on multiple sets of sample images, and the multiple sets of sample images comprise sample images and source information of the sample images; and the image recognition model takes pre-extracted original image source information as a training target during the training process, and the original image source information is source information corresponding to an original image of the sample image. The application solves the technical problem of low image source recognition accuracy caused by poor extraction accuracy when extracting source information of an image in the related art, and achieves the technical effect of accurately identifying a camera that captures a commodity image and more accurately determining whether a commodity image is a stolen image when identifying the commodity image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition, and in particular, to an image processing method and device and computer equipment. BACKGROUND

[0002] In recent years, with the continuous development of hardware and software technology, the threshold for image creation and editing has also been lowered, and the authenticity and integrity of images are therefore difficult to guarantee. In the related art, the image source recognition is usually performed by directly extracting image source information (for example, light response non-uniformity information) from the image using a denoising algorithm and classifying. For example, the light response non-uniformity information and noise residual are extracted from the image, and a convolutional neural network is used to identify the source information of the image. However, the above two methods have defects when extracting the image source information. For example, the former directly extracts the light response non-uniformity information from the image that has been compressed, transmitted, and the like, and the accuracy of the image source recognition result obtained according to the light response non-uniformity information is poor, and the latter has the shortcomings of poor extraction effect of the light response non-uniformity information and noise residual and poor stability of the recognition result.

[0003] Therefore, in the related art, when the source information of the image is extracted, there is a problem of low accuracy of image source recognition caused by poor extraction accuracy.

[0004] At present, no effective solution has been proposed to solve the above problems. SUMMARY

[0005] Embodiments of the present application provide an image processing method, device and computer equipment to at least solve the technical problem of low accuracy of image source recognition caused by poor extraction accuracy when the source information of the image is extracted in the related art.

[0006] According to an aspect of an embodiment of the present application, an image processing method is provided, comprising: obtaining a target image; performing source recognition on the target image using an image recognition model to obtain source information of the target image, wherein the image recognition model is obtained based on a plurality of sets of sample images, and the plurality of sets of sample images comprise sample images and source information of the sample images, wherein the image recognition model takes pre-extracted original image source information as a training target in the training process, and the original image source information is source information corresponding to an original image of the sample image.

[0007] Optionally, performing source recognition on the target image using the image recognition model to obtain the source information of the target image comprises: performing source recognition on the target image using an autoencoder of the image recognition model to obtain estimated source information; and performing classification processing on the estimated source information using a classifier of the image recognition model to obtain the source information of the target image.

[0008] Optionally, the source of the target image is identified by using a self-encoder of an image recognition model to obtain estimated source information, including: in an encoder of the self-encoder of the image recognition model, a convolutional layer is used to extract features of the target image; in a decoder of the self-encoder of the image recognition model, an inverse convolutional layer is used to reconstruct an image from the extracted features to obtain the estimated source information.

[0009] Optionally, the estimated source information is classified by using a classifier of the image recognition model to obtain the source information of the target image, including: in the classifier of the image recognition model, a multi-layer convolutional layer is used to process the estimated source information to obtain a multi-layer convolutional processing result; in the classifier of the image recognition model, a cascade operation is used on the multi-layer convolutional processing result to obtain the source information of the target image.

[0010] Optionally, the method further includes: obtaining original image source information of the sample image by using a predetermined denoising method, wherein the original image source information includes light response non-uniformity information of the sample image.

[0011] According to another aspect of the embodiment of the present application, an image processing method is also provided, including: obtaining a plurality of sets of sample image data, wherein the plurality of sets of sample image data include sample images and source information of the sample images; respectively extracting original image source information of the sample images in the plurality of sets of sample image data, the original image source information being source information corresponding to an original image of the sample image; and using the plurality of sets of sample image data for machine training, and taking the original image source information of the sample image as a training target in the training process to obtain an image recognition model.

[0012] Optionally, the plurality of sets of sample image data are used for machine training, and the original image source information of the sample image is taken as a training target in the training process to obtain the image recognition model, including: inputting the sample images in the plurality of sets of sample image data into an initial self-encoder to obtain estimated source information; inputting the estimated source information into an initial classifier to obtain classified source information; constructing a first loss function based on the estimated source information and the original image source information, and constructing a second loss function based on the classified source information and the source information of the sample image; constructing a target loss function based on the first loss function and the second loss function; and using the plurality of sets of sample image data for machine training by minimizing the target loss function to obtain the image recognition model.

[0013] Optionally, the target loss function is constructed based on the first loss function and the second loss function, including: determining a first weight of the first loss function and a second weight of the second loss function; multiplying the first loss function by the first weight to obtain a first value, and multiplying the second loss function by the second weight to obtain a second value; and summing the first value and the second value to obtain the target loss function.

[0014] According to another aspect of the embodiments of the present application, there is also provided an image processing method, comprising: displaying an import control on an interactive interface; in response to an import operation on the import control, importing a target image; and in response to an identification operation on an identification control on the interactive interface, displaying an identification result of the target image on the interactive interface, wherein the identification result comprises source information of the target image, the source information of the target image is obtained based on an image identification model, the image identification model is obtained based on training of a plurality of sets of sample image data, the plurality of sets of sample image data comprises: a sample image and source information of the sample image, and the image identification model takes pre-extracted original image source information as a training target in a training process, the original image source information is source information corresponding to an original image of the sample image.

[0015] According to another aspect of the embodiments of the present application, there is also provided an image processing method, comprising: displaying a plurality of sets of sample image data on an interactive interface, wherein the plurality of sets of sample image data comprises: a sample image and source information of the sample image; displaying original image source information of the sample image in the plurality of sets of sample image data on the interactive interface, the original image source information is pre-extracted source information corresponding to an original image of the sample image; and displaying a model icon of an image identification model on the interactive interface, wherein the image identification model is trained by using the plurality of sets of sample image data, and the image identification model takes the original image source information of the sample image as a training target in a training process.

[0016] According to another aspect of the embodiments of the present application, there is also provided an image processing method, comprising: obtaining a commodity image; and identifying a source of the commodity image by using an image identification model to obtain source information of the commodity image, wherein the source information comprises a camera that captures the commodity image, the image identification model is obtained based on training of a plurality of sets of sample image data, the plurality of sets of sample image data comprises: a sample image and source information of the sample image, and the image identification model takes pre-extracted light response non-uniformity information of an original image of the sample image as a training target in a training process.

[0017] According to another aspect of the embodiments of the present application, there is also provided an image processing device, comprising: a first obtaining module configured to obtain a target image; and an identification module configured to identify a source of the target image by using an image identification model to obtain source information of the target image, wherein the image identification model is obtained based on training of a plurality of sets of sample image data, the plurality of sets of sample image data comprises: a sample image and source information of the sample image, and the image identification model takes pre-extracted original image source information as a training target in a training process, the original image source information is source information corresponding to an original image of the sample image.

[0018] According to another aspect of the embodiments of the present application, there is also provided an image processing apparatus, comprising: a second obtaining module, configured to obtain a plurality of sets of sample image data, wherein the plurality of sets of sample image data comprises sample images and source information of the sample images; an extracting module, configured to extract original image source information of the sample images in the plurality of sets of sample image data respectively, wherein the original image source information is source information corresponding to original images of the sample images; and a training module, configured to perform machine training by using the plurality of sets of sample image data, and obtain an image recognition model by taking the original image source information of the sample images as a training target in the training process.

[0019] According to another aspect of the embodiments of the present application, there is also provided an image processing apparatus, comprising: a third obtaining module, configured to obtain a product image; and a second identifying module, configured to perform source identification on the product image by using an image recognition model, and obtain source information of the product image, wherein the source information comprises a camera used to capture the product image, and the image recognition model is obtained by training based on a plurality of sets of sample image data, wherein the plurality of sets of sample image data comprises sample images and source information of the sample images, and the image recognition model is trained based on pre-extracted original image light response non-uniformity information of the sample images as a training target in the training process.

[0020] According to another aspect of the embodiments of the present application, there is also provided a computer device, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program stored in the memory, and the computer program is configured to enable the processor to execute any of the image processing methods described above when running.

[0021] According to another aspect of the embodiments of the present application, there is also provided a computer readable storage medium, which enables an electronic device to execute any of the image processing methods described above when an instruction in the computer readable storage medium is executed by a processor of the electronic device.

[0022] In the embodiments of the present application, the image recognition model is used to perform source identification on the target image, and the image recognition model is trained based on pre-extracted original image source information as a training target in the training process. Therefore, the image recognition is effectively prevented from being affected by image post-processing, the image source identification is more accurate, and the accuracy of the source information identification of the image is effectively improved when the image recognition model is used to identify the target image, thereby solving the technical problem of low image source identification accuracy caused by low extraction accuracy when extracting the source information of the image in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application. In the drawings:

[0024] Figure 1 A hardware structure block diagram of a computer terminal for implementing the image processing method is shown;

[0025] Figure 2 is a flowchart of the image processing method one according to the embodiment 1 of the present application;

[0026] Figure 3 is a flowchart of the image processing method two according to the embodiment 1 of the present application;

[0027] Figure 4 is a flowchart of the image processing method three according to the embodiment 1 of the present application;

[0028] Figure 5 is a flowchart of the image processing method four according to the embodiment 1 of the present application;

[0029] Figure 6 is a flowchart of the image processing method five according to the embodiment 1 of the present application;

[0030] Figure 7 is a classification schematic diagram of the image source identification problem provided by the optional embodiment of the present application;

[0031] Figure 8 is a schematic diagram of tampering with Make and Model in the image Exif data using ExifTool listed in the optional embodiment of the present application;

[0032] Figure 9 is a schematic diagram of the image processing method provided by the embodiment of the present application;

[0033] Figure 10 is a structural schematic diagram of the U-Net provided by the optional embodiment of the present application;

[0034] Figure 11 is a structural schematic diagram of the CSNet network adopted in the optional embodiment of the present application;

[0035] Figure 12 is a structural schematic diagram of the classifier provided by the optional embodiment of the present application;

[0036] Figure 13 is a structural block diagram of the image processing device one provided by the embodiment 2 of the present application;

[0037] Figure 14 is a structural block diagram of the image processing device two provided by the embodiment 2 of the present application;

[0038] Figure 15 is a structural block diagram of the image processing device three provided by the embodiment 2 of the present application;

[0039] Figure 16 Figure 4 is a structural block diagram of an image processing device four according to the embodiment 2 of the present application;

[0040] Figure 17 Figure 5 is a structural block diagram of an image processing device five according to the embodiment 2 of the present application;

[0041] Figure 18 Figure 6 is a structural block diagram of a computer terminal according to the embodiment 3 of the present application. DETAILED DESCRIPTION

[0042] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person of ordinary skill in the art without making creative labor should belong to the scope of protection of the present application.

[0043] It should be noted that the terms “first”, “second” and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0044] First, some nouns or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

[0045] Photo Response Non-Uniformity (PRNU) is a characteristic noise trace left by a camera sensor on all captured images.

[0046] Image Source Identification (ISI) is used to determine which camera the image comes from.

[0047] The auto-encoder-decoder structure is a kind of self-encoding network, and the main idea is that a small multi-layer neural network can convert high-dimensional data into low-dimensional data, and then a similar self-encoding network can recover the original data from the low-dimensional data.

[0048] Global average pooling refers to adding all pixel values of a feature map and averaging to obtain a numerical value, that is, using the numerical value to represent the corresponding feature map.

[0049] Embodiment 1

[0050] According to the embodiments of the present application, a method for image processing is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] The method provided by the embodiment one of the present application can be executed in a mobile terminal, a computer terminal or a similar operation device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the image processing method is shown. As shown in Figure 1 , the computer terminal 10 (or mobile device) can include one or more processors (shown in the figure as 102a, 102b, …, 102n, the processor can include but not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device for communication function. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports in the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can also include more or less components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0052] It should be noted that the one or more processors and / or other data processing circuitry described above can be referred to herein generically as "data processing circuitry". The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any one of the other elements of the computer terminal 10 (or mobile device). As referred to in the embodiments of the present application, the data processing circuitry functions as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0053] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the image processing method in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the image processing method described above. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0054] The transmission device is used to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device includes a network interface controller (NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (RF) module which is used to communicate with the Internet in a wireless manner.

[0055] The display can be, for example, a touch screen type liquid crystal display (LCD) which can enable a user to interact with the user interface of the computer terminal 10 (or mobile device).

[0056] In the above operating environment, the present application provides an image processing method as shown in Figure 2 Figure 2 is a flowchart of the image processing method one according to the embodiment 1 of the present application, as shown in Figure 2

[0057] Step S202, obtaining a target image;

[0058] ​​In step S204, the image recognition model is used to perform source recognition on the target image to obtain source information of the target image. The image recognition model is trained based on a plurality of sets of sample images. The plurality of sets of sample images include sample images and source information of the sample images. In the training process, the image recognition model takes pre-extracted original image source information as a training target. The original image source information corresponds to source information of an original image of the sample image.

[0059] Through the above steps, the image recognition model is used to perform source recognition on the target image. Since the image recognition model is trained based on pre-extracted original image source information as a training target, the image recognition is effectively prevented from being affected by image post-processing. This makes the image source recognition more accurate. Furthermore, when the image recognition model is used to recognize the target image, the accuracy of the source information recognition of the image is effectively improved. Thus, the technical problem of low image source recognition accuracy caused by low extraction accuracy when extracting source information of an image in related technologies is solved.

[0060] As an optional embodiment, the image recognition model can be used to perform source recognition on the image to obtain source information of the target image in the following manner: an auto-encoder of the image recognition model is used to perform source recognition on the target image to obtain estimated source information; and a classifier of the image recognition model is used to perform classification processing on the estimated source information to obtain the source information of the target image. The image recognition model can have various structures. For example, the image recognition model can include an auto-encoder and a classifier. The auto-encoder is used to recognize the target image to obtain estimated source information. The classifier is used to classify the source of the target image based on the estimated source information. The final classification result is the source recognition result of the target image. The use of the auto-encoder can effectively realize flexible dimension conversion during feature extraction and processing, making the image recognition more efficient. In addition, since the auto-encoder can be trained based on original image source information as a training target, the image can be effectively prevented from being affected by post-processing. Thus, the image recognition is performed based on the original image of the image, realizing the accuracy of the image recognition.

[0061] As an optional embodiment, when the image recognition model is used to identify the source of the target image, the following steps can be used: in the encoder of the auto-encoder of the image recognition model, a convolutional layer is used to extract the features of the target image; in the decoder of the auto-encoder of the image recognition model, an inverse convolutional layer is used to reconstruct the extracted features to obtain the estimated source information. By using the convolutional layer and the inverse convolutional layer to extract the features of the target image and reconstruct the image, the main features of the target image can be extracted in a way with less calculation, and at the same time, the convolutional layer can be used to retain the image features as much as possible and reduce the loss of feature information. By retaining the intermediate features in the down-sampling process of the convolutional layer and fusing these features into the image reconstruction stage, the details of the original image can be better retained. In addition, according to the image recognition requirements, different sampling depths of the model can be achieved by adjusting the number of convolutions and inverse convolutions, that is, different precision feature extraction can be achieved, for example, the number of convolutional and inverse convolutional layers can be 3, 4 or 5, etc. According to the feature information extracted from the target image, the source information of the target image can be estimated.

[0062] It should be noted that the above-mentioned convolution-inverse convolution structure is only one of the implementation ways of the auto-encoder, for example, the auto-encoder can also use a compressed sensing network (CSNet). The CSNet can complete the reconstruction of the image by sampling the target image and performing initial reconstruction and deep reconstruction on the measurement values obtained after sampling. By limiting the output of the network, the estimated source information of the target image can also be obtained. The above-mentioned convolution-inverse convolution structure and CSNet network are only used as examples, and other auto-encoding networks can also be used as implementation ways of the auto-encoder to complete the estimation of the source information of the target image. Other auto-encoding networks also belong to the embodiments of the present application, which will not be described one by one.

[0063] As an optional embodiment, when the image recognition model is used to identify the source of the target image, the following steps can be used: in the encoder of the auto-encoder of the image recognition model, a convolutional layer is used to extract the features of the target image; in the decoder of the auto-encoder of the image recognition model, an inverse convolutional layer is used to reconstruct the extracted features to obtain the estimated source information. By using the convolutional layer and the inverse convolutional layer to extract the features of the target image and reconstruct the image, the main features of the target image can be extracted in a way with less calculation, and at the same time, the convolutional layer can be used to retain the image features as much as possible and reduce the loss of feature information. By retaining the intermediate features in the down-sampling process of the convolutional layer and fusing these features into the image reconstruction stage, the details of the original image can be better retained. In addition, according to the image recognition requirements, different sampling depths of the model can be achieved by adjusting the number of convolutions and inverse convolutions, that is, different precision feature extraction can be achieved, for example, the number of convolutional and inverse convolutional layers can be 3, 4 or 5, etc. According to the feature information extracted from the target image, the source information of the target image can be estimated.

[0064] It should be noted that the number of convolution layers in the above classifier is indefinite, for example, it can be 5 layers, 6 layers or 7 layers, etc. The step length of the above convolution layers when sampling is indefinite, which can be adjusted according to the sampling requirements in the actual application. For example, when the classifier is composed of 7 convolution layers, and it is required to quickly extract high-level feature information, the step length of the first to fourth layers can be set to 2, and the step length of the fifth to seventh layers can be set to 1, so that the high-level feature information can be quickly extracted. Before the cascading operation of the multi-layer convolution processing results, various processing can be performed on the convolution processing results of each layer, for example, the convolution processing results of each layer can be cascaded after global average pooling, so that the parameter quantity and the calculation quantity can be reduced, and the problem of overfitting of the classifier can be avoided. In addition, when the cascading operation is performed on the multi-layer convolution processing results, the convolution results of all convolution layers can be cascaded, or the convolution results in some specified convolution layers can be cascaded, for example, assuming that the classifier is composed of 7 convolution layers, the convolution results of the first layer, the third layer and the seventh layer can be selected for cascading, so that the loss and importance of each level of feature information and other factors can be supplemented to achieve better recognition and classification effect.

[0065] It should be noted that the above examples are only one of the contents in the implementation process of the embodiments of the present application, and other convolution layer numbers in the classifier, step lengths when sampling in each convolution layer, processing before cascading the multi-layer convolution results, and selection of convolution layers when cascading, all belong to the contents of the embodiments of the present application, which will not be exemplified one by one here.

[0066] As an optional embodiment, the above method further comprises: obtaining the original image source information of the sample image by a predetermined denoising method, wherein the original image source information comprises: light response non-uniformity information of the sample image. Since the light response uniformity information can represent the source of the corresponding image, the exact source information of the sample image can be obtained by extracting the light response non-uniformity information of the original image of the sample image, and the training of the image recognition model can be efficiently completed, and the accuracy of the model recognition result can be greatly improved.

[0067] Figure 3 is a flowchart of the image processing method two according to the embodiment 1 of the present application, as shown in Figure 3 the method comprises the following steps:

[0068] In step S302, a plurality of sets of sample image data are obtained, wherein the plurality of sets of sample image data comprise: sample images and source information of the sample images;

[0069] In step S304, the original image source information of the sample images in the plurality of sets of sample image data is extracted respectively, and the original image source information is the source information corresponding to the original image of the sample image;

[0070] In step S306, the plurality of sets of sample image data are used for machine training, and during the training process, the original image source information of the sample image is used as a training target to obtain the image recognition model.

[0071] Through the above steps, since the source information extracted from the original image of the sample image can accurately represent the source of the sample image, i.e., the image source can be accurately identified according to the source information, and on this basis, the original image source information of the sample image is used as a training target, and the sample image is used as an input, the image recognition model can continuously and infinitely approach the original image source information of the sample image during the training process, thereby improving the accuracy of the image source information output by the image recognition model and the accuracy of the image source recognition result. Therefore, the image recognition model obtained by using the above training method is more accurate, which provides a basis for effectively improving the recognition accuracy by using the above image recognition model, and further solves the technical problem of low image source recognition accuracy caused by poor extraction accuracy in related technologies when extracting image source information.

[0072] As an optional embodiment, when the machine is trained by using the plurality of sets of sample image data and the image recognition model is obtained by taking the original image source information of the sample image as a training target in the training process, the following steps can be used: inputting the sample image in the plurality of sets of sample image data into the initial auto-encoder to obtain estimated source information; inputting the estimated source information into the initial classifier to obtain classified source information; constructing a first loss function based on the estimated source information and the original image source information, and constructing a second loss function based on the classified source information and the source information of the sample image; constructing a target loss function based on the first loss function and the second loss function; and obtaining the image recognition model by minimizing the target loss function through machine training by using the plurality of sets of sample image data. By using the first loss function to constrain the estimated source information output by the auto-encoder, the estimated source information output by the auto-encoder in the training process can gradually approach the original image source information of the sample image, that is, the auto-encoder can extract information that can accurately represent the source of the image according to the input image, without being affected by image-related processing. At the same time, by using the second loss function to constrain the classified source information output by the classifier, the estimated source information output by the auto-encoder can obtain more and more accurate image source classification results after passing through the classifier, that is, on the basis of improving the accuracy of the estimated source information of the auto-encoder, the accuracy of classification based on the source information is further improved. Through the above operation, not only can the trained image recognition model be immune to image-related processing when identifying the source of the image, but also on the basis of approaching the source information of the original image, the classifier can identify a higher recognition result than the original image, so that the accuracy of image recognition based on the image recognition model is maximized, and the maximum image source recognition result is obtained.

[0073] As an optional embodiment, when the target loss function is constructed based on the first loss function and the second loss function, the following steps can be used: determining a first weight of the first loss function and a second weight of the second loss function; multiplying the first loss function by the first weight to obtain a first value, and multiplying the second loss function by the second weight to obtain a second value; and summing the first value and the second value to obtain the target loss function. By determining the weights of the two loss functions respectively, the constraint strength of the loss function can be adjusted according to the importance or other needs in actual application, thereby ensuring the effectiveness and flexibility in the training process. In addition, the target loss function can be constructed in various ways, and the above way is only used as an example. Other ways of processing the first loss function and the second loss function and constructing the target loss function also belong to the embodiments of the present application, which will not be illustrated one by one here.

[0074] Figure 4 is a flowchart of the image processing method three according to Embodiment 1 of the present application, as Figure 4As shown, the method comprises the following steps:

[0075] Step S402, displaying an import control on the interactive interface;

[0076] Step S404, in response to an import operation on the import control, importing a target image;

[0077] Step S406, in response to an identification operation on the identification control on the interactive interface, displaying an identification result of identifying the target image on the interactive interface, wherein the identification result comprises source information of the target image, the source information of the target image is obtained based on an image identification model, the image identification model is obtained based on training of a plurality of sets of sample image data, and the plurality of sets of sample image data comprises a sample image and source information of the sample image.

[0078] Through the above operations, the target image can be imported on the interactive interface in response to the import control, and the target image can be identified by the trained image identification model in response to the identification operation on the identification control on the interactive interface, to obtain the source information of the target, and thus the source identification of the target image is completed. In addition, based on the above interactive operation on the interactive interface, the identification result of identifying the target image can be more intuitively displayed, and thus the technical problem of low image source identification accuracy caused by poor extraction precision in related technologies when extracting the source information of the image is solved.

[0079] Figure 5 is a flowchart of the image processing method four according to Embodiment 1 of the present application, as shown, the method comprises the following steps: Figure 5

[0080] Step S502, displaying a plurality of sets of sample image data on the interactive interface, wherein the plurality of sets of sample image data comprises a sample image and source information of the sample image;

[0081] Step S504, displaying original image source information of the sample image in the plurality of sets of sample image data on the interactive interface, wherein the original image source information is pre-extracted source information corresponding to an original image of the sample image;

[0082] Step S506, displaying a model icon of an image identification model on the interactive interface, wherein the image identification model is trained by the plurality of sets of sample image data, and the original image source information of the sample image is used as a training target in the training process.

[0083] ​Through the above operations, the multiple sets of sample image data, the original image source information of the sample image, and the model icon of the image recognition model can be displayed on the interactive interface. By taking the original image source information of the sample image as a training target and using the sample image and the source information of the sample image to machine train the image recognition model, the recognition capability of the image recognition model for the source information of the target image can be improved, the accuracy of the recognition result can be improved, and thus the technical problem of low image source recognition accuracy caused by poor extraction accuracy when extracting the source information of the image in the related art is solved.

[0084] Figure 6 is a flowchart of the image processing method five according to the embodiment 1 of the present application, as shown in the figure, the method comprises the following steps: Figure 6

[0085] In step S602, the image of the commodity is acquired.

[0086] In step S604, the image recognition model is used to identify the source of the image of the commodity, and the source information of the image of the commodity is obtained, wherein the source information includes the camera that captures the image of the commodity, the image recognition model is obtained based on the training of multiple sets of sample images, the multiple sets of sample images include: sample images and source information of the sample images, and in the training process of the image recognition model, the light response non-uniformity information of the original image of the sample image extracted in advance is taken as a training target.

[0087] Through the above steps, the image recognition model is used to identify the source of the image of the commodity. Since the image recognition model takes the light response non-uniformity information of the original image of the sample image extracted in advance as a training target in the training process, the influence of image post-processing on the identification of the source of the image of the commodity is effectively avoided. The training of the image recognition model is guided, and the accuracy of the identification of the source information of the image is effectively improved when the image recognition model is used to identify the image of the commodity. Thus, the technical problem of low image source recognition accuracy caused by poor extraction accuracy when extracting the source information of the image in the related art is solved. The camera that captures the image of the commodity can be accurately identified, and whether the image of the commodity is a stolen image can be accurately determined.

[0088] Based on the above embodiments and optional embodiments, an optional implementation is provided, which is described in detail as follows.

[0089] ​With the help of various hardware or software tools, the creation and editing of images have become very simple, so the authenticity and authority of images have been greatly challenged. Image forensics technology can help us evaluate the authenticity and integrity of a given image, and one of the most important areas is the image source identification problem (ISI). The purpose of the ISI problem is mainly to identify the fingerprint of the digital image acquisition device, Figure 7 is a classification diagram of the image source identification problem provided by the optional implementation manner of the present application, as shown in Figure 7 , including specific camera model identification and specific camera individual identification, for example, camera model identification and camera individual identification.

[0090] The main application scenarios of image source identification technology are:

[0091] (1) Image leakage tracing

[0092] (2) For various (such as commodity) picture stealing scenarios

[0093] (3) Scenarios requiring various certificates

[0094] (4) Forensic identification

[0095] (5) As a supplement to other forensics technologies.

[0096] Figure 8 is a diagram for tampering with the Make and Model in the Exif data of an image using ExifTool as listed in the optional implementation manner of the present application, as shown in Figure 8 , wherein the camera of the same image is changed from the original Casio to iPhone4, and only looking at the metadata and other surface evidence cannot draw a conclusion, but image source identification technology can easily distinguish images from cameras and mobile phones, thereby making an accurate judgment on the authenticity of the image.

[0097] As described above, the result of image source identification can be used as the lowest reference in the image forensics field for screening and assisting in identifying suspicious images, and can help track the source of the image, which is also of great significance for pointing out the owner of abnormal images (such as fake retouching) and ensuring the security and credibility of data information.

[0098] To identify the source of the image, the following two schemes can be used.

[0099] (1) Scheme one: camera model identification based on PRNU

[0100] PRNU is a specific fingerprint that a camera leaves in the pictures it takes, due to the subtle random imperfections of the camera sensor. Each device has its own specific PRNU pattern, which can be accurately estimated through complex processing steps, provided that a large number of images taken by the device are available. In fact, due to the inevitable errors in the manufacture of the sensor, and the non-uniformity of the sensor cells, which produce slightly different pixel brightness even under the same light intensity. Therefore, for images generated by a given camera, scheme one gives a simplified multiplication model:

[0101]

[0102] where, is the original image, is the PRNU pattern, is other noise. PRNU is unique for each device, stable over time, and present in all images taken by the device itself. The PRNU of a camera can be obtained from a series of images , and when calculating the PRNU, the noise residual needs to be considered first:

[0103] where, is the denoising algorithm. Then, the PRNU ( ) can be calculated as follows:

[0104]

[0105] After calculating the PRNU, in order to compare whether a certain image belongs to the PRNU, the correlation (NCC) between and can be calculated, as shown in the following formula:

[0106]

[0107] where, and are the inner product and Euclidean norm, respectively. By calculating the correlation between and , it can be determined whether the image belongs to a certain camera.

[0108] However, scheme one also has disadvantages. The extraction method belongs to the traditional noise-based extraction method, which generally regards PRNU as a specific noise, uses traditional denoising algorithms to extract PRNU, and finally classifies the extracted PRNU. ​The commonly used de-noising algorithm is mainly Fourier transform + Wiener filter. When the image is not compressed and is not affected by channel transmission and other post-processing, the method based on traditional noise extraction can maintain a high accuracy of source identification. However, when the image is compressed and transmitted through the channel, these post-processing will affect the accuracy of PRNU extraction, so that the extracted PRNU is quite different from the PRNU without the influence, thereby affecting the subsequent classification, and further leading to a great reduction in the accuracy of source identification.

[0109] (2) Scheme two: camera model identification based on PRNU and image noise fingerprint

[0110] When the amount of available data is small and the quality is poor, such as the obtained picture is only a part of the original picture or the number of pictures is small, these will cause the performance of the PRNU-based scheme to decrease rapidly. Moreover, even if the image data is sufficient, calculating PRNU from all images will also lead to high computational complexity of the algorithm. Therefore, considering these limitations, scheme two introduces a convolutional neural network to process the camera model identification based on PRNU and image noise fingerprint, and good experimental results are obtained. The method is briefly introduced as follows.

[0111] For a given set of images from the same device , the PRNU value of the device is obtained by de-noising and maximum likelihood estimation . For a given test image , the noise residual of the image can be extracted by a de-noising program .

[0112] For the traditional PRNU-based image source identification method, the noise residual and are compared to find possible matches. Since this method will lead to a sharp increase in computational complexity, a CNN network is needed to help comparison and calculation.

[0113] For each pair of query image and candidate device , the noise residuals and are sent to a two-channel based convolutional neural network. Assuming they are geometrically synchronous, the network will return a CNN-based identification score , which is directly related to the consistency between the image and the device. By analyzing the identification score , it can be inferred whether the query image belongs to the candidate device Therefore, the identification of the image source device can be quickly completed in the case of needing to scan a large number of device fingerprint databases. Experimental test results show that even if a shallow two-channel CNN architecture is used, the method is faster and more accurate than the traditional PCE method.

[0114] However, scheme two also has disadvantages. The CNN model with two channels is used to process the camera model identification problem based on the PRNU and image noise fingerprint, which is faster than the traditional PCE method in the case of investigating a large number of potential source devices, and requires less query image content to obtain higher source device identification accuracy and better processing effect. However, the method is still insufficient for the extraction of camera PRNU and standard image noise residuals, and does not fuse an effective neural network extraction method, which will show obvious identification lag when facing a large number of images and devices. In addition, the method also shows great instability when facing the potential pixel misalignment between the image to be processed and the fingerprint. Therefore, the future improvement direction of the method should focus on the research of the extraction learning method of the noise residual, the performance stability improvement of the image compression and the potential pixel misalignment.

[0115] To solve the above problems, the optional embodiment of the present application provides a solution for image source identification.

[0116] Figure 9 is a schematic diagram of the image processing method provided according to the embodiment of the present application, as Figure 9 shown, the optional embodiment of the present application will be described in detail below.

[0117] (1) Target-guided source identification

[0118] The starting point of the optional embodiment of the present application is to make the PRNU extracted by the network more accurate. From some existing research, PRNU is the most commonly used and effective method for image source identification. Therefore, the optional embodiment of the present application hopes that the PRNU extracted after the image is processed by compression and other post-processing can also be unaffected by the post-processing, so as to obtain a higher source identification accuracy.

[0119] Therefore, the optional embodiment of the present application adopts a target guiding method to make the PRNU generated by the network more accurate. The PRNU given in the input is extracted based on a traditional denoising algorithm (mainly Fourier transform + Wiener filter), wherein the extracted PRNU is based on an original image that is not affected by post-processing, because the extracted PRNU has been verified to have high source identification accuracy. Then, the PRNU is taken as a target PRNU, and the output of the auto-encoding network (U-Net) is made as similar as possible to the target PRNU by loss function one. At the same time, by connecting a classifier after the generated estimated PRNU, the extracted PRNU is not limited to being close to the target PRNU, but can also exceed the target PRNU, so that loss function two is getting smaller and smaller, and the accuracy of classification is getting higher and higher.

[0120] (2) Optimized Encoder-Decoder Structure (U-Net)

[0121] The U-Net has very good transferability and robustness in medical image segmentation tasks. Figure 10 is a structural diagram of the U-Net provided according to the optional embodiment of the present application, as shown in Figure 10 The U-Net structure only uses convolution layers to extract semantic features and deconvolution layers to reconstruct images when designed, so it is a typical fully convolutional neural network. The advantage of this structure model is that it is very easy to adjust the structure according to the requirements of the task, such as increasing or decreasing the number of convolutions and deconvolutions to adjust the depth of the model. Moreover, the U-Net has no great limitation on the input size of the image, only the setting of the input image size and the convolution kernel size needs to be paid attention to. Due to the loss of information in the downsampling process, the ordinary encoder-decoder structure cannot restore the details of the image in the image reconstruction process. The U-Net model structure preserves the intermediate features in the feature extraction process, and fuses these intermediate features into the corresponding upsampling layer in the image reconstruction stage, so as to preserve as many details of the original image as possible.

[0122] For the encoder, since the optional embodiment of this invention focuses more on capturing details in the image, it aims to minimize the loss of feature information during the feature extraction stage. Therefore, the optional embodiment of this invention chooses to replace the max pooling layer used for downsampling in the original UNet model structure with a convolutional layer (3 × 3 kernel size, stride 2). This reduces information loss, but increases the number of model parameters. The downsampling iterations of the model continue to use the 4 iterations of the UNet model structure. For the decoder, the optional embodiment of this invention chooses to use a deconvolutional layer (2 × 2 kernel size, stride 2) to reconstruct the image.

[0123] Furthermore, in the autoencoding stage, the optional embodiments of the present invention described above employ a U-Net network. This network can be replaced by other autoencoding networks, such as the CSNet network used in compressed sensing. Figure 11 This is a schematic diagram of the structure of a CSNet network used in an optional embodiment of the present invention, as shown below. Figure 11 As shown, this CSNet network obtains measurement values ​​by sampling the original image, and then performs initial reconstruction and depth reconstruction on the measurement values ​​to obtain the original image. Figure 1 For reconstructed images of similar size, the network output can be limited to PRNU using a loss function. To ensure good reconstruction results, the sampling rate is generally set to 1.0 in CSNet, i.e., uncompressed sampling reconstruction.

[0124] (3) Feature pyramid-based classifier

[0125] Figure 12 This is a schematic diagram of the structure of a classifier provided in an optional embodiment of the present invention, such as... Figure 12 As shown, the optional embodiment of this invention utilizes a feature pyramid structure to enable the classifier to better focus on both global and local information of the image. During downsampling, intermediate feature maps are preserved, and cascading operations are used to recover the shallow semantic information lost during convolution before the final fully connected layer. Therefore, the classifier designed in this optional embodiment is a simple regression network consisting of seven convolutional layers. All convolutional operations in the classifier use 3x3 kernels. To quickly extract high-level semantic information, downsampling is performed using four consecutive convolutional layers with a stride of 2. Simultaneously, the number of channels in the feature map is continuously increased in multiples of 2. Since downsampling in the first 1-4 layers results in significant loss of shallow semantic information, the feature maps from the first, third, and final downsampling layers are fused together using a concatenation operation after global average pooling. Finally, the fused feature map is fed into a fully connected layer for regression, and the final output dimension of the fully connected layer is the camera category.

[0126] In the related technical solution, the quality of the classification result is seriously dependent on the effect of PRNU extraction, and in the compression, channel transmission and other post-processing, the method of extracting PRNU is easily affected by these post-processing, or the noise generated by the post-processing is extracted as part of the PRNU, which undoubtedly affects the accuracy of PRNU extraction, thereby causing the accuracy of subsequent classification to decrease.

[0127] Through the above processing, the optional embodiment of the present application proposes a network structure based on target guidance, by inputting the pre-extracted target PRNU in the network, the network learns the ability to extract the PRNU of any input image, of course, the ability is only to make the extracted PRNU infinitely close to the original PRNU unaffected by post-processing, so the accuracy is also close to the original accuracy. In this way, the network knows that no matter for any input image, even after post-processing, the PRNU to be output is close to the original PRNU.

[0128] At the same time, in order to further improve the extraction accuracy of PRNU and identification, the optional embodiment connects a classification network after the estimated PRNU, the result of the classification network is taken as the second loss, and it is found in the experiment that the value of the second loss function is greater than that of the first loss function, so the weight occupied is greater, and at the same time of network convergence, the constraint of the second loss function will make the extracted PRNU not limited to close to the target PRNU, but also possible to surpass the target PRNU, so that the second loss function becomes smaller and smaller, and the classification accuracy becomes higher and higher.

[0129] The optional embodiment of the present application solves the problem that the compression and other post-processing greatly affect the image source identification, so that the PRNU can also get a good result in the post-processing.

[0130] In addition, in the optional embodiment of the present application, the PRNU and the image are processed in blocks, that is, the batch size is 512, the network is trained for 300 generations, the optional embodiment of the present application uses the Adam optimizer to adaptively optimize the parameters of the network, the initial learning rate is 0.0004, and after the training starts, the learning rate is reduced to 0.5 times of the original every 30 generations. For the case of post-processing such as compression, the learning rate is reduced to 0.5 times of the original every 50 generations. The training set that can be used by the optional embodiment of the present application is the VISION data set, which is a commonly used data set in source identification, including 35 low, medium and high-priced mobile devices, 4821 flat (shooting flat scenes, such as the sky) data and 7565 nat (shooting natural scenes) data in the image. In this paper, the PRNU is extracted in advance from the flat image, and then 6620 of the nat images are used as the training set and 945 are used as the test set. The experimental results are shown in Table 1:

[0131]

[0132] Table 1 Source identification accuracy (%) under different image block size and different post-processing conditions

[0133] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0134] From the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, and of course it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method of each embodiment of the present application.

[0135] Embodiment 2

[0136] According to the embodiments of the present application, an apparatus for implementing the above image processing method is also provided, Figure 13 is a structure block diagram of the image processing apparatus 1 provided by the embodiment 2 of the present application, as shown in Figure 13As shown in the figure, the device comprises a first acquisition module 1301 and a first identification module 1302, which are described below.

[0137] The first acquisition module 1301 is configured to acquire a target image; the first identification module 1302 is connected to the first acquisition module 1301 and configured to perform source identification on the target image by using an image identification model to obtain source information of the target image, wherein the image identification model is obtained by training based on a plurality of sets of sample images, and the plurality of sets of sample images comprise sample images and source information of the sample images, wherein the image identification model takes pre-extracted original image source information as a training target in the training process, and the original image source information is source information corresponding to an original image of the sample image.

[0138] It should be noted that the first acquisition module 1301 and the first identification module 1302 correspond to steps S202 to S304 in Embodiment 1, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0139] According to the embodiments of the present application, a device for implementing the above image processing method is also provided, Figure 14 is a structural block diagram of the image processing device two provided in Embodiment 2 of the present application, as Figure 14 As shown in the figure, the device comprises a second acquisition module 1401, an extraction module 1402 and a training module 1403, which are described below.

[0140] The second acquisition module 1401 is configured to acquire a plurality of sets of sample image data, wherein the plurality of sets of sample image data comprise sample images and source information of the sample images; the extraction module 1402 is connected to the second acquisition module 1401 and configured to extract original image source information of the sample images in the plurality of sets of sample image data respectively, wherein the original image source information is source information corresponding to an original image of the sample image; and the training module 1403 is connected to the extraction module 1402 and configured to perform machine training by using the plurality of sets of sample image data, and takes the original image source information of the sample image as a training target in the training process to obtain an image identification model.

[0141] It should be noted that the second acquisition module 1401, the extraction module 1402 and the training module 1403 correspond to steps S302 to S306 in Embodiment 1, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0142] According to an embodiment of the present application, a device for implementing the image processing method is also provided, Figure 14 is a structural block diagram of the image processing device three provided according to the embodiment 2 of the present application, as shown in the figure, the device comprises a first display module 1501, a first response module 1502 and a second response module 1503, and the device will be described below. Figure 15 The first display module 1501 is configured to display an import control on the interactive interface; the first response module 1502 is connected to the first display module 1501 and configured to import a target image in response to an import operation on the import control; and the second response module 1503 is connected to the first response module 1502 and configured to display a recognition result of the target image on the interactive interface in response to a recognition operation on a recognition control on the interactive interface, wherein the recognition result comprises source information of the target image, the source information of the target image is obtained based on an image recognition model, the image recognition model is obtained based on a plurality of sample images, and the plurality of sample images comprise a sample image and source information of the sample image, wherein the image recognition model takes pre-extracted original image source information as a training target in a training process, and the original image source information is source information corresponding to an original image of the sample image.

[0143] It should be noted that the first display module 1501, the first response module 1502 and the second response module 1503 correspond to steps S402 to S406 in the embodiment, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above embodiment 1. It should be noted that the above modules can run in the computer terminal 10 provided in the embodiment 1 as a part of the device.

[0144] According to an embodiment of the present application, a device for implementing the image processing method is also provided,

[0145] is a structural block diagram of the image processing device four provided according to the embodiment 2 of the present application, as shown in the figure, the device comprises a second display module 1601, a third display module 1602 and a fourth display module 1603, and the device will be described below. Figure 16 Figure 16 The second display module 1601 is configured to display a target image on the interactive interface; the third display module 1602 is connected to the second display module 1601 and configured to display a recognition control on the interactive interface; and the fourth display module 1603 is connected to the third display module 1602 and configured to display a recognition result of the target image on the interactive interface in response to a recognition operation on the recognition control on the interactive interface, wherein the recognition result comprises source information of the target image, the source information of the target image is obtained based on an image recognition model, the image recognition model is obtained based on a plurality of sample images, and the plurality of sample images comprise a sample image and source information of the sample image, wherein the image recognition model takes pre-extracted original image source information as a training target in a training process, and the original image source information is source information corresponding to an original image of the sample image.

[0146] ​The second display module 1601 is configured to display a plurality of sets of sample image data on the interactive interface, wherein the plurality of sets of sample image data comprises sample images and source information of the sample images.

[0147] It should be noted that the second display module 1601, the third display module 1602, and the fourth display module 1603 correspond to steps S502 to S506 in the embodiment, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the modules can be run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0148] According to the embodiment of the present application, a device for implementing the image processing method is also provided. Figure 17 is a structural block diagram of the image processing device five provided in Embodiment 2 of the present application, as shown in Figure 17 The device comprises a third acquisition module 1701 and a second identification module 1702, which will be described below.

[0149] The third acquisition module 1701 is configured to acquire a product image. The second identification module 1702 is connected to the third acquisition module 1701 and is configured to identify the source of the product image by using an image recognition model to obtain source information of the product image, wherein the source information comprises a camera that captures the product image. The image recognition model is obtained based on a plurality of sets of sample images. The plurality of sets of sample images comprise sample images and source information of the sample images. In the training process of the image recognition model, the light response non-uniformity information of a pre-extracted original image of the sample image is used as a training target.

[0150] It should be noted that the third acquisition module 1701 and the second identification module 1702 correspond to steps S602 to S604 in the embodiment, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the modules can be run in the computer terminal 10 provided in Embodiment 1 as part of the device.

[0151] Embodiment 3

[0152] The embodiment of the present application can provide a computer terminal (or computer device), which can be any computer terminal device in a computer terminal group. Alternatively, in the embodiment, the computer terminal can be replaced by a terminal device such as a mobile terminal.

[0153] Alternatively, in the embodiment, the computer terminal can be located in at least one network device of a plurality of network devices of a computer network.

[0154] In the embodiment, the computer terminal can execute program codes of the following steps in the image processing method: obtaining a target image; performing source identification on the target image by using an image recognition model to obtain source information of the target image, wherein the image recognition model is obtained based on a plurality of sets of sample images, and the plurality of sets of sample images include sample images and source information of the sample images, and the image recognition model takes the pre-extracted original image source information as a training target in the training process, and the original image source information is source information corresponding to an original image of the sample image.

[0155] Alternatively, Figure 18 is a structural block diagram of a computer terminal according to the embodiment 3 of the present application. As shown in Figure 18 the computer terminal can include one or more (only one is shown in the figure) processors 1802, memories 1804, and the like.

[0156] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the image processing method and device in the embodiment of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned image processing method. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, and the remote memory can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0157] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: obtaining a target image; performing source identification on the target image by using an image recognition model to obtain source information of the target image, wherein the image recognition model is obtained based on a plurality of sets of sample images, and the plurality of sets of sample images include sample images and source information of the sample images, and the image recognition model takes the pre-extracted original image source information as a training target in the training process, and the original image source information is source information corresponding to an original image of the sample image.

[0158] Optionally, the processor can further execute program codes of the following steps: performing source identification on the target image by using a self-encoder of the image recognition model to obtain estimated source information; and performing classification processing on the estimated source information by using a classifier of the image recognition model to obtain the source information of the target image.

[0159] Optionally, the processor can further execute program codes of the following steps: in an encoder of the self-encoder of the image recognition model, extracting features of the target image by using a convolutional layer; and in a decoder of the self-encoder of the image recognition model, performing image reconstruction on the extracted features by using a deconvolutional layer to obtain the estimated source information.

[0160] Optionally, the processor can further execute program codes of the following steps: in the classifier of the image recognition model, processing the estimated source information by using a plurality of convolutional layers to obtain a plurality of convolutional processing results; and in the classifier of the image recognition model, performing a cascade operation on the plurality of convolutional processing results to obtain the source information of the target image.

[0161] Optionally, the processor can further execute program codes of the following steps: obtaining original image source information of the sample image by using a predetermined denoising method, wherein the original image source information includes light response non-uniformity information of the sample image.

[0162] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: obtaining a plurality of sets of sample image data, wherein the plurality of sets of sample image data include sample images and source information of the sample images; extracting original image source information of the sample images in the plurality of sets of sample image data, respectively, the original image source information being source information corresponding to original images of the sample images; and performing machine training by using the plurality of sets of sample image data, and obtaining the image recognition model in a training process with the original image source information of the sample images as a training target.

[0163] Optionally, the processor can further execute program codes of the following steps: inputting the sample images in the plurality of sets of sample image data into an initial self-encoder to obtain estimated source information; inputting the estimated source information into an initial classifier to obtain classified source information; constructing a first loss function based on the estimated source information and the original image source information, and constructing a second loss function based on the classified source information and the source information of the sample images; constructing a target loss function based on the first loss function and the second loss function; and performing machine training by using the plurality of sets of sample image data by minimizing the target loss function to obtain the image recognition model.

[0164] Optionally, the processor can further execute program codes of the following steps: determining a first weight of the first loss function and a second weight of the second loss function; multiplying the first loss function by the first weight to obtain a first value, and multiplying the second loss function by the second weight to obtain a second value; summing the first value and the second value to obtain the target loss function.

[0165] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: displaying an import control on the interactive interface; in response to an import operation on the import control, importing a target image; in response to an identification operation on an identification control on the interactive interface, displaying an identification result of identifying the target image on the interactive interface, wherein the identification result includes source information of the target image, and the source information of the target image is obtained based on an image identification model, the image identification model is obtained based on a plurality of sets of sample image data, and the plurality of sets of sample image data include: sample images and source information of the sample images, wherein the image identification model takes pre-extracted original image source information as a training target in a training process, and the original image source information is source information corresponding to an original image of the sample image.

[0166] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: displaying a plurality of sets of sample image data on the interactive interface, wherein the plurality of sets of sample image data include: sample images and source information of the sample images; displaying original image source information of the sample images in the plurality of sets of sample image data on the interactive interface, wherein the original image source information is pre-extracted source information corresponding to an original image of the sample image; and displaying a model icon of an image identification model on the interactive interface, wherein the image identification model is trained by the plurality of sets of sample image data, and the original image source information of the sample image is taken as a training target in a training process.

[0167] The processor can call information and application programs stored in the memory through the transmission device to execute the following steps: obtaining a product image; and identifying the source of the product image by using an image identification model to obtain source information of the product image, wherein the source information includes a camera that captures the product image, the image identification model is obtained based on a plurality of sets of sample image data, and the plurality of sets of sample image data include: sample images and source information of the sample images, wherein the image identification model takes pre-extracted light response non-uniformity information of an original image of the sample image as a training target in a training process.

[0168] This invention provides an image processing method. An image recognition model is used to identify the source of a target image. Since this image recognition model is trained based on pre-extracted source information from the original image, it effectively avoids the influence of post-processing on image recognition, making source identification more accurate. Furthermore, when using the image recognition model to identify the target image, it effectively improves the accuracy of source information identification, thus solving the technical problem in related technologies where poor extraction accuracy leads to low accuracy in image source identification.

[0169] Those skilled in the art will understand that Figure 16 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 16 This does not limit the structure of the aforementioned electronic devices. For example, a computer terminal may also include components that are more... Figure 16 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 16 The different configurations shown.

[0170] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0171] Example 4

[0172] Embodiments of the present invention also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the image processing method provided in Embodiment 1.

[0173] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0174] Optionally, in the embodiment, the computer readable storage medium is configured to store program code for performing the following steps: obtaining the target image; performing source identification on the target image by using the image recognition model to obtain source information of the target image, wherein the image recognition model is obtained by training based on a plurality of sets of sample images, and the plurality of sets of sample images include sample images and source information of the sample images, and the image recognition model takes pre-extracted original image source information as a training target in the training process, and the original image source information is source information corresponding to an original image of the sample image.

[0175] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: performing source identification on the target image by using a self-encoder of the image recognition model to obtain estimated source information; and performing classification processing on the estimated source information by using a classifier of the image recognition model to obtain the source information of the target image.

[0176] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: in an encoder of the self-encoder of the image recognition model, a convolutional layer is used to extract features of the target image; and in a decoder of the self-encoder of the image recognition model, an inverse convolutional layer is used to perform image reconstruction on the extracted features to obtain the estimated source information.

[0177] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: in the classifier of the image recognition model, a plurality of convolutional layers are used to process the estimated source information to obtain a plurality of convolutional processing results; and in the classifier of the image recognition model, a cascade operation is performed on the plurality of convolutional processing results to obtain the source information of the target image.

[0178] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: obtaining original image source information of the sample image by using a predetermined denoising method, wherein the original image source information includes light response non-uniformity information of the sample image.

[0179] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: obtaining a plurality of sets of sample image data, wherein the plurality of sets of sample image data include sample images and source information of the sample images; extracting original image source information of the sample images in the plurality of sets of sample image data respectively, wherein the original image source information is source information corresponding to an original image of the sample image; and performing machine training by using the plurality of sets of sample image data, and taking the original image source information of the sample image as a training target in the training process to obtain the image recognition model.

[0180] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: inputting the sample image in the plurality of sets of sample image data into the initial autoencoder to obtain estimated source information; inputting the estimated source information into the initial classifier to obtain classified source information; constructing a first loss function based on the estimated source information and the original image source information, and constructing a second loss function based on the classified source information and the source information of the sample image; constructing a target loss function based on the first loss function and the second loss function; and performing machine training on the plurality of sets of sample image data by minimizing the target loss function to obtain the image recognition model.

[0181] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: determining a first weight of the first loss function and a second weight of the second loss function; multiplying the first loss function by the first weight to obtain a first value, and multiplying the second loss function by the second weight to obtain a second value; and summing the first value and the second value to obtain the target loss function.

[0182] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: displaying an import control on the interactive interface; in response to an import operation on the import control, importing the target image; and in response to an identification operation on the identification control on the interactive interface, displaying an identification result of the target image on the interactive interface, wherein the identification result includes source information of the target image, the source information of the target image is identified based on the image recognition model, the image recognition model is trained based on the plurality of sets of sample image data, the plurality of sets of sample image data include: the sample image and the source information of the sample image, and the image recognition model takes the original image source information as a training target during the training process, the original image source information is source information corresponding to an original image of the sample image.

[0183] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: displaying the plurality of sets of sample image data on the interactive interface, wherein the plurality of sets of sample image data include: the sample image and the source information of the sample image; displaying the original image source information of the sample image in the plurality of sets of sample image data on the interactive interface, the original image source information being pre-extracted source information corresponding to an original image of the sample image; and displaying a model icon of the image recognition model on the interactive interface, wherein the image recognition model is trained by the plurality of sets of sample image data, and the original image source information of the sample image is taken as a training target during the training process.

[0184] Optionally, in the embodiment, the computer readable storage medium is further configured to store program code for performing the following steps: obtaining the product image; performing source identification on the product image by using the image recognition model to obtain source information of the product image, wherein the source information comprises a camera for capturing the product image, and the image recognition model is obtained by training based on a plurality of sets of sample images, and the plurality of sets of sample images comprise a sample image and source information of the sample image, wherein the image recognition model takes pre-extracted light response non-uniformity information of an original image of the sample image as a training target during the training process.

[0185] The above-mentioned serial numbers of the embodiments of the application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0186] In the above-mentioned embodiments of the application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0187] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other manners. Among them, the above-mentioned device embodiments are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0188] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0189] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0190] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0191] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a target image; performing source identification on the target image by using an image recognition model to obtain source information of the target image, wherein the image recognition model is trained based on a plurality of sets of sample images, and the plurality of sets of sample images comprise sample images and source information of the sample images, wherein the image recognition model takes pre-extracted original image source information as a training target in a training process, and the original image source information is source information corresponding to an original image of the sample image; wherein, in response to a network of a self-encoder of the image recognition model being a CSNet network, the CSNet network is used to sample the target image to obtain measurement values of the target image, the CSNet network is used to reconstruct the measurement values to obtain a reconstructed image of the target image, the CSNet network is used to perform source identification on the reconstructed image to obtain estimated source information of the target image, and a classifier of the image recognition model is used to perform classification processing on the estimated source information to obtain the source information of the target image.

2. The method of claim 1, wherein, The method comprises: in response to the network of the self-encoder being a U-Net network, the U-Net network is used to perform source identification on the target image to obtain estimated source information; the classifier of the image recognition model is used to perform classification processing on the estimated source information to obtain the source information of the target image.

3. The method of claim 2, wherein, in response to the network of the self-encoder being a U-Net network, the U-Net network is used to perform source identification on the target image to obtain estimated source information, which comprises: in an encoder of the U-Net network, a convolutional layer is used to extract features of the target image; in a decoder of the U-Net network, an inverse convolutional layer is used to perform image reconstruction on the features to obtain the estimated source information.

4. The method of claim 2, wherein, The method comprises: in the classifier of the image recognition model, a plurality of convolutional layers are used to process the estimated source information to obtain a plurality of convolutional processing results; in the classifier of the image recognition model, a cascade operation is performed on the plurality of convolutional processing results to obtain the source information of the target image.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: obtaining original image source information of the sample images by using a predetermined denoising method, wherein the original image source information comprises light response non-uniformity information of the sample images.

6. An image processing method characterized by, The method comprises: acquiring a plurality of sets of sample image data, wherein the plurality of sets of sample image data comprise sample images and source information of the sample images; extracting original image source information of the sample images in the plurality of sets of sample image data, wherein the original image source information is source information corresponding to an original image of the sample image; The multiple sets of sample image data are used for machine training, and the original image source information of the sample image is used as a training target during the training process to obtain an image recognition model, wherein the image recognition model comprises a self-encoder, and a network of the self-encoder is a CSNet network; the CSNet network is used for sampling a target image to obtain a measurement value of the target image, reconstructing the measurement value to obtain a reconstructed image of the target image, and identifying the source of the reconstructed image to obtain estimated source information of the target image; and a classifier of the image recognition model is used for classifying the estimated source information to obtain the source information of the target image.

7. The method of claim 6, wherein, The multiple sets of sample image data are used for machine training, and the original image source information of the sample image is used as a training target during the training process to obtain an image recognition model, wherein the image recognition model comprises a self-encoder, and a network of the self-encoder is a CSNet network; the CSNet network is used for sampling a target image to obtain a measurement value of the target image, reconstructing the measurement value to obtain a reconstructed image of the target image, and identifying the source of the reconstructed image to obtain estimated source information of the target image; and a classifier of the image recognition model is used for classifying the estimated source information to obtain the source information of the target image. The sample image in the multiple sets of sample image data is input into an initial self-encoder to obtain estimated source information. The estimated source information is input into an initial classifier to obtain classified source information. A first loss function is constructed based on the estimated source information and the original image source information, and a second loss function is constructed based on the classified source information and the source information of the sample image. A target loss function is constructed based on the first loss function and the second loss function. The multiple sets of sample image data are used for machine training by minimizing the target loss function to obtain the image recognition model.

8. The method of claim 7, wherein, The target loss function is constructed based on the first loss function and the second loss function, comprising: A first weight of the first loss function and a second weight of the second loss function are determined. The first loss function is multiplied by the first weight to obtain a first value, and the second loss function is multiplied by the second weight to obtain a second value. The first value and the second value are summed to obtain the target loss function.

9. An image processing method characterized by, It comprises: Display an import control on an interactive interface; In response to an import operation on the import control, import a target image; In response to an identification operation on the identification control on the interactive interface, an identification result of the target image after identification is displayed on the interactive interface, wherein the identification result includes source information of the target image, the source information of the target image is obtained by classifying estimated source information of the target image using a classifier of an image recognition model, and the estimated source information is obtained by source identification of a reconstructed image of the target image using a CSNet network; the reconstructed image is obtained by reconstruction of a measurement value of the target image using the CSNet network; the measurement value of the target image is obtained by sampling the target image using the CSNet network in response to a network of a self-encoder of the image recognition model being the CSNet network; the image recognition model is obtained by training based on a plurality of sets of sample images, and the plurality of sets of sample images include a sample image and source information of the sample image; and in a training process, pre-extracted original image source information of the sample image is a training target, and the original image source information is source information corresponding to an original image of the sample image.

10. An image processing method characterized by, Comprise: Display a plurality of sets of sample image data on an interactive interface, wherein the plurality of sets of sample image data include a sample image and source information of the sample image; Display original image source information of a sample image in the plurality of sets of sample image data on the interactive interface, wherein the original image source information is pre-extracted source information corresponding to an original image of the sample image; Display a model icon of an image recognition model on the interactive interface, wherein the image recognition model is trained by machine using the plurality of sets of sample image data, and is obtained by taking the original image source information of the sample image as a training target in a training process, the image recognition model includes a self-encoder, and a network of the self-encoder is a CSNet network; the CSNet network is used for sampling a target image to obtain a measurement value of the target image, reconstructing the measurement value to obtain a reconstructed image of the target image, and source identifying the reconstructed image to obtain estimated source information of the target image; and a classifier of the image recognition model is used for classifying the estimated source information to obtain source information of the target image.

11. An image processing apparatus characterized by comprising: Comprise: A first acquisition module is configured to acquire a target image; A first identification module is configured to identify source information of the target image using an image recognition model, wherein the image recognition model is obtained by training based on a plurality of sets of sample images, and the plurality of sets of sample images include a sample image and source information of the sample image; and in a training process, pre-extracted original image source information of the sample image is a training target, and the original image source information is source information corresponding to an original image of the sample image. The first identification module is configured to identify the source of the target image by performing the following steps: in response to the network of the auto-encoder of the image recognition model being a CSNet network, sampling the target image by using the CSNet network to obtain measurement values of the target image; reconstructing the measurement values by using the CSNet network to obtain a reconstructed image of the target image; identifying the source of the reconstructed image by using the CSNet network to obtain estimated source information of the target image; and classifying the estimated source information by using the classifier of the image recognition model to obtain the source information of the target image.

12. An image processing method, characterized by, The method comprises: obtaining a product image; identifying the source of the product image by using an image recognition model to obtain source information of the product image, wherein the source information includes a camera that captures the product image, and the image recognition model is obtained by training based on a plurality of sets of sample images, wherein the plurality of sets of sample images include sample images and source information of the sample images, and the image recognition model takes, as a training target, light response non-uniformity information of a raw image of a sample image extracted in advance during the training process; wherein the step of identifying the source of the product image by using the image recognition model to obtain the source information of the product image comprises: in response to the network of the auto-encoder of the image recognition model being a CSNet network, sampling the product image by using the CSNet network to obtain measurement values of the product image; reconstructing the measurement values by using the CSNet network to obtain a reconstructed image of the product image; identifying the source of the reconstructed product image by using the CSNet network to obtain estimated source information of the product image; and classifying the estimated source information by using the classifier of the image recognition model to obtain the source information of the product image.

13. A computer device, comprising: The method comprises: a memory and a processor, the memory stores a computer program; the processor is configured to execute the computer program stored in the memory, and the computer program, when executed, causes the processor to perform the image processing method of any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the image processing method of any one of claims 1 to 10.

Images