Method, device and computer readable storage medium for recognizing a rephotographed image

Abstract

The application provides a kind of rephotograph image identification method, device and computer readable storage medium, rephotograph image identification method includes: obtaining image to be identified;First identification model is input to the image to be identified, and first identification result is obtained, and first identification model is the lightweight model for carrying out rephotograph image identification to the image input;When first identification result meets first preset condition, first identification result and the image to be identified are input into second identification model, and second identification result is obtained, and second identification model is the heavyweight model for carrying out rephotograph image identification to the image input;When second identification result meets second preset condition, it is determined that the image to be identified is rephotograph image.The identification result of first identification model is used in the second identification model identification process, and the cooperation of first identification model and second identification model is enhanced, and the identification accuracy of rephotograph image is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a retouched image recognition method and device and a computer readable storage medium. BACKGROUND

[0002] At present, with the development of the Internet industry, matters handled offline are gradually converted to online server remote handling. For this reason, the online server usually needs to collect real information. However, retouched images will make the server receive edited, forged or tampered information, thereby leading to low credibility of the information received by the server or even a large amount of false information. Taking various real-name authentication of users (such as electronic bank account opening) as an example, the user needs to take a picture of his own certificate for uploading. However, part of the certificate pictures are not taken from the real certificate, but are formed by retouching the certificate pictures on the computer screen or the mobile phone screen. The certificate in these retouched pictures may not belong to the user himself, or may have been edited, forged or tampered, thereby leading to incorrect authentication information and the phenomenon of using other people's information to commit crimes.

[0003] However, due to the small difference between the retouched image and the real image, it is difficult to accurately identify the retouched image using the conventional recognition model. Therefore, how to accurately identify the retouched image has become a difficult problem for those skilled in the art. SUMMARY

[0004] The present application provides a retouched image recognition method, device and computer readable storage medium, aiming at the problem of low accuracy of the conventional model for retouched image. The present application uses a lightweight first recognition model and a heavy second recognition model to identify the retouched image, ensuring the identification speed of non-retouched photos and improving the accuracy of identifying retouched images with high identification difficulty. At the same time, the second recognition model takes the identification result of the first recognition model and the image to be identified as input, and identifies and judges the image to be identified again based on the identification result of the first recognition model, ensuring the cooperation degree of the first recognition model and the second recognition model, and finally improving the accuracy of identifying the retouched image.

[0005] In a first aspect, the present application provides a retouched image recognition method, which comprises:

[0006] obtaining an image to be identified;

[0007] inputting the image to be identified into a first recognition model to obtain a first identification result, the first recognition model being a lightweight model for identifying the input image as a retouched image;

[0008] When the first recognition result meets the first preset condition, the first recognition result and the to-be-recognized image are input into a second recognition model to obtain a second recognition result, the second recognition model being a heavy model used for flip image recognition of an input image;

[0009] When the second recognition result meets the second preset condition, it is determined that the to-be-recognized image is a flip image.

[0010] With reference to the first aspect of the present application, in a first possible implementation manner of the first aspect of the present application, the second recognition model comprises a feature extraction layer and a classification layer, and obtaining the second recognition result by inputting the first recognition result and the to-be-recognized image into the second recognition model comprises:

[0011] inputting the to-be-recognized image into the feature extraction layer to extract first classification features;

[0012] taking the first recognition result as second classification features;

[0013] inputting the first classification features and the second classification features into the classification layer to obtain the second recognition result through classification of the classification layer.

[0014] With reference to the first aspect of the present application, in a second possible implementation manner of the first aspect of the present application, the training of the first recognition model and the second recognition model comprises:

[0015] obtaining a plurality of images labeled with flip image features as a first training data set;

[0016] training a first original model based on the first training data set to obtain the first recognition model, the first original model being a light model.

[0017] determining a second training data set based on the first training data set and a recognition result of the first training data set by the trained first recognition model;

[0018] training a second original model based on the second training data set to obtain the second recognition model, the second original model being a heavy model.

[0019] With reference to the first aspect of the present application, in a third possible implementation manner of the first aspect of the present application, determining the second training data set based on the first training data set and a recognition result of the first training data set by the trained first recognition model comprises:

[0020] obtaining the recognition result of the first training data set output by the first recognition model;

[0021] correspondingly labeling the recognition result of the first training data set to each training data in the first training data set to obtain the second training data set.

[0022] In a fourth possible implementation of the first aspect of the present application, the training of the first recognition model and the second recognition model comprises:

[0023] obtaining first training data, the first training data being any one data in a first training data set, the first data set comprising a plurality of images labeled with features of the retouched images;

[0024] training a first original model based on the first training data to obtain a first output result, the first original model being an initialized lightweight model corresponding to the first recognition model;

[0025] labeling the first output result to the first training data to obtain second training data;

[0026] training a second original model based on the second training data to obtain a second output result, the second original model being an initialized heavyweight model corresponding to the second recognition model.

[0027] In a fifth possible implementation of the first aspect of the present application, the training of the recognition model further comprises:

[0028] determining a first loss result based on the first output result and an expected result of the first training data, and determining a second loss result based on the second output result and an expected result of the second training data;

[0029] determining a third loss result based on the first loss result and the second loss result;

[0030] optimizing parameters of the first original model and the second original model according to the third loss result.

[0031] In a sixth possible implementation of the first aspect of the present application, determining the third loss result based on the first loss result and the second loss result comprises:

[0032] determining a ratio of positive samples to negative samples in the first training data set;

[0033] determining a first weight of the first loss result and a second weight of the second loss result based on the ratio of the positive samples to the negative samples;

[0034] determining the third loss result based on the first loss result, the second loss result, the first weight and the second weight.

[0035] In a seventh possible implementation of the first aspect of the present application, the training of the first recognition model and the second recognition model further comprises:

[0036] determining a ratio of positive samples to negative samples in the first training data set;

[0037] determine the first loss result based on the ratio of positive samples to negative samples, the expected result of the first training data, and the first output result, and determine the second loss result based on the ratio of positive samples to negative samples, the expected result of the second training data, and the second output result;

[0038] determine the third loss result based on the first loss result and the second loss result;

[0039] optimize the parameters of the first original model and the second original model according to the third loss result.

[0040] In a second aspect, the present application provides a device for recognizing a retouched image, the device comprising:

[0041] an acquisition module configured to acquire an image to be recognized;

[0042] a first recognition module configured to input the image to be recognized into a first recognition model to obtain a first recognition result, the first recognition model being a lightweight model used for primary retouched image recognition of an input image;

[0043] a second recognition module configured to, when the first recognition result meets a first preset condition, input the first recognition result and the image to be recognized into a second recognition model to obtain a second recognition result, the second recognition model being a heavyweight model used for secondary retouched image recognition of an input image;

[0044] a determination module configured to, when the second recognition result meets a second preset condition, determine that the image to be recognized is a retouched image.

[0045] In a third aspect, the present application provides a device for recognizing a retouched image, the device comprising:

[0046] one or more processors;

[0047] a memory; and

[0048] one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the method for recognizing a retouched image in the first aspect.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to execute the steps in the method for recognizing a retouched image in the first aspect.

[0050] From the above, the present application has the following beneficial effects:

[0051] The application improves the recognition accuracy of the screen-captured image by comprehensively judging the recognition result of the screen-captured image through the first recognition model and the second recognition model, and meanwhile, the second recognition model uses the recognition result of the first recognition model in the recognition process, which enhances the cooperation of the first recognition model and the second recognition model and further improves the recognition accuracy of the screen-captured image.

[0052] In addition, the application uses the light-weight first recognition model and the heavy-weight second recognition model to recognize the screen-captured image, and in the case that the initial recognition is a non-screen-captured image, the recognition process is directly completed by using only the light-weight first recognition model, while in the case that the initial recognition is a screen-captured image, the heavy-weight second recognition model is used for re-recognition, which ensures the recognition rapidity of the non-screen-captured image and improves the accuracy of the recognition of the screen-captured image with high recognition difficulty. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0054] Figure 1 is a scene schematic diagram of a screen-captured image recognition system provided by an embodiment of the application;

[0055] Figure 2 is a flowchart of a screen-captured image recognition method provided by an embodiment of the application;

[0056] Figure 3 is a flowchart of a second recognition model recognition process provided by an embodiment of the application;

[0057] Figure 4 is a flowchart of training a first recognition model and a second recognition model provided by an embodiment of the application;

[0058] Figure 5 is a flowchart of determining a second training data set provided by an embodiment of the application;

[0059] Figure 6 is another flowchart of training a first recognition model and a second recognition model provided by an embodiment of the application;

[0060] Figure 7 is a flowchart of determining a third loss result provided by an embodiment of the application;

[0061] Figure 8 is a structural schematic diagram of a screen-captured image recognition device provided by an embodiment of the application;

[0062] Figure 9 is a structural schematic diagram of a photographing image recognition device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person skilled in the art without creative work fall within the scope of protection of the present application.

[0064] In the description of the present application, it should be understood that the terms "first", "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0065] In the present application, the word "exemplary" is used to mean "serving as an example, instance, or illustration." Any implementation described as "exemplary" in the present application is not necessarily to be construed as preferred or advantageous over other implementations. The following description is presented to enable any person skilled in the art to make and use the present application. In the following description, for the purpose of explanation, details are set forth. It is apparent to those skilled in the art that the present application can be practiced without using these specific details. In other instances, well-known structures and processes are not described in detail in order to avoid obscuring the description of the present application. Therefore, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0066] The present application provides a photographing image recognition method, device and computer readable storage medium, which are described in detail below.

[0067] First, refer to Figure 1 , Figure 1 a scene schematic diagram of a photographing image recognition system in an embodiment of the present application is shown.

[0068] In the present application, the retouched image recognition system can be applied to various authentication recognitions of retouched images and real images, where the retouched image refers to image information obtained by shooting other images that are not original images (for example, images displayed on a mobile phone, a computer display, a tablet computer, etc., and for example, images displayed on a printed copy). For example, the retouched image recognition system can be used in the financial industry to identify whether the certificate photo uploaded by a user to open an electronic bank is formed by retouching a certificate picture on a computer screen or a mobile phone screen. For another example, the retouched image recognition system can be used in online e-commerce registration to identify whether the business license uploaded by an enterprise to open an online e-commerce store is formed by retouching a certificate picture on a computer screen or a mobile phone screen. It is worth noting that the above application scenarios of the retouched image recognition system are only exemplary examples, and in addition, the retouched image recognition system can be used in various authentication recognitions of retouched images and real images in various industries such as economy, culture, education, medical treatment, and public management.

[0069] In some embodiments, the retouched image recognition system includes a server 110, a network 120, a storage device 130, and an image acquisition terminal 140. In some embodiments, the retouched image recognition system can identify whether the image is a picture on a computer screen or a mobile phone screen through the acquired image information, thereby avoiding the phenomenon of incorrect input of user information caused by retouched pictures.

[0070] The server 110 can process data and / or information from at least one component of the retouched image recognition system or an external data source (for example, the storage device 130, the image acquisition terminal 140), for example, the server 110 can acquire image information from the image acquisition terminal 140 to facilitate retouched identification, for another example, the server 110 can acquire training data from the storage device 130 to facilitate model training of the first identification model and the second identification model, and for another example, the server 110 can acquire data of the first identification model and the second identification model to facilitate retouched identification of the acquired image. In some embodiments, the server 110 can be a single server, or a server group. The server group can be a centralized server group connected to the network 120 via an access point, or a distributed server group connected to the network 120 via at least one access point, respectively. In some embodiments, the server 110 can be locally connected to the network 120 or remotely connected to the network 120. For example, the server 110 can access information and / or data stored in the storage device 130 via the network 120. In some embodiments, the server 110 can be implemented on a cloud platform. For example only, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc. or any combination thereof.

[0071] The network 120 connects the components of the rephotographed image recognition system such that the components can communicate with each other to facilitate the exchange of information and / or data. In some embodiments, at least one component of the rephotographed image recognition system (e.g., the server 110, the storage device 130, the image acquisition terminal 140) can send information and / or data (e.g., an image to be recognized) to other components of the rephotographed image recognition system via the network 120. In some embodiments, the network between the components of the rephotographed image recognition system can be any one or more of a wired network or a wireless network. For example, the network 120 can include a cable network, a wired network, a fiber optic network, a telecommunication network, an intranet, the Internet, a Local Area Network (LAN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a Metropolitan Area Network (MAN), a Public Switched Telephone Network (PSTN), a Bluetooth network, a ZigBee network, a Near Field Communication (NFC) network, an intra-device bus, an intra-device wire, a cable connection, or the like, or any combination thereof. The network connection between each two components can be in one of the above manners or in multiple manners.

[0072] The storage device 130 can store data and / or instructions. In some embodiments, the storage device 130 can store data obtained from the image acquisition terminal 140, e.g., the identity image data. For example, the storage device 130 can store training data for the first identification model, the second identification model. In some embodiments, the storage device 130 can store data and / or instructions that the server 110 can execute. In some embodiments, the storage device 130 can include a mass storage, a removable storage, a volatile read / write memory, a read only memory (ROM), etc., or any combination thereof. An exemplary mass storage can include a magnetic disk, an optical disk, a solid state disk, etc. An exemplary removable storage can include a flash drive, a floppy disk, an optical disk, a memory card, a compact disk, a magnetic tape, etc. An exemplary volatile read / write memory can include a random access memory (RAM). An exemplary RAM can include a dynamic random access memory (DRAM), a double-data-rate synchronous dynamic random access memory (DDR SDRAM), a static random access memory (SRAM), a thyristor random access memory (T-RAM), and a zero capacitance random access memory (Z-RAM), etc. An exemplary read only memory can include a masked read only memory (MROM), a programmable read only memory (PROM), a programmable erasable read only memory (PEROM), an electrically erasable programmable read only memory (EEPROM), a compact disc read only memory (CD-ROM), a digital versatile disk read only memory, etc. In some embodiments, the storage device 130 can be implemented on a cloud platform. For example only, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an on-premise cloud, a multi-layer cloud, etc., or any combination thereof.

[0073] The image acquisition terminal 140 can send the image data of the rephotographed certificate to the server 110. In some embodiments, the image acquisition terminal 140 can be a device with information receiving and / or sending functions to facilitate sending data to the server 110 for processing. In some embodiments, the image acquisition terminal 140 can be a plurality of terminals 141, 142, 143,..., with camera functions, for example, the image acquisition terminal 140 can include a mobile device 141, a tablet computer 142, a notebook computer 143, and the like, or any combination thereof.

[0074] It should be noted that the above description of the rephotographed image recognition system is only for example and illustration, and does not limit the scope of application of the present application. Those skilled in the art can make various modifications and changes to the rephotographed image recognition system under the guidance of the present application, for example, the rephotographed image recognition system can also include an alarm device connected to the server 110 to facilitate identification of the rephotographed image and alarm.

[0075] Next, the rephotographed image recognition method provided by the present application will be introduced.

[0076] First, refer to Figure 2 , Figure 2 is a flowchart of the rephotographed image recognition method in the embodiments of the present application. Specifically, the rephotographed image recognition method can include:

[0077] Step S201, obtaining an image to be recognized;

[0078] The image to be recognized is an image containing various certificates, wherein the certificates can include personal certificates or collective certificates, the personal certificates can be, for example, an ID card, a household booklet, a degree certificate, etc., and the collective certificates can include a business license, a qualification certificate, an honor certificate, etc. In some embodiments of the present application, for example, for embodiments in which the recognition process is real-time recognition, obtaining the image to be recognized can be that the server 110 receives data sent by the image acquisition terminal 140 in real time. In some other embodiments of the present application, for example, for embodiments in which the recognition process is a delayed process, obtaining the image to be recognized can be that the server 110 receives data sent by the storage device 130.

[0079] Step S202, inputting the image to be recognized into a first recognition model to obtain a first recognition result;

[0080] The first identification model refers to a model for pre-identifying the to-be-identified image. Specifically, the first identification model is a lightweight model. The lightweight model refers to a neural network model with few parameters, so as to quickly identify the result by the first identification model with fewer parameters. For example, the lightweight model can be a neural network model such as an object classification model (SqueezeNet), a mobile model (MobileNet), or a shuffle model (ShuffleNet).

[0081] The first identification result refers to a primary classification result of the to-be-identified image. The classification result can include a “reprinted image”, a “non-reprinted image”, a “non-identity photo”, and an “unidentifiable” classification result. Specifically, the first identification model can extract the features (such as convolution, pooling, etc.) of the to-be-identified image, and then classify the features of the to-be-identified image (such as through a fully connected layer), and finally obtain the score of each classification result through normalization (such as a Softmax function). The classification result with the highest score is the first identification result of the to-be-identified image. For example, the features of the to-be-identified image can be the color feature values corresponding to each pixel point of the to-be-identified image, and can also be the spectral features and texture features of the to-be-identified image.

[0082] In step S203, when the first identification result meets the first preset condition, the first identification result and the to-be-identified image are input into a second identification model to obtain a second identification result. The second identification model is a heavy model for secondary reprinted image identification of the input image.

[0083] The first preset condition refers to a condition for determining whether to further identify and confirm the to-be-identified image. For example, if the first identification result is a real identity image, no further identification is performed, and if the first identification result is a reprinted image, further identification is performed, so as to ensure the identification efficiency of the real identity image and improve the identification accuracy of the reprinted image. In some embodiments of the present application, the first preset condition can be that the score of the classification result in the first identification result meets a certain threshold. For example, if the classification result score corresponding to the “reprinted image” is greater than 0.7, further identification is performed, so as to re-identify and determine the to-be-identified image identified as the “reprinted image”. For example, if the classification result score corresponding to the “non-identity photo” is greater than 0.8, further identification is performed, so as to re-identify and determine the to-be-identified image identified as the “non-identity photo”. In some other embodiments of the present application, the first preset condition can be that the first identification result determines that the to-be-identified image is a “reprinted image”, and then the second identification model is used to reconfirm the to-be-identified image.

[0084] The second identification model refers to a model for further identifying the to-be-identified image. Specifically, the second identification model is a heavyweight model. The heavyweight model refers to a neural network model with more parameters, so as to improve the recognition accuracy by using the second identification model with more parameters and more accurate recognition. For example, the heavyweight model can be an efficient model (efficientnet), a classic neural network (Alexnet), or the like.

[0085] The second identification result is a re-classification result of the to-be-identified image. The classification result can include a "reprinted image", a "non-reprinted image", a "non-identity photo", and an "unidentifiable" classification result. Specifically, the first identification result and the to-be-identified image can be input into the second identification model to obtain the second identification result, so as to combine the identification result of the first identification model and improve the coordination of the first identification model and the second identification model. For example, the second identification model can extract the features of the to-be-identified image (such as convolution, pooling, or the like), and take the first identification result as a label, and then classify the features extracted from the to-be-identified image and the first identification result (for example, by using a full connection layer), and finally process by normalization (for example, a Softmax function) to obtain the score of each classification result. The classification result with the highest score is the second identification result of the to-be-identified image. For example, the features of the to-be-identified image can be the color feature value corresponding to each pixel point of the to-be-identified image, and can also be the frequency spectrum feature and the texture feature of the to-be-identified image.

[0086] In step S204, when the second identification result meets the second preset condition, the to-be-identified image is determined to be a reprinted image.

[0087] The second preset condition is a condition for identifying the to-be-identified image as a reprinted image, so as to finally confirm the identification result of the reprinted image. In some embodiments of the present application, the second preset condition can be that the score of the classification result in the second identification result meets a certain threshold. For example, the score of the classification result corresponding to the "reprinted image" is greater than 0.7, and the to-be-identified image is determined to be a reprinted image. For example, the score of the classification result corresponding to the "non-identity photo" is less than 0.1, and the to-be-identified image is determined to be a reprinted image. In some other embodiments of the present application, the first preset condition can also be that the classification result with the highest score in the second identification result is "reprinted image", and the to-be-identified image is determined to be a reprinted image.

[0088] It is worth noting that the above description of the retouched image recognition method is intended to clearly illustrate the verification process of the present application, and various modifications and changes can be made to the retouched image recognition method under the guidance of the present application for those skilled in the art, for example, the second recognition model can also be other deep convolutional neural network model (Convolutional Neural Networks, CNN), and for example, the first recognition result and the second recognition result can be the meaning of other character (for example, number) classification.

[0089] With reference to the drawings Figure 3 , Figure 3 is a flowchart of the second recognition model recognition process in the embodiments of the present application. In some embodiments of the present application, the second recognition model includes a feature extraction layer and a classification layer, wherein the feature extraction layer can be a convolutional layer, a pooling layer, etc. of a neural network, and the classification layer can be a fully connected layer, a Softmax layer, etc. of a neural network, and the second recognition model recognition process includes:

[0090] Step S301, inputting the to-be-recognized image into the feature extraction layer to extract first classification features;

[0091] The first classification features refer to features associated with the to-be-recognized image itself, such as pixel points, spectral features, texture features, etc. of the to-be-recognized image. Specifically, the feature extraction layer can be a convolutional layer, a pooling layer, etc. of a neural network, and extracting the first classification features can refer to performing convolution or pooling operations on the to-be-recognized image through the convolutional layer, the pooling layer, etc. of the neural network.

[0092] Step S302, taking the first recognition result as the second classification feature;

[0093] The second classification feature is the first recognition result of the to-be-recognized image in the first recognition model, and by taking the first recognition result as the classification feature of the second recognition model, the previous recognition result is utilized when classifying in the second recognition model, thereby improving the coordination of the first recognition model and the second recognition model.

[0094] Step S303, inputting the first classification feature and the second classification feature into the classification layer to obtain the second recognition result through classification of the classification layer.

[0095] After obtaining the first classification feature and the second classification feature, they can be input into the classification layer for classification, and finally the second recognition result is obtained. Specifically, the classification layer can be a fully connected layer, a Softmax layer, etc. of a neural network. Since the second recognition result is obtained by classifying the first classification feature and the second classification feature corresponding to the first recognition result, the classification result of the first recognition model is utilized in the classification process of the second recognition model, thereby improving the coordination and recognition accuracy of the first recognition model and the second recognition model.

[0096] It is worth noting that the above description of the identification process with respect to the second identification model is only exemplary, and various modifications and changes can be made to the identification process of the second identification model under the guidance of the present application, for example, random forest, decision tree, etc. can also be used as the second identification model for classification.

[0097] Referring to Figure 4 , Figure 4 is a flowchart of training the first identification model and the second identification model in an embodiment of the present application. Specifically, the training method can include:

[0098] Step S401, obtaining a plurality of images labeled with a retouched image feature as a first training data set;

[0099] The retouched image feature refers to a feature unique to the retouched image. For example, the retouched image feature can be a pixel point of the image, such as a pixel point corresponding to a retouched device (such as a screen frame, a mobile phone operation button, etc.) in the image, and can also be a feature held by the retouched image, such as a texture feature corresponding to a periodic color stripe on the retouched image. Specifically, obtaining a plurality of images labeled with a retouched image feature can be obtained by the server 110 receiving the images sent by the storage device 130, and can also be obtained by the server 110 receiving the images sent by the image acquisition device 140.

[0100] Step S402, training a first original model based on the first training data set to obtain a first identification model;

[0101] The first original model refers to a network model without optimized parameters. Specifically, the first original model is a lightweight model to form a first identification model with fewer parameters for fast pre-identification. For example, the first original model can be an object classification model (SqueezeNet), a mobile model (MobileNet), a shuffle model (ShuffleNet), etc.

[0102] Specifically, training the first original model based on the first training data set can input each training data in the first training data set into the first original model, compare the prediction result of the first original model with the expected result, and adjust the parameters of the first original model according to the prediction result and the expected result through the back propagation algorithm (BP algorithm) until the loss of the prediction result of the first original model and the expected result is less than a certain threshold, then the training of the first original model can be completed and the first identification model is obtained. For example, an exponential loss function, a cross-entropy loss function, a square loss function, a mean square error loss function, etc. can be used as the loss function.

[0103] In step S403, the second training data set is determined based on the first training data set and the recognition result of the first training data set by the trained first recognition model.

[0104] The second training data set refers to a data set for training the second recognition model. Specifically, the second training data set can be determined based on the first training data set and the recognition result of the first training data set by the trained first recognition model. Then, the recognition result of the first training data set is taken as the input of the second recognition model, so as to improve the coordination degree of the first recognition model and the second recognition model. In some embodiments of the present application, the recognition result of the first training data set can be labeled for each piece of training data in the first training data set, and finally the second training data set is obtained. Specifically, the labeling can refer to adding the recognition result of each piece of training data to the first training data set. For example, the training data A {image, expected result} is labeled, and finally the training data {image, expected result, recognition result of the first recognition model} is obtained.

[0105] In step S404, the second original model is trained based on the second training data set to obtain the second recognition model.

[0106] The second original model refers to a network model without optimized parameters. Specifically, the second original model is a heavyweight model, so as to form a second recognition model with more parameters and accurate recognition. For example, the second original model can be a neural network model such as efficient model (efficientnet) and classic neural network (Alexnet).

[0107] Specifically, based on the second training data set, each piece of training data in the second training data set can be input into the second original model, the prediction result of the second original model is compared with the expected result, and the parameters of the second original model are adjusted according to the prediction result and the expected result by the back propagation algorithm (BP algorithm), until the loss of the prediction result of the second original model and the expected result is less than a certain threshold, then the training of the second original model can be completed and the second recognition model is obtained. For example, an exponential loss function, a cross-entropy loss function, a square loss function, a mean square error loss function and the like can be used as the loss function.

[0108] It is worth noting that the above description of training the first recognition model and the second recognition model is intended to clearly illustrate the verification process of the present application, and those skilled in the art can make various modifications and changes to the training process under the guidance of the present application, for example, the second original model can also be other deep convolutional neural network model (Convolutional Neural Networks, CNN), and for example, the first training data set can be filtered or increased after adding part of the data as the second training data set, and for example, the first training data set can include a training set and a test set, and when the test set test result meets the requirement, the model meeting the requirement is used as the first recognition model.

[0109] With reference to the foregoing description of the verification process of the present application, the second training data set can be determined in the following manner. Figure 5 , Figure 5 A flowchart for determining the second training data set in an embodiment of the present application is shown in FIG. 5. Specifically, the method for determining the second training data set can include the following steps.

[0110] In step S501, the recognition result of the first training data set output by the first recognition model is obtained.

[0111] The recognition result of the first training data set refers to the recognition result of the first training data set obtained by the first recognition model after the parameters of the first recognition model are optimized and fixed, so as to use the recognition result of the first training data set as a part of the second training data set, and improve the cooperation degree of the first recognition model and the second recognition model. In some embodiments of the present application, for example, in the embodiment in which the storage device stores the recognition result of the first training data set by the first recognition model, the recognition result of the first training data set can be obtained by the server 110 receiving the data sent by the storage device 140.

[0112] In step S502, the recognition result of the first training data set is labeled to each piece of training data in the first training data set to obtain the second training data set.

[0113] After obtaining the recognition result of the first training data set, the second training data set of the second original model can be determined in combination with the first training data set, so that the second recognition model can be trained according to the recognition result of the first training data set, so as to improve the cooperation degree of the first recognition model and the second recognition model. In some embodiments of the present application, the recognition result of the first training data set can be labeled to each piece of training data in the first training data set to obtain the second training data set, and specifically, the labeling can refer to adding the recognition result of each piece of training data to the first training data, for example, the training data A {image, expected result} is labeled, and finally the training data {image, expected result, recognition result of the first recognition model} is obtained.

[0114] It is worth noting that the above content about determining the second training data set is intended to clearly illustrate the verification process of the present application, and various modifications and changes can be made to the process of determining the second training data set under the guidance of the present application, for example, the first training data set can be screened out or increased after part of the data is used as the second training data set.

[0115] The above Figure 4 And Figure 5 The process of training the first recognition model and the second recognition model separately in the present application, that is, the process of adjusting the parameters of the two models according to the loss of the prediction result and the expected result is independent and not associated with each other. The coordination degree of the first recognition model and the second recognition model is limited, therefore, the present application further proposes a training method for further improving the coordination degree of the first recognition model and the second recognition model.

[0116] Continuing to refer to Figure 6 , Figure 6 is another flowchart of training the first recognition model and the second recognition model in the embodiment of the present application. Specifically, the training process can include:

[0117] Step S601, obtaining first training data, the first training data being any one data in the first training data set, the first data set including a plurality of images labeled with features of a retouched image;

[0118] The first training data refers to any image data containing retouched image features or non-retouched image features, and a plurality of first training data participating in training constitute a first training data set. In some embodiments, the first training data set includes positive samples and negative samples, the positive samples and the negative samples referring to training data with opposite expected results in the first training data set. For example, the "retouched image" in the first training data set and the "non-retouched image" can be positive samples and negative samples, respectively. As another example, the "non-retouched image" in the first training data set can be a positive sample, and "retouched image", "non-identity photo" and other images can be negative samples. Specifically, obtaining the first training data can be obtained by the server 110 receiving the data sent by the storage device 140.

[0119] Step S602, training a first original model based on the first training data to obtain a first output result, the first original model being an initialized lightweight model corresponding to the first recognition model;

[0120] The first output result refers to a training prediction result of the first original model on the first training data. The training prediction result can include classification results such as “reprinted image”, “non-reprinted image”, “non-identity photo”, and “unrecognizable”. Specifically, the first original model can perform feature extraction (such as convolution and pooling) on the features of the first training data, then output classification results according to the features of the first training data (such as full connection layer classification), and finally process the classification results through normalization (such as Softmax function) to obtain scores of each classification result. The classification result with the highest score is the first output result of the first training data.

[0121] In step S603, the first output result is labeled to the first training data to obtain second training data.

[0122] When the first output result is determined, the second training data can be determined in combination with the first training data to ensure that the first recognition model and the second recognition model have differences and connections on the training data, thereby improving the coordination degree of the first recognition model and the second recognition model. In some embodiments of the present application, the first output result can be labeled to the first training data to finally obtain the second training data. For example, the first training data {image, expected result} is labeled to finally obtain the second training data {image, expected result, first output result}.

[0123] In step S604, a second original model is trained based on the second training data to obtain a second output result. The second original model is an initialized heavyweight model corresponding to the second recognition model.

[0124] The second output result refers to a training prediction result of the second original model on the second training data. The training prediction result can include classification results such as “reprinted image”, “non-reprinted image”, “non-identity photo”, and “unrecognizable”. Specifically, the second original model can perform feature extraction (such as convolution and pooling) on the features of the second training data, then output classification results according to the extracted features of the second training data and the first output result (such as through full connection layer classification), and finally process the classification results through normalization (such as Softmax function) to obtain scores of each classification result. The classification result with the highest score is the second output result of the second training data.

[0125] In step S605, a first loss result is determined based on the first output result and an expected result of the first training data.

[0126] The first loss result refers to a difference value between the first output result of the first original model predicting the first training data and an expected result, where the expected result refers to an expected prediction value for the training data, for example, for training data of a "screen shot image", the expected result can be 1, and for example, for training data of a "non-screen shot image", the expected result can be 0. Illustratively, the difference value between the first output result and the expected result can be an absolute value difference, for example, the first output result is a predicted value of a "screen shot image" of the first training data, which is 0.7, and the actual expected result of the "screen shot image" of the first training data is 1, and the first loss result is 0.3. As another illustrative example, the difference value between the first output result and the expected value can also be a squared difference of an absolute value, an exponential difference, a logarithmic difference, etc.

[0127] In some embodiments of the present application, the proportion of positive samples and negative samples in the first training data set can also be determined, and then the first loss result is determined according to the proportion of positive samples and negative samples in the training data together with the first output result. For training data with a proportion of positive samples and negative samples of 10:30, in order to strengthen the punishment for the identification error of the positive samples, for example, the first output result is a predicted value of a "screen shot image" of the first training data, which is 0.7, and the actual expected value of the "screen shot image" of the first training data is 1, and the first loss result is 0.9, i.e. the first loss result is enlarged by 3 times according to the proportion of positive samples and negative samples, to strengthen the punishment for the identification error of the positive samples, so as to improve the accuracy of the first identification model for the positive samples.

[0128] Step S606, determining a second loss result based on the second output result and the expected result of the second training data;

[0129] The second loss result refers to a difference value between the second output result of the second original model predicting the first training data and an expected value. Illustratively, the difference value between the second output result and the expected value can be an absolute value difference, for example, the second output result is a predicted value of a "screen shot image" of the first training data, which is 0.6, and the actual expected value of the "screen shot image" of the second training data is 1, and the second loss result is 0.4. As another illustrative example, the difference value between the second output result and the expected value can also be a squared difference of an absolute value, an exponential difference, a logarithmic difference, etc.

[0130] In some embodiments of the present application, the proportion of positive samples and negative samples in the first training data set can also be determined, and then the proportion of positive samples and negative samples in the training data is determined together with the second output result to determine the second loss result. For training data with a proportion of positive samples and negative samples of 10:40, in order to strengthen the punitive effect of the identification error of positive samples, the second output result is that the predicted value of the "rephotographed image" of the second training data is 0.6, and the actual expected value of the "rephotographed image" of the second training data is 1, then the second loss result is 1.6, that is, the second loss result is enlarged by 4 times according to the proportion of positive samples and negative samples, and the punitive effect of the identification error of positive samples is strengthened, so as to improve the accuracy of the second identification model for positive samples.

[0131] In step S607, a third loss result is determined based on the first loss result and the second loss result, and the parameters of the first original model and the second original model are optimized according to the third loss result.

[0132] When the first loss result and the second loss result corresponding to the first data are determined, the parameters of the first original model and the second original model can be optimized in combination with the first loss result and the second loss result, so as to improve the coordination degree of the first original model and the second original model. Specifically, the first loss result and the second loss result can be comprehensively processed into a third loss result, and the parameters of the first original model and the second original model are adjusted and optimized according to the third loss result. The comprehensive processing can be summing, weighted summing, square summing, etc. Since the third loss result obtained by the first loss result and the second loss result is used to adjust and optimize the parameters of the first original model and the second original model in the back propagation, the parameters of the first original model are optimized in association with the second loss result, and the parameters of the second original model are optimized in association with the first loss result, so that the first identification model and the second identification model with further enhanced coordination degree can be obtained after the training is completed.

[0133] When the first original model and the second original model reach the training end condition, the first identification model and the second identification model with optimized parameters can be obtained. For example, the training end condition can be that each first training data in the training data set is trained. As another example, the training end condition can be that the first loss result and the second loss result are less than a certain threshold, for example, less than 0.05.

[0134] It is worth noting that the above content about training the first identification model and the second identification model is an example of one training data, and the above process can be repeated for other data of the first training data set. In addition, the above content about training the first identification model and the second identification model is intended to clearly explain the verification process of the present application, and those skilled in the art can make various modifications and changes to the training process under the guidance of the present application, for example, the first original model can also be other deep convolutional neural network model (Convolutional Neural Networks, CNN), and for example, the training end condition can also be that the recognition accuracy of the test set reaches 80%.

[0135] With reference to the foregoing Figure 7 , Figure 7 is a flowchart for determining the third loss result in an embodiment of the present application. Specifically, the optimization process can include:

[0136] Step S701, determining the ratio of positive samples and negative samples in the first training data set;

[0137] The first training data set refers to the data set for training the first original model and the second original model. Specifically, the first training data set is composed of the first training data participating in the training.

[0138] The positive sample and the negative sample refer to the training data with opposite expected results in the first training data set. For example, the "repeated picture" and the "non-repeated picture" in the first training data set can be positive samples and negative samples, respectively. As another example, the "non-repeated picture" in the first training data set can be a positive sample, and other pictures such as "repeated pictures" and "non-identity photos" can be negative samples. Specifically, the ratio can be determined by the number of positive samples and negative samples in the first training data set, for example, the number of positive samples is 1000, and the number of negative samples is 100, then the ratio of positive and negative samples can be determined as 10:1.

[0139] Step S702, determining the first weight of the first loss result and the second weight of the second loss result based on the ratio of the positive sample and the negative sample;

[0140] The first weight refers to a weight proportion of the first loss result determining the third loss result, and the second weight refers to a weight proportion of the second loss result determining the third loss result. Specifically, the first weight and the second weight can be determined according to the ratio of positive samples to negative samples. For example, when the ratio difference of positive samples to negative samples is large (for example, 100:5), the number of negative samples is small, and therefore the identification accuracy of the second identification model needs to be ensured. Therefore, the second weight can be set to a large value, for example, the second weight can be set to 0.6, and the first weight can be set to 0.4, so that the second original model can be optimized mainly based on the second loss result when the first original model and the second original model are optimized based on the third loss result. As another example, when the ratio difference of positive samples to negative samples is small (for example, 100:100), the number of negative samples is large, and therefore the identification accuracy of the first identification model needs to be ensured, so that the second identification model is used to identify as many difficult-to-identify pictures as possible. Therefore, the first weight can be set to a large value, for example, the first weight can be set to 0.6, and the second weight can be set to 0.4, so that the first original model can be optimized mainly based on the first loss result when the first original model and the second original model are optimized based on the third loss result.

[0141] In step S703, the third loss result is determined based on the first loss result, the second loss result, the first weight, and the second weight.

[0142] After the first weight of the first loss result and the second weight of the second loss result are determined, the third loss result for adjusting the first original model and the second original model can be determined. For example, the third loss result can be determined by weighting and adding the first loss result and the second loss result, for example, the first loss result is 0.3, the second loss result is 0.2, the first weight is 0.6, and the second weight is 0.4, and the third loss result can be determined as 0.26. As another example, the third loss result can be determined by weighting and adding the squares of the first loss result and the second loss result, for example, for the first loss result, the second loss result, the first weight, and the second weight in the above example, the third loss result can be determined as 0.07.

[0143] It is worth noting that the above content about determining the third loss result is intended to clearly explain the verification process of the present application. Those skilled in the art can make various modifications and changes to the training process under the guidance of the present application, for example, the first loss result and the second loss result can be divided by the first weight and the second weight respectively, and then added to obtain the third loss result.

[0144] In order to better implement the retouched image recognition method in the embodiments of the present application, on the basis of the retouched image recognition method, the present application further provides a retouched image recognition device, as shown in Figure 8 The retouched image recognition device includes:

[0145] The acquisition module 801 is configured to acquire a to-be-recognized image.

[0146] The first recognition module 802 is configured to input the to-be-recognized image into a first recognition model to obtain a first recognition result, the first recognition model being a lightweight model used for retouched image recognition of an input image.

[0147] The second recognition module 803 is configured to, when the first recognition result meets a first preset condition, input the first recognition result and the to-be-recognized image into a second recognition model to obtain a second recognition result, the second recognition model being a heavyweight model used for retouched image recognition of an input image.

[0148] The determination module 804 is configured to, when the second recognition result meets a second preset condition, determine that the to-be-recognized image is a retouched image.

[0149] In some embodiments of the present application, the second recognition module 803 is specifically configured to:

[0150] input the to-be-recognized image into a feature extraction layer to extract a first classification feature;

[0151] use the first recognition result as a second classification feature;

[0152] input the first classification feature and the second classification feature into a classification layer to obtain the second recognition result through classification of the classification layer.

[0153] In some embodiments of the present application, the first recognition module 802 and the second recognition module 803 are specifically configured to train the first recognition model and the second recognition model, wherein the training of the first recognition model and the second recognition model includes:

[0154] acquire a plurality of images labeled with retouched image features as a first training data set;

[0155] train a first original model based on the first training data set to obtain the first recognition model, the first original model being a lightweight model;

[0156] determine a second training data set based on the first training data set and a recognition result of the first training data set by the trained first recognition model;

[0157] train a second original model based on the second training data set to obtain the second recognition model, the second original model being a heavyweight model.

[0158] In some embodiments of the present application, the second identification module 803 is specifically configured to:

[0159] obtain the identification result of the first training data set output by the first identification model;

[0160] corresponding labels the identification result of the first training data set to each piece of training data in the first training data set to obtain a second training data set.

[0161] In some embodiments of the present application, the first identification module 802 and the second identification module 803 are specifically configured to train the first identification model and the second identification model, wherein the training of the first identification model and the second identification model comprises:

[0162] obtain first training data, the first training data being any data in the first training data set, and the first data set comprising a plurality of images labeled with features of a retouched image;

[0163] train a first original model based on the first training data to obtain a first output result, the first original model being an initialized lightweight model corresponding to the first identification model;

[0164] label the first output result to the first training data to obtain second training data;

[0165] train a second original model based on the second training data to obtain a second output result, the second original model being an initialized heavy model corresponding to the second identification model.

[0166] In some embodiments of the present application, the first identification module 802 and the second identification module 803 are specifically configured to:

[0167] determine a first loss result based on the first output result and an expected result of the first training data, and determine a second loss result based on the second output result and an expected result of the second training data;

[0168] determine a third loss result based on the first loss result and the second loss result;

[0169] optimize parameters of the first original model and the second original model according to the third loss result.

[0170] In some embodiments of the present application, the first identification module 802 and the second identification module 803 are specifically configured to:

[0171] determine a ratio of positive samples to negative samples in the first training data set;

[0172] determine a first weight of the first loss result and a second weight of the second loss result based on the ratio of the positive samples to the negative samples;

[0173] The third loss result is determined based on the first loss result, the second loss result, the first weight, and the second weight.

[0174] In some embodiments of the present application, the first identification module 802 and the second identification module 803 are specifically configured to:

[0175] determine the ratio of positive samples to negative samples in the first training data set;

[0176] determine the first loss result based on the ratio of positive samples to negative samples, the expected result of the first training data, and the first output result, and determine the second loss result based on the ratio of positive samples to negative samples, the expected result of the second training data, and the second output result;

[0177] determine the third loss result based on the first loss result and the second loss result;

[0178] optimize the parameters of the first original model and the second original model according to the third loss result.

[0179] The retouched image identification device in the present application identifies images through a lightweight first identification model and a heavyweight second identification model, ensuring the rapidity of identifying non-retouched photos, and improving the accuracy of identifying retouched images with high identification difficulty. At the same time, the second identification model takes the identification result of the first identification model and the image to be identified as input together, and identifies and judges the image to be identified again on the basis of the identification result of the first identification model, ensuring the cooperation degree of the first identification model and the second identification model, and improving the accuracy of identifying retouched images.

[0180] It should be understood that Figure 8 The apparatus and its modules shown can be implemented in various ways. For example, in some embodiments, the apparatus and its modules can be implemented in hardware, software, or a combination of software and hardware. The hardware portion can be implemented with special logic, while the software portion can be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art can understand that the above-mentioned methods and systems can be implemented using computer executable instructions and / or included in processor control code, such as provided on a carrier medium, such as a magnetic disk, CD or DVD-ROM, a programmable memory, such as a read-only memory (firmware), or a data carrier, such as an optical or electronic signal carrier. The system and its modules of the present application can not only be implemented by hardware circuits, such as very large scale integrated circuits or gate arrays, semiconductors, such as logic chips, transistors, or programmable hardware devices, such as field programmable gate arrays, programmable logic devices, etc., but also by software, for example, executed by various types of processors, and also by a combination of the above-mentioned hardware circuits and software (for example, firmware).

[0181] It should be noted that the above description of the device and its modules is for the convenience of description only, and cannot limit the application within the scope of the embodiments. It can be understood that, for those skilled in the art, after understanding the principles of the system, the modules can be combined arbitrarily or connected with other modules to form a subsystem without departing from the principles. For example, Figure 8 The acquisition module 801, the first identification module 802, the second identification module 803, and the determination module 804 disclosed in the embodiment can be different modules in a system, or one module can implement the functions of two or more modules, for example, the first identification module 802 and the second identification module 803 can be two modules with identification functions respectively, or one module with identification functions.

[0182] In order to better implement the method for identifying a flipped image in the embodiment of the application, on the basis of the method for identifying a flipped image, the embodiment of the application further provides a flipped image identification device which integrates any of the flipped image identification devices provided in the embodiment of the application. The system comprises:

[0183] one or more processors;

[0184] a memory; and

[0185] one or more application programs, wherein the one or more application programs are stored in the memory and are configured to execute, by the processor, the steps in the method for identifying a flipped image according to any one of the embodiments of the method for identifying a flipped image.

[0186] As Figure 9 shown, a structure diagram of a flipped image identification method system is shown, and specifically:

[0187] The flipped image identification method system can include a processor 901 with one or more processing cores, a memory 902 with one or more computer readable storage media. Those skilled in the art can understand that Figure 9 the structure shown in the embodiment does not constitute a limitation on the flipped image identification method system, and can include more or fewer components than those shown, or combine certain components, or different component arrangements. Among them:

[0188] The processor 901 is the control center of the system, connects various parts of the whole system by using various interfaces and lines, executes various functions of the system and processes data by running or executing software programs and / or modules stored in the memory 902 and calling data stored in the memory 902, thereby overall monitoring the system. Optionally, the processor 901 can include one or more processing cores; the processor 901 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like, preferably, the processor 901 can integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems, user interfaces, application programs and the like, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 901.

[0189] The memory 902 can be used to store software programs and modules, and the processor 901 executes various functions and data processing by running the software programs and modules stored in the memory 902. The memory 902 can mainly include a program storage area and a data storage area, wherein the program storage area can store operating systems, application programs required by at least one function (such as sound playing function, image playing function, etc.), etc.; the data storage area can store data created according to the use of the method system for identifying a photographed image, etc. In addition, the memory 902 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 902 can also include a memory controller to provide the processor 901 with access to the memory 902.

[0190] Although not shown, the method system for identifying a photographed image can also include a display unit and the like, which will not be described here. Specifically, in the embodiment, the processor 901 in the method system for identifying a photographed image will load the executable file corresponding to the process of one or more application programs into the memory 902 according to the following instructions, and run the application programs stored in the memory 902 by the processor 901, thereby realizing various functions, as follows:

[0191] Obtaining an image to be identified;

[0192] inputting the to-be-identified image into a first identification model to obtain a first identification result, the first identification model being a lightweight model used for flip image identification on an input image;

[0193] when the first identification result meets a first preset condition, inputting the first identification result and the to-be-identified image into a second identification model to obtain a second identification result, the second identification model being a heavyweight model used for flip image identification on an input image;

[0194] when the second identification result meets a second preset condition, determining that the to-be-identified image is a flip image.

[0195] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by related hardware controlled by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0196] To this end, an embodiment of the present application provides a computer readable storage medium, which can include a read-only memory, a random access memory, a magnetic disk or an optical disk, etc. A computer program is stored on the storage medium, and the computer program is loaded by a processor to execute steps in any flip image identification method provided by the embodiments of the present application. For example, the computer program loaded by the processor can execute the following steps:

[0197] obtaining a to-be-identified image;

[0198] inputting the to-be-identified image into a first identification model to obtain a first identification result, the first identification model being a lightweight model used for flip image identification on an input image;

[0199] when the first identification result meets a first preset condition, inputting the first identification result and the to-be-identified image into a second identification model to obtain a second identification result, the second identification model being a heavyweight model used for flip image identification on an input image;

[0200] when the second identification result meets a second preset condition, determining that the to-be-identified image is a flip image.

[0201] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the detailed description of other embodiments above, which will not be repeated here.

[0202] Having described the basic concepts, it is obvious to those skilled in the art that the foregoing detailed description of the application is intended to be illustrative only and not limiting of the scope of the application as set forth in the claims. Various modifications, improvements, and changes can occur to those skilled in the art upon reading this description. Such modifications, improvements, and changes are intended to be within the spirit and scope of the application and are intended to be encompassed by the claims.

[0203] Also, the use of "one embodiment", "an embodiment", or "some embodiments" throughout this specification is not necessarily a reference to the same embodiment or embodiments. Furthermore, words such as "that", "said", or the like are not used as the singular or exclusive alternatives in this description, but rather as an open transition meaning "one or more" while following the description provided above so as not to unnecessarily clutter the disclosure with terminology that will be appreciated by those skilled in the art upon a reading of the above description.

[0204] Accordingly, various aspects of the application can be thought of as being entirely hardware-based, entirely software-based (including firmware, resident software, microcode, etc.), or combinations of hardware and software that can all generally be collectively referred to as a "circuit," "module," "engine," "unit," "component", or "system." In addition, various aspects of the application can be provided, produced, or manufactured as a computer program product having computer readable program code embodied thereon. The computer readable program code can be downloaded from a network or a data store.

[0205] A computer storage medium can include a propagated data signal with computer program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer storage medium can be any medium that can be used to store computer program code momentarily, temporarily, or permanently when facilitating a process, such as a computer program code that is written into a RAM of a computer system for execution by a processor. A computer storage medium can be any available medium that can be accessed by a general purpose or special purpose computer system including the Internet, intranet, LAN, WAN, or any suitable combination thereof. The computer storage medium can be a computer readable storage medium that can include a tangible computer readable storage medium or a computer readable transmission medium.

[0206] It should be noted that while the above description of the application has sometimes used language that could be characterized as meaning that certain features are preferred or particularly advantageous over others, no limitation of the scope of the application is intended by such language. Rather, decisions should be directed by the preambles of the claims, rather than by a description of the application. Furthermore, it should be noted that while the description of the application has included description of one or more embodiments, features of the application can be combined or used in other combinations than the ones specifically referred to in the above description.

[0207] The above provides a detailed introduction to the copy image recognition method, device, system and computer storage medium provided by the embodiment of the application. The principle and implementation mode of the application are described by applying specific examples. The above embodiment is only used to help understand the method of the application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the application, the specific implementation mode and application range will be changed. In conclusion, the content of the specification should not be understood as a limitation of the application.

Claims

1. A method for recognizing reproduced images, characterized in that, include: Acquire the image to be recognized; The image to be identified is input into the first recognition model to obtain the first recognition result. The first recognition model is a lightweight model used for recognizing the image of the input image. When the first recognition result meets the first preset condition, the first recognition result and the image to be recognized are input into the second recognition model to obtain the second recognition result. The second recognition model is a heavyweight model for recognizing the input image by re-photographing. When the second recognition result meets the second preset condition, the image to be recognized is determined to be a reproduced image; wherein... The second recognition model includes a feature extraction layer and a classification layer. The step of inputting the first recognition result and the image to be recognized into the second recognition model to obtain the second recognition result includes: The image to be identified is input into the feature extraction layer to extract the first classification features; The first identification result is used as the second classification feature; The first classification feature and the second classification feature are input into the classification layer, and the second recognition result is obtained by classification by the classification layer.

2. The method as described in claim 1, characterized in that, The training of the first recognition model and the second recognition model includes: Obtain multiple images labeled with features of reproduced images, and use them as the first training dataset; The first original model is trained based on the first training dataset to obtain the first recognition model. The first original model is a lightweight model. Based on the first training dataset and the recognition results of the trained first recognition model on the first training dataset, the second training dataset is determined. The second original model is trained based on the second training dataset to obtain the second recognition model. The second original model is a heavyweight model.

3. The method as described in claim 2, characterized in that, The determination of the second training dataset based on the recognition results of the first training dataset and the trained first recognition model on the first training dataset includes: Obtain the recognition results of the first training dataset output by the first recognition model; The recognition results of the first training dataset are labeled to each training data point in the first training dataset to obtain the second training dataset.

4. The method as described in claim 1, characterized in that, The training of the first recognition model and the second recognition model includes: Obtain first training data, which is any one data point in the first training dataset, which includes multiple images labeled with features of reproduced images; A first original model is trained based on the first training data to obtain a first output result. The first original model is an initialized lightweight model corresponding to the first recognition model. The first output result is labeled with the first training data to obtain the second training data; The second original model is trained based on the second training data to obtain the second output result. The second original model is the initialized heavyweight model corresponding to the second recognition model.

5. The method as described in claim 4, characterized in that, The training of the first recognition model and the second recognition model also includes: A first loss result is determined based on the first output result and the expected result of the first training data, and a second loss result is determined based on the second output result and the expected result of the second training data; A third loss result is determined based on the first loss result and the second loss result; The parameters of the first original model and the second original model are optimized based on the third loss result.

6. The method as described in claim 5, characterized in that, The determination of the third loss result based on the first loss result and the second loss result includes: Determine the ratio of positive to negative samples in the first training dataset; The first weight of the first loss result and the second weight of the second loss result are determined based on the ratio of the positive samples to the negative samples. The third loss result is determined based on the first loss result, the second loss result, the first weight, and the second weight.

7. The method as described in claim 4, characterized in that, The training of the first recognition model and the second recognition model also includes: Determine the ratio of positive to negative samples in the first training dataset; A first loss result is determined based on the ratio of positive samples to negative samples, the expected result of the first training data, and the first output result; and a second loss result is determined based on the ratio of positive samples to negative samples, the expected result of the second training data, and the second output result. A third loss result is determined based on the first loss result and the second loss result; The parameters of the first original model and the second original model are optimized based on the third loss result.

8. A device for recognizing reproduced images, characterized in that, The device includes: The acquisition module is used to acquire the image to be recognized; The first recognition module is used to input the image to be recognized into the first recognition model to obtain the first recognition result. The first recognition model is a lightweight model used to perform image recognition on the input image. The second recognition module is used to input the first recognition result and the image to be recognized into the second recognition model when the first recognition result meets the first preset condition, so as to obtain the second recognition result. The second recognition model is a heavyweight model for recognizing the input image by taking a picture. The second recognition model includes a feature extraction layer and a classification layer. The determining module is used to determine that the image to be identified is a reproduced image when the second recognition result meets the second preset condition; wherein, The second recognition module is specifically used to input the image to be recognized into the feature extraction layer to extract a first classification feature; use the first recognition result as a second classification feature; input the first classification feature and the second classification feature into the classification layer, and obtain the second recognition result through classification by the classification layer.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to execute the steps of the image recognition method according to any one of claims 1 to 7.

Images