Copy image recognition method and device, electronic equipment and storage medium

Abstract

The invention provides a copied image recognition method and device, electronic equipment and a storage medium, and relates to the field of artificial intelligence, and the method comprises the steps: obtaining a clear frame image in a to-be-detected item, the clear frame image being the clearest image in the to-be-detected item; performing feature extraction on the clear frame image by using a gray level co-occurrence matrix method to obtain a first image feature; performing spectrogram feature extraction on the clear frame image to obtain a second image feature; fusing the first image feature and the second image feature to obtain a fused feature; and inputting the fusion feature into a neural network model to obtain a detection result output by the neural network model. According to the scheme, the recognition of the copied image does not depend on GPS positioning information, and the application range is wide; and the duplicated image is identified based on the texture features of the image and the frequency distribution features of the image, so that the accuracy of duplicating identification is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic equipment and storage medium for re-photographed image recognition. Background Art

[0002] Mobile communication wireless base stations are key infrastructure in mobile communication networks. To ensure the safety and reliability of wireless signal transmission, it is necessary to regularly maintain and inspect the equipment in the wireless base stations to understand their operating status and changes in the surrounding environment. Timely discovery of equipment defects and hidden dangers that endanger equipment safety is an indispensable part of the daily network maintenance work of communication operators. The use of artificial intelligence to carry out wireless base station acceptance and inspection has many advantages, such as improving work efficiency, reducing costs, improving accuracy, preventive maintenance, improving safety, traceability, optimizing resource allocation, and improving service quality. However, in the wireless base station acceptance and inspection based on AI image recognition technology, there will be problems of cheating by inspectors, such as inspectors taking screenshots or printing out historical images or images collected online instead of taking physical photos in the machine room. This cheating behavior may cause the intelligent inspection and acceptance system to be unable to accurately identify the equipment status in the wireless base station, thereby affecting the accuracy and reliability of the inspection. In response to this type of cheating problem, corresponding measures need to be taken to solve and optimize it to ensure the normal acceptance and inspection of wireless base stations.

[0003] At present, watermarks with location and time information are usually added to the photos when taking photos, which can prevent the problem of falsification of security inspection results to a certain extent. However, due to the instability of GPS positioning services, the positioning results of equipment rooms deployed in basements, mountainous areas and other places with poor GPS signals will be inaccurate, and relying on GPS positioning to determine whether the inspectors are present is unreliable.

[0004] Therefore, how to identify whether the image of the wireless base station inspection is a re-shot image has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a method, device, electronic equipment and storage medium for recognizing a re-photographed image, so as to solve the defect that the re-photographed image cannot be accurately detected in the prior art.

[0006] The present invention provides a method for identifying a re-shot image, comprising the following steps: Acquire a clear frame image in the item to be detected, wherein the clear frame image is the clearest image in the item to be detected; Using a gray level co-occurrence matrix method to extract features from the clear frame image, to obtain a first image feature; Extracting the features of the spectrum graph of the clear frame image to obtain a second image feature; fusing the first image feature and the second image feature to obtain a fused feature; The fusion features are input into a neural network model to obtain a detection result output by the neural network model.

[0007] According to a method for identifying a re-shot image provided by the present invention, before obtaining a clear frame image in the item to be detected, the method further includes: Calculating the Laplacian operator of each frame image in the item to be detected to obtain the Laplacian operator matrix of each frame image; For any frame image, calculating the variance of the Laplace operator corresponding to the image; The maximum variance image is used as the clear frame image, and the maximum variance image is a frame image with the largest variance.

[0008] According to a method for recognizing a re-shot image provided by the present invention, the method of extracting features from the clear frame image using a gray level co-occurrence matrix method to obtain a first image feature includes: Performing grayscale processing on the clear frame image to obtain a grayscale image; Based on a preset sliding window size and a preset sliding window direction, calculating a gray level co-occurrence matrix of the gray level image, wherein the number of the gray level co-occurrence matrices is the same as the number of the preset sliding window directions; Merging the gray-level co-occurrence matrices to obtain a total gray-level co-occurrence matrix of the clear frame image; The total gray level co-occurrence matrix is ​​normalized to obtain the first image feature.

[0009] According to a method for identifying a re-shot image provided by the present invention, extracting a spectrum feature of the clear frame image to obtain a second image feature includes: Performing grayscale processing on the clear frame image to obtain a grayscale image; After Fourier transform processing is performed on the grayscale image, a center shift is performed on the frequency of the Fourier transform result to move the low-frequency part to the center position to obtain a shifted spectrum diagram; After calculating the modulus of the offset spectrum graph, taking the logarithm of the modulus spectrum graph to obtain a logarithmic spectrum graph; A normalization operation is performed on the logarithmic frequency spectrum to obtain the second image feature.

[0010] According to a method for recognizing a re-photographed image provided by the present invention, the neural network model is obtained based on the following steps: Acquire a real image set and a reproduced image set, wherein the real image set includes a plurality of real images, and the reproduced image set includes a plurality of reproduced images; Acquire the fusion features of each image in the real image set and the fusion features of each image in the re-photographed image set; Based on the fusion features of each image in the real image set and the fusion features of each image in the re-shot image set, the neural network model to be trained is trained to obtain the neural network model.

[0011] According to a reproduced image recognition method provided by the present invention, the real image set includes a plurality of real clear frame images, and the reproduced image set includes a printed reproduced image subset and a screen-captured reproduced image subset; The images in the print-reproduced image subset are images obtained by reproducing the printed paper; The images in the screen-shot reproduced image subset are images obtained by reshooting images displayed on the screen, and the screen-shot reproduced image subset includes a plurality of reproduced images with moiré patterns and a plurality of reproduced images without moiré patterns.

[0012] The present invention also provides a re-photographed image recognition device, comprising the following modules: An image acquisition module is used to: acquire a clear frame image in the item to be detected, wherein the clear frame image is the clearest image in the item to be detected; A first feature extraction module is used to: extract features from the clear frame image using a gray level co-occurrence matrix method to obtain a first image feature; A second feature extraction module is used to: extract the spectrum feature of the clear frame image to obtain a second image feature; A feature fusion module, used to: fuse the first image feature and the second image feature to obtain a fused feature; The model prediction module is used to: input the fusion features into the neural network model to obtain the detection results output by the neural network model.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the reproduced image recognition method as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for identifying a reproduced image as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for recognizing a reproduced image as described above is implemented.

[0016] The method, device, electronic device and storage medium for identifying reproduced images provided by the present invention obtain a clear frame image in an item to be detected, wherein the clear frame image is the clearest image in the item to be detected, that is, the clearest image in the item to be detected is obtained, so as to avoid blurring and out-of-focus phenomena caused by improper optical focal length, ensure the clarity of the texture of the surface of the photographed object, and provide a reliable data basis for subsequent reproduced recognition; use a gray level co-occurrence matrix method to extract features of the clear frame image to obtain a first image feature, and the gray level co-occurrence matrix method can effectively extract the texture feature of the image by counting the gray levels of pixels in the image and the relative position relationship between them; perform spectrum map feature extraction on the clear frame image to obtain a second image feature, and the spectrum map feature of the image describes the frequency distribution of pixel values ​​in the image; fuse the first image feature and the second image feature to obtain a fused feature; input the fused feature into a neural network model to obtain a detection result output by the neural network model. This solution extracts features from images and then performs re-photographed image recognition based on the extracted features. It does not rely on GPS positioning information and has a wide range of applications. It can still effectively perform re-photographed image recognition in a computer room with poor signals. The first image feature and the second image feature are fused, that is, the texture feature of the image is combined with the frequency distribution feature of the image, and then recognition and detection are performed through a neural network model to improve the accuracy of re-photographed recognition. Clear frame images in the items to be recognized are used for recognition to ensure the clarity of the image, provide reliable image data for re-photographed recognition, and further improve the accuracy of re-photographed recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is one of the flow charts of the reproduced image recognition method provided by the present invention.

[0019] Figure 2 This is the second flow chart of the reproduced image recognition method provided by the present invention.

[0020] Figure 3 It is a comparison diagram of the real image and the reproduced image provided by the present invention.

[0021] Figure 4 It is a local detail texture map of a screen-reproduced image with moiré provided by the present invention.

[0022] Figure 5It is a local detail texture map of a screen-photographed image without moiré provided by the present invention.

[0023] Figure 6 It is a spectrum diagram of a real image provided by the present invention.

[0024] Figure 7 It is a spectrum diagram of the re-shot image provided by the present invention.

[0025] Figure 8 It is a structural schematic diagram of the neural network model provided by the present invention.

[0026] Fig. 9 It is a structural schematic diagram of the reproduced image recognition device provided by the present invention.

[0027] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or position relationship indicated by the terms "upper", "lower" and the like is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be a connection between two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0030] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0031] Mobile communication wireless base stations are key infrastructure in mobile communication networks. To ensure the safety and reliability of wireless signal transmission, it is necessary to regularly maintain and inspect the equipment in the wireless base stations to understand their operating status and changes in the surrounding environment. Timely discovery of equipment defects and hidden dangers that endanger equipment safety is an indispensable part of the daily network maintenance work of communication operators. The use of artificial intelligence to carry out wireless base station acceptance and inspection has many advantages, such as improving work efficiency, reducing costs, improving accuracy, preventive maintenance, improving safety, traceability, optimizing resource allocation, and improving service quality. However, in the wireless base station acceptance and inspection based on AI image recognition technology, there will be problems of cheating by inspectors, such as inspectors taking screenshots or printing out historical images or images collected online instead of taking physical photos in the machine room. This cheating behavior may cause the intelligent inspection and acceptance system to be unable to accurately identify the equipment status in the wireless base station, thereby affecting the accuracy and reliability of the inspection. For this type of cheating problem, advanced means need to be taken to solve and optimize it to ensure the normal acceptance and inspection of wireless base stations.

[0032] The following three methods are usually used to accurately identify mobile network derogatory users: Method 1: By adding a watermark with location and time information to the generated photos when taking photos, the problem of falsification of security inspection results is eliminated to a certain extent. However, due to the instability of GPS positioning services, the positioning results will be inaccurate when deployed in basements, mountainous areas and other places with poor GPS signals. Therefore, relying on GPS positioning as a proxy for whether the inspector is present is unreliable.

[0033] Method 2: Use an edge detection algorithm to perform edge detection on the test image to obtain an edge image; use a line detection algorithm to extract lines from the edge image to obtain initial lines; use a line density algorithm to obtain copied lines from the initial lines; calculate the ratio of the number of copied lines to the number of initial lines as the line ratio value, if the line ratio value is greater than or equal to the preset line ratio value threshold, then determine that the test image is a suspected copied image. This solution is based on traditional image signal processing methods such as image edge detection and line detection, but this type of algorithm itself is affected by factors such as noise and uneven illumination, resulting in false detection or missed detection; traditional empirical threshold solutions require too much manual intervention and adjustment when facing new data or abnormal situations, and performance or efficiency may be greatly reduced; and for images of different complexity, size or color, the set empirical threshold is also not of reference value.

[0034] Method 3: By simultaneously judging whether there are screen elements and moiré patterns in the photo, it is determined whether the image is a re-photographed image. Although this solution introduces screen element detection as a supplement, it ignores the problem of missed detection due to the failure to detect screen elements due to avoidance, occlusion, cropping, etc. during the shooting or image processing process, which will cause misjudgment; and for the computer room scene, there will be other instruments and equipment with built-in screen elements within the shooting range. The use of this solution will cause the normal photos of screen devices to be misjudged as re-photographed photos, affecting the normal inspection process. In addition, not all screen-shot re-photographed images have moiré features. In some cases, the generation of moiré patterns can be effectively avoided by changing the shooting angle, adjusting the shooting distance, or changing the camera settings (such as resolution and focal length). This method cannot effectively identify screen-shot re-photographed images without moiré patterns.

[0035] In view of the above problems, the present invention proposes the following embodiments. Figure 1-Figure 10 The invention describes a method, device, electronic device and storage medium for recognizing a reproduced image.

[0036] Figure 1 FIG. 1 is one of the flow charts of the method for recognizing a reproduced image provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: S110, obtaining a clear frame image in the item to be detected, wherein the clear frame image is the clearest image in the item to be detected; S120, extracting features from the clear frame image using a gray level co-occurrence matrix method to obtain a first image feature; S130, extracting a spectrum feature of the clear frame image to obtain a second image feature; S140, fusing the first image feature and the second image feature to obtain a fused feature; S150, inputting the fusion features into a neural network model to obtain a detection result output by the neural network model.

[0037] It should be noted that the executor of the reproduced image recognition method provided in the embodiment of the present application can be a server, a computer device, such as a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc.

[0038] Here, the item to be detected is an item that needs to be detected currently, and specifically, it can be a video shot during the inspection of a wireless base station, or it can be an image set including multiple images shot during the inspection.

[0039] Preferably, the captured video or image set includes images of various focal lengths. Taking the item to be detected as a video as an example, the shooting lenses of the terminal devices currently used for wireless base station inspections basically all use zoom lenses. When the camera is called to take pictures through the APP of the terminal device (the APP side disables the gallery import function, and can only call the camera to take pictures now, to avoid cheating with historical photos or online pictures), the focal length change is controlled by an algorithm and recorded as a short video. Specifically, each frame of the image is read when recording a short video with a variable focal length. For example, if the focal length range of the camera is 2-16mm and the frame rate of the video recording is 30FPS, then the APP will automatically control the focal length from 2mm to 16mm according to the focal length range, and record one frame at each focal length at intervals of 1mm, so a short video clip with 15 frames and a duration of 0.5 seconds can be obtained.

[0040] In S140, the specific structure of the neural network model used is not limited here.

[0041] The method for recognizing copied images provided in an embodiment of the present invention obtains a clear frame image in an item to be detected, wherein the clear frame image is the clearest image in the item to be detected, that is, the clearest image in the item to be detected is obtained, so as to avoid blurry or out-of-focus images caused by improper optical focal length, ensure the clarity of the texture of the surface of the photographed object, and provide a reliable data basis for subsequent copy recognition; a gray level co-occurrence matrix method is used to perform feature extraction on the clear frame image to obtain a first image feature, and the gray level co-occurrence matrix method can effectively extract the texture feature of the image by counting the gray levels of pixels in the image and the relative position relationship between them; a spectrum map feature is performed on the clear frame image to obtain a second image feature, and the spectrum map feature of the image describes the frequency distribution of pixel values ​​in the image; the first image feature and the second image feature are fused to obtain a fused feature; the fused feature is input into a neural network model to obtain a detection result output by the neural network model. This solution extracts features from images and then performs re-photographed image recognition based on the extracted features. It does not rely on GPS positioning information and has a wide range of applications. It can still effectively perform re-photographed image recognition in a computer room with poor signals. The first image feature and the second image feature are fused, that is, the texture feature of the image is combined with the frequency distribution feature of the image, and then recognition and detection are performed through a neural network model to improve the accuracy of re-photographed recognition. Clear frame images in the items to be recognized are used for recognition to ensure the clarity of the image, provide reliable image data for re-photographed recognition, and further improve the accuracy of re-photographed recognition.

[0042] In an optional embodiment, before obtaining a clear frame image in the item to be detected, the method further includes: Calculating the Laplacian operator of each frame image in the item to be detected to obtain the Laplacian operator matrix of each frame image; For any frame image, calculating the variance of the Laplace operator corresponding to the image; The maximum variance image is used as the clear frame image, and the maximum variance image is a frame image with the largest variance.

[0043] Here, the Laplace operator is used to evaluate the clarity of the image. The Laplace operator is a second-order differential operator that can highlight the edge and detail information in the image and enhance the clarity of the image by operating the second-order derivative of the image pixels. The larger the Laplace operator, the higher the clarity of the image frame.

[0044] Still taking the item to be detected as a video as an example, assuming that the dimension of each frame image in the video is H*W, and the number of video frames is T; traverse each frame, and calculate the Laplace operator results of each frame image respectively, and the dimension H*W remains unchanged; during the traversal process, the variance of the Laplace operator results of each frame image is calculated at the same time, and a 1-dimensional variance result is obtained; find the maximum value of the Laplace operator variance, and its corresponding image is taken as the clear frame image.

[0045] The method for recognizing copied images provided in an embodiment of the present invention provides a clear frame image to be detected for copy detection, avoiding phenomena such as blurred or out-of-focus images caused by improper optical focal length, and especially ensuring that the texture of the surface of the photographed object is as clear and visible as possible, providing a reliable data basis for the subsequent extraction of texture features of the image, thereby obtaining accurate recognition results when performing detection based on the extracted image texture features.

[0046] In an optional embodiment, the neural network model is obtained based on the following steps: Acquire a real image set and a reproduced image set, wherein the real image set includes a plurality of real images, and the reproduced image set includes a plurality of reproduced images; Acquire the fusion features of each image in the real image set and the fusion features of each image in the re-photographed image set; Based on the fusion features of each image in the real image set and the fusion features of each image in the re-shot image set, the neural network model to be trained is trained to obtain the neural network model.

[0047] Specifically, the dataset consists of two parts: real images and re-photographed images. All real images constitute the real image set, and all re-photographed images constitute the re-photographed image set. Optionally, real images and re-photographed images are distinguished by setting labels, for example, the labels of all real images are set to 0, and the labels of all re-photographed images are set to 1.

[0048] Furthermore, the real image set includes a plurality of real clear frame images, and the reproduced image set includes a printed reproduced image subset and a screen-reproduced image subset; The images in the print-reproduced image subset are images obtained by reproducing the printed paper; The images in the screen-shot reproduced image subset are images obtained by reshooting images displayed on the screen, and the screen-shot reproduced image subset includes a plurality of reproduced images with moiré patterns and a plurality of reproduced images without moiré patterns.

[0049] In the specific implementation process, for the collection of real images, in the wireless base station scenario, after using the above video shooting method to shoot videos of multiple types of acceptance items (such as baseband processing units, radio frequency remote units, active antenna units, etc.), the clear frame images are obtained by using the method for determining clear frame images described in the above embodiment. It should be understood that in order to ensure the richness of image data, the number of clear frame images collected needs to reach a preset number, such as 10,000, and the collection should cover as many lighting conditions, shooting distances, image sizes, etc. as possible.

[0050] For the acquisition of the copied images, the collected real images (taking 10,000 images as an example) are copied in two ways: printing paper copying and screen copying: Printing paper reshoot: Use different types of printers and different types of paper to print a portion of the images randomly selected from them, taking 20% ​​as an example, that is, randomly select 2000 images from 10,000 real images for printing. After printing, use multiple models of mobile phones to reshoot the paper, and collect clear frame images according to the clear frame image collection process of the above embodiment; Screen capture: Use mobile phones, computer monitors, tablets, LED display screens and other screens of different types, sizes and resolutions to display the remaining 8,000 images (80%), and use multiple models of mobile phones to capture the images on the screen; it should be noted that this process needs to ensure that a certain proportion (such as 50%) of moiré-free captured images are collected (through methods such as adjusting the focal length and shooting at a wide angle) to ensure that the neural network model obtained by subsequent training can recognize the moiré-free captured images.

[0051] Furthermore, the image set is divided into a training set, a validation set, and a test set according to a certain ratio (such as 7:2:1) for training the neural network model.

[0052] The method for recognizing reproduced images provided in an embodiment of the present invention has a reproduced image set covering different types of reproduced images, different lighting scenes, and images of different complexity and sizes to ensure generalization, especially covering a moiré-free screen-reproduced data set to ensure data richness, thereby improving the versatility of the learned neural network model.

[0053] Figure 3 is a comparison diagram of the real image and the re-photographed image provided by the present invention, such as Figure 3The figure shows a comparison of a real image of an active antenna unit device and various types of re-photographed images. From left to right, the first column of images is an image of an active antenna unit device taken by a mobile phone (real image), the second column of images is an image of a real image printed on paper and re-photographed by a mobile phone (printed re-photographed image), the third column of images is an image of a real image displayed on a monitor screen (display resolution 1920*1080) and then re-photographed by a mobile phone (re-photographed image with moiré), and the fourth column of images is an image of a real image displayed on a monitor screen and then re-photographed by a mobile phone after zooming out the optical focal length (0.9x) (re-photographed image without moiré). The above four images were all taken by the same mobile phone (mobile phone camera pixel 3840x2160).

[0054] Figure 4 It is a local detail texture map of a screen-reproduced image with moiré provided by the present invention; Figure 5 It is a local detail texture map of a screen-reproduced image without moiré provided by the present invention; like Figure 3-Figure 5 As shown in the figure, although the real image and the reproduced image are basically the same in the overall content of the picture, they are very different in the local details and textures. In the real image, it can be seen that the image is the clearest, and after being magnified, the local details are normal; in the printed reproduced image, the local details are seriously lost, and after being magnified, the texture of the ink and paper can be clearly seen, and even the reflection of the paper can be seen; in the screen-reproduced image with moiré, obvious color moiré in the form of strips and blocks can be seen, such as Figure 4 As shown in FIG. 1 , by further enlarging the local details in the screen-reproduced image with moiré patterns, the detailed texture of the pixels on the screen surface hidden in the stripes of the moiré patterns can be clearly seen; the screen-reproduced image without moiré patterns cannot see the moiré patterns, as shown in FIG. Figure 5 As shown, if the local details in the moiré-free screen copy image are further enlarged, the detailed texture of the pixels on the screen surface in neatly arranged vertical stripes can be seen.

[0055] In summary, no matter which reproduction method is used, the surface detail texture of the image is obviously different from the real image. Therefore, the present invention proposes to extract the features of surface micro-texture that distinguish the real image and various reproduction images through the Gray Level Co-occurrence Matrix (GLCM) method and the spectrum diagram.

[0056] In an optional embodiment, the step of extracting features from the clear frame image using a gray level co-occurrence matrix method to obtain a first image feature includes: Performing grayscale processing on the clear frame image to obtain a grayscale image; Based on a preset sliding window size and a preset sliding window direction, calculating a gray level co-occurrence matrix of the gray level image, wherein the number of the gray level co-occurrence matrices is the same as the number of the preset sliding window directions; Merging the gray-level co-occurrence matrices to obtain a total gray-level co-occurrence matrix of the clear frame image; The total gray level co-occurrence matrix is ​​normalized to obtain the first image feature.

[0057] Gray-level co-occurrence matrix (GLCM) is an effective texture feature analysis method that extracts texture features by counting the gray levels of pixels in an image and their relative positional relationship. Specifically, for an image, two pixels separated by a certain distance in a certain direction have the same gray value, and this relationship can be used to describe the texture information in the image.

[0058] In the specific implementation process, the RGB clear frame image is converted into a grayscale image; grayscale quantization, that is, the grayscale level 256 is quantized into a specified level, and quantized into 16 levels in the embodiment of the present invention; parameter selection, for the scene, it is necessary to extract the fine textures such as paper and screen in the reprinted image, therefore, the sliding window size and step size during GLCM feature calculation should be as small as possible, and the preferred parameters are shown in Table 1 below, where in order to detect all the image surface micro-textures as much as possible, there are 4 sliding window directions, namely 0°, 45°, 90° and 135°.

[0059] Table 1 Gray-level co-occurrence matrix parameter table

[0060] According to the parameter settings in Table 1, the gray level co-occurrence matrix is ​​calculated in four directions in turn, and the calculation results of the gray level co-occurrence matrix in the four directions are merged to obtain the total gray level co-occurrence matrix, whose dimension is H*W*4; the extracted total gray level co-occurrence matrix features are normalized to improve the quality and stability of the features.

[0061] In an optional embodiment, extracting a spectrum feature from the clear frame image to obtain a second image feature includes: Performing grayscale processing on the clear frame image to obtain a grayscale image; After Fourier transform processing is performed on the grayscale image, a center shift is performed on the frequency of the Fourier transform result to move the low-frequency part to the center position to obtain a shifted spectrum diagram; After calculating the modulus of the offset spectrum graph, taking the logarithm of the modulus spectrum graph to obtain a logarithmic spectrum graph; A normalization operation is performed on the logarithmic frequency spectrum to obtain the second image feature.

[0062] The spectrogram feature of an image is an important attribute of the image, which describes the frequency distribution of pixel values ​​in the image. In the spectrogram, the central part represents the low-frequency part of the original image, and the surrounding part represents the high-frequency part of the original image.

[0063] Figure 6 is a spectrum diagram of a real image provided by the present invention; Figure 7 is a spectrum diagram of the re-shot image provided by the present invention; like Figure 6-Figure 7 As shown in the figure, the high-frequency characteristics of the re-photographed image are obvious due to the existence of periodic striped micro-textures on the surface, so there are more spots around the spectrum graph, while the spectrum graph of the real image does not have such spots, and the difference is obvious. Therefore, the spectrum graph feature can also be used as one of the feature bases for re-photographed image detection.

[0064] In the specific implementation process, the RGB clear frame image is converted into a grayscale image and then Fourier transformed; the frequency of the Fourier transform result is center-shifted so that the low-frequency part of the spectrum is located at the center of the array to facilitate subsequent processing and analysis; the modulus of the spectrum after the center shift is calculated, and then 1 is added (to avoid taking the logarithm of 0), and then the logarithm is taken to convert it into a logarithmic spectrum. This operation is usually to introduce a logarithmic scale on the spectrum so that the high-frequency part and the low-frequency part have a closer dynamic range, thereby better displaying and visualizing the spectrum; the extracted spectrum features are normalized to improve the quality and stability of the features.

[0065] In an optional embodiment, the neural network model adopts a convolutional neural network.

[0066] Figure 8 It is a structural schematic diagram of the neural network model provided by the present invention, such as Figure 8As shown in the figure, the convolution layer 2D-conv (2-dimension convolution layer) has three layers, and the convolution kernel of each convolution layer is 5*5. After each layer of 2D-conv, there are activation layer ReLU, normalization layer BatchNorm and downsampling layer maxPooling in order. ReLU is used to increase the nonlinear ability of the model; BatchNorm is used to adjust the data distribution to avoid overfitting; and maxpooling is used to reduce the feature dimension and reduce the amount of calculation at the same time. After three layers of convolution layers, there are three layers of full connection layers to integrate and classify the features extracted by the convolution layer. The full connection layers are also connected by ReLU, which also increases the nonlinear ability of the model. Finally, the classification result is output at the output end through the softmax function and the classification layer classification layer. In addition, the input dimension of the first layer of convolution is 960*960, which is consistent with the size of the preprocessed image; the number of nodes in the last layer of classification layer is 2, representing two types of classification results 0 and 1. 0 represents the real image and 1 represents the re-photographed image.

[0067] The preferred method for identifying a reproduced image according to the present invention is described below.

[0068] Figure 2 FIG. 2 is a flow chart of the second method for identifying a reproduced image provided by the present invention. Figure 2 As shown in the figure, after zooming in and out of the video, the clear frame images in the video are collected to obtain a real image set, and a re-shot image set is generated based on the real image set; the image size is uniformly adjusted to a preset size, which is 960*960 here; the image set is divided into a training set, a test set, and a validation set, and features are extracted from the images through GLCM and a spectrum diagram, and then the extracted features are used to train the convolutional neural network to obtain a neural network model for detecting re-shot images.

[0069] After the neural network model is trained and evaluated, the application is deployed in the local environment of the terminal device and can be used in the production environment. In actual use, photos that pass the recognition are uploaded as real inspection photos and processed later; photos that fail the recognition are rejected as re-photographed photos and the cheating behavior is recorded. In addition, the images collected by the inspection personnel can be recycled and manually annotated, and the data can be expanded and the model updated to complete the version iteration and improve the model recognition effect.

[0070] In summary, the method for recognizing copied images provided by the present invention introduces a clear frame extraction algorithm based on the Laplace operator to obtain the clearest frame of the item to be detected, so that the surface texture is as clear and visible as possible, which is convenient for the subsequent extraction of texture features and the recognition of the copy model; the copy image data set covers different types of copies, different lighting scenes, images of different complexity and size to ensure generalization, especially covers the data set of screen copy without moiré; a deep learning solution is used for copy detection to avoid judgment through empirical thresholds; GLCM and spectrum diagrams are used to extract micro-texture features of the screen or paper surface as the main features, and it can solve the accurate recognition of various types of copies such as screen copy without moiré, screen copy with moiré, and print copy.

[0071] The reproduced image recognition device provided by the present invention is described below. The reproduced image recognition device described below and the reproduced image recognition method described above can be referred to each other.

[0072] Fig. 9 is a schematic diagram of the structure of the reproduced image recognition device provided by the present invention, such as Fig. 9 As shown, the re-shot image recognition device may include but is not limited to; The image acquisition module 910 is used to: acquire a clear frame image in the item to be detected, wherein the clear frame image is the clearest image in the item to be detected; A first feature extraction module 920 is used to: extract features from the clear frame image using a gray level co-occurrence matrix method to obtain a first image feature; The second feature extraction module 930 is used to: extract the spectrum feature of the clear frame image to obtain the second image feature; A feature fusion module 940 is used to fuse the first image feature and the second image feature to obtain a fused feature; The model prediction module 950 is used to: input the fusion feature into the neural network model to obtain the detection result output by the neural network model.

[0073] It should be noted that the reproduced image recognition device provided in the embodiment of the present invention can execute the reproduced image recognition method described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0074] Fig.10 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.10As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020 and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the re-photographed image recognition method, which includes: obtaining a clear frame image in the item to be detected, wherein the clear frame image is the clearest image in the item to be detected; extracting features from the clear frame image using a gray level co-occurrence matrix method to obtain a first image feature; extracting features from a spectrum diagram of the clear frame image to obtain a second image feature; fusing the first image feature and the second image feature to obtain a fused feature; inputting the fused feature into a neural network model to obtain a detection result output by the neural network model.

[0075] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0076] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the reproduced image recognition method provided by the above-mentioned methods, which includes: obtaining a clear frame image in the item to be detected, and the clear frame image is the clearest image in the item to be detected; using a grayscale co-occurrence matrix method to extract features of the clear frame image to obtain a first image feature; performing a spectrum graph feature extraction on the clear frame image to obtain a second image feature; fusing the first image feature and the second image feature to obtain a fused feature; inputting the fused feature into a neural network model to obtain a detection result output by the neural network model.

[0077] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the reproduced image recognition method provided by the above-mentioned methods, the method comprising: obtaining a clear frame image in the item to be detected, the clear frame image being the clearest image in the item to be detected; performing feature extraction on the clear frame image using a grayscale co-occurrence matrix method to obtain a first image feature; performing spectrum graph feature extraction on the clear frame image to obtain a second image feature; fusing the first image feature and the second image feature to obtain a fused feature; inputting the fused feature into a neural network model to obtain a detection result output by the neural network model.

[0078] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying a re-photographed image, characterized in that: include: Acquire a clear frame image in the item to be detected, wherein the clear frame image is the clearest image in the item to be detected; Using a gray level co-occurrence matrix method to extract features from the clear frame image, to obtain a first image feature; Extracting the features of the spectrum graph of the clear frame image to obtain a second image feature; fusing the first image feature and the second image feature to obtain a fused feature; The fusion features are input into a neural network model to obtain a detection result output by the neural network model.

2. The method for recognizing a reproduced image according to claim 1, characterized in that: Before obtaining the clear frame image in the item to be detected, the method further includes: Calculating the Laplacian operator of each frame image in the item to be detected to obtain the Laplacian operator matrix of each frame image; For any frame image, calculating the variance of the Laplace operator corresponding to the image; The maximum variance image is used as the clear frame image, and the maximum variance image is a frame image with the largest variance.

3. The method for recognizing a reproduced image according to claim 1, characterized in that: The step of extracting features from the clear frame image using a gray level co-occurrence matrix method to obtain a first image feature includes: Performing grayscale processing on the clear frame image to obtain a grayscale image; Based on a preset sliding window size and a preset sliding window direction, calculating a gray level co-occurrence matrix of the gray level image, wherein the number of the gray level co-occurrence matrices is the same as the number of the preset sliding window directions; Merging the gray-level co-occurrence matrices to obtain a total gray-level co-occurrence matrix of the clear frame image; The total gray level co-occurrence matrix is ​​normalized to obtain the first image feature.

4. The method for recognizing a reproduced image according to claim 1, characterized in that: The extracting the spectrum feature of the clear frame image to obtain the second image feature includes: Performing grayscale processing on the clear frame image to obtain a grayscale image; After Fourier transform processing is performed on the grayscale image, a center shift is performed on the frequency of the Fourier transform result to move the low-frequency part to the center position to obtain a shifted spectrum diagram; After calculating the modulus of the offset spectrum graph, taking the logarithm of the modulus spectrum graph to obtain a logarithmic spectrum graph; A normalization operation is performed on the logarithmic frequency spectrum to obtain the second image feature.

5. The method for recognizing a reproduced image according to claim 1, characterized in that: The neural network model is obtained based on the following steps: Acquire a real image set and a reproduced image set, wherein the real image set includes a plurality of real images, and the reproduced image set includes a plurality of reproduced images; Acquire the fusion features of each image in the real image set and the fusion features of each image in the re-photographed image set; Based on the fusion features of each image in the real image set and the fusion features of each image in the re-shot image set, the neural network model to be trained is trained to obtain the neural network model.

6. The method for recognizing a reproduced image according to claim 5, characterized in that: The real image set includes a plurality of real clear frame images, and the reproduced image set includes a printed reproduced image subset and a screen-reproduced image subset; The images in the print-reproduced image subset are images obtained by reproducing the printed paper; The images in the screen-shot reproduced image subset are images obtained by reshooting images displayed on the screen, and the screen-shot reproduced image subset includes a plurality of reproduced images with moiré patterns and a plurality of reproduced images without moiré patterns.

7. A device for recognizing a re-photographed image, characterized in that: include: An image acquisition module is used to: acquire a clear frame image in the item to be detected, wherein the clear frame image is the clearest image in the item to be detected; A first feature extraction module is used to: extract features from the clear frame image using a gray level co-occurrence matrix method to obtain a first image feature; A second feature extraction module is used to: extract the spectrum feature of the clear frame image to obtain a second image feature; A feature fusion module, used to: fuse the first image feature and the second image feature to obtain a fused feature; The model prediction module is used to: input the fusion features into the neural network model to obtain the detection results output by the neural network model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the reproduced image recognition method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the reproduced image recognition method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the reproduced image recognition method according to any one of claims 1 to 6 is implemented.

Images