A method and system for realizing mobile phone screenshot determination based on image metadata

Abstract

The present invention discloses a method for determining mobile phone screenshots based on image metadata. Another technical solution of the present invention is to provide a system for determining mobile phone screenshots based on image metadata, which implements the above method and is characterized by comprising an image analysis module, a verification model module, and an output module. The method of the present invention for determining mobile phone screenshots based on image metadata can determine whether a target image is a mobile phone screenshot. The present invention is based on image metadata, and metadata is simple to obtain and used as a basis for determination, thereby reducing resource consumption of mobile phone screenshot recognition and improving the efficiency of mobile phone screenshot recognition. The determination rules in the present invention can be dynamically fine-tuned according to specific scenarios, and are highly flexible.

Description

Technical Field

[0001] The present invention relates to a method for realizing mobile phone screenshot determination based on picture metadata and a system realized based on the method, belonging to the field of Internet technology. Background Art

[0002] In the era of mobile Internet, mobile phone pictures are generated and circulated very frequently. It is very important to mine and use the data of mobile phone pictures. Especially in business scenarios that require users to upload real data, how to reasonably use the data of the picture itself to judge mobile phone screenshots has become an important part of image data mining.

[0003] However, the current mobile phone screenshot determination method has the following shortcomings:

[0004] (1) Image metadata, as the “identity card” of an image, contains rich image identity information, but it has not been properly utilized;

[0005] (2) There is no detailed plan or idea on how to more effectively implement mobile phone screenshot identification based on metadata;

[0006] (3) When the image metadata format is fixed, how to flexibly improve the success rate of mobile phone screenshot identification is also an important issue. Summary of the Invention

[0007] The purpose of the present invention is to enable mobile phone screenshot recognition to be completed in a simpler, more efficient, safer and more flexible manner.

[0008] In order to achieve the above object, the technical solution of the present invention is to provide a method for determining mobile phone screenshots based on image metadata, which is characterized by comprising the following steps:

[0009] Step 1: Obtain the target image uploaded by the user, where the target image is the original image that has not been compressed or modified;

[0010] Step 2: Get the metadata information of the target image;

[0011] Step 3: Determine whether the target image is the original image and the metadata information obtained. If so, proceed to step 4; otherwise, exit the method.

[0012] Step 4: Input the metadata information into a verification model built based on the image metadata. The verification model is formed by combining multi-dimensional verification rules based on the image metadata information. Only when the metadata information meets the verification rules of all dimensions will the verification model output a verification pass message; otherwise, the verification model outputs a verification fail message.

[0013] Step 5: If the verification model outputs verification pass information, it is determined that the target image uploaded by the user is a mobile phone screenshot; otherwise, it is determined that the target image uploaded by the user is not a valid mobile phone screenshot.

[0014] Preferably, the metadata information includes EXIF ​​standard information specified by the image, image pixel information, and data information customized by the manufacturer of the shooting mobile phone.

[0015] Preferably, verification rules corresponding to each dimension are set based on different categories of the image metadata information, and the verification rules of all dimensions are combined to form the verification model.

[0016] Preferably, the multi-dimensional verification rules include size verification rules, key character verification rules, general identification pixel distribution verification rules, specific identification pixel distribution verification rules and device information verification rules, wherein:

[0017] If the width and height of the target image in the metadata information obtained in step 2 belong to the preset image size finite width dataset and image size finite height dataset, then the size verification rule is met; otherwise, the size verification rule is not met;

[0018] If the metadata information obtained in step 2 does not contain lens related information, aperture related information, flash related information, shutter related information, GPS related information, and / or thumbnail related information, then the key character verification rule is met; otherwise, the key character verification rule is not met;

[0019] Whether the general identification pixel distribution verification rule or the specific identification pixel distribution verification rule is met is determined by the following steps:

[0020] Step 401: Capture the area of ​​the target image that contains the general identifier or the specific identifier to obtain a screenshot of the area;

[0021] Step 402: Calculate the mean R and variance of the red primary color of the screenshot. Blue primary color mean B, blue primary color variance Green primary color mean G and green primary color variance

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028] Where: w and h are the width and height of the screenshot respectively; r i 、b i 、g i are the red, blue, and green primary color values ​​of the i-th pixel in the screenshot respectively;

[0029] Step 403: If the calculated mean red primary color R, mean blue primary color B and mean green primary color G of the screenshot are within the preset ranges with the preset mean red primary color, mean blue primary color and mean green primary color, respectively, and the calculated variance value of the red primary color of the screenshot is Blue primary color variance And the green primary color variance value If the error with the preset red primary color variance value, blue primary color variance value, and green primary color variance value is within the preset range, then it complies with the general identification pixel distribution verification rule or the specific identification pixel distribution verification rule; otherwise, it does not comply;

[0030] If the device information in the metadata information obtained in step 2 belongs to the finite data set of the manufacturer of the operating system installed on the mobile phone, then the device information verification rule is met; otherwise, it is not met.

[0031] Another technical solution of the present invention is to provide a system for determining mobile phone screenshots based on image metadata, which implements the above method and is characterized by comprising an image analysis module, a verification model module, and an output module, wherein:

[0032] Image analysis module: used to obtain the target image uploaded by the user, analyze the target image to obtain the corresponding metadata information, and determine whether the target image is the original image and whether the metadata information is complete;

[0033] Verification model module: used to verify the original image using metadata information, and output verification pass information or verification fail information based on the verification result;

[0034] Output module: used to output the final result of whether the target image is a screenshot taken by a mobile phone.

[0035] The present invention implements a method for mobile phone screenshot identification based on image metadata, capable of determining whether a target image is a valid mobile phone screenshot. This method is based on image metadata, which is easily accessible and used as a basis for identification, reducing resource consumption and improving the efficiency of screenshot identification. The identification rules in this invention can be dynamically fine-tuned based on specific scenarios, offering high flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a method for determining mobile phone screenshots based on image metadata provided in one or more embodiments of this specification;

[0037] Figure 2 A schematic diagram illustrating a specific implementation principle for establishing verification rules for a method for determining mobile phone screenshots based on image metadata provided in one or more embodiments of this specification;

[0038] Figure 3 A schematic diagram of the module composition of a system for implementing mobile phone screenshot determination based on image metadata provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0039] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the appended claims of the application.

[0040] One aspect of the present invention is to disclose a method for determining mobile phone screenshots based on image metadata. Figure 1 As shown, the specific steps include:

[0041] Step 1: Get the original image uploaded by the user. The original image must be an uncompressed and unmodified original image.

[0042] Step 2: Obtain metadata for the original image to be verified. This metadata includes, but is not limited to, EXIF ​​standard information required by the image and customized data from the camera manufacturer. In a Java development environment, use the generic method provided by the metadata-extractor plugin to obtain metadata for the original mobile screenshot to be verified.

[0043] Example of using metadata-extractor to obtain image metadata information:

[0044] / / Read the sample file, i.e. the original mobile phone screenshot

[0045] File file=new File("example.JPG")

[0046] / / Read the image metadata information through metadata-extractor

[0047] Metadata metadata=ImageMetadataReader.readMetadata(file)

[0048] Step 3: Classify the parsed metadata information into image size information, key character information, pixel distribution information, and device information.

[0049] Step 4: Input the classified metadata information into the verification model, which is formed by combining multiple verification rules. In this embodiment, based on the different categories of metadata information, the verification rules can be divided into the following aspects:

[0050] 1) Dimension verification rules:

[0051] As we know, the size of mobile phone screenshots generally matches the size of the mobile phone screen, so there is a finite width dataset W and a finite height dataset H for the image size.

[0052] The width w of the mobile phone screenshot should belong to W, that is, w∈W;

[0053] The height h of the mobile phone screenshot should belong to H, that is, h∈H.

[0054] 2) Key character verification rules:

[0055] After analyzing the metadata of different images from multiple manufacturers, we found that: unlike images taken with mobile phone cameras, the metadata of mobile phone screenshots does not contain relevant information about the lens: such as the shooting scene and the lens focal length;

[0056] The metadata of mobile screenshots does not contain information about aperture: such as aperture size and aperture coefficient;

[0057] The metadata of mobile screenshots does not contain flash-related information: such as flash status;

[0058] The metadata of mobile screenshots does not contain shutter-related information, such as shutter speed;

[0059] The metadata of mobile screenshots does not contain GPS-related information: such as longitude, latitude, GPS time, altitude, etc.

[0060] The metadata of mobile screenshots does not contain relevant information about thumbnails: such as compression type, thumbnail offset, thumbnail size, and thumbnail dimensions.

[0061] 3) General logo pixel distribution verification rules:

[0062] As we know, mobile phone screenshots have some similarities. For example, there will be information such as battery level, network signal, and operator at the top. We intercept the area of ​​the original image P where similar universal symbols are located, obtain the screenshot p of this area, and calculate the mean of the three primary colors of the pixels in the screenshot p:

[0063] If the width of the screenshot p is w and the height is h, then the screenshot p has a total of w*h pixels. Let the red primary color value of the i-th pixel be r i , the blue primary color value is b i , the green primary color value is g i , then we can get:

[0064] The mean value of the red primary color R:

[0065]

[0066] Red primary color variance

[0067]

[0068] Blue primary color mean B:

[0069]

[0070] Blue primary color variance

[0071]

[0072] Green primary color mean G:

[0073]

[0074] Green primary color variance

[0075]

[0076] In summary, we can take a screenshot p′ of the area where the universal logo is located and calculate the mean and variance of the three primary colors in advance, and then we can get:

[0077] The mean of the screenshot p containing the general mark obtained from the original image to be verified and the regional pre-screenshot p ′ The mean error is within a certain range;

[0078] The mean of the screenshot p containing the general mark obtained from the original image to be verified and the regional pre-screenshot p ′ The variance error is within a certain range.

[0079] 4) Specific identification pixel distribution verification rules:

[0080] In addition to general identifiers such as battery level, network signal, and operator, mobile phone screenshots also have some specific identifiers in the screenshot scene, such as QR codes and barcodes. The range of the specific identifier (such as QR code, barcode, etc.) of the original image P is intercepted to obtain the screenshot p1 of the range, and the mean of the three primary colors of the pixels in the range is calculated, that is, the mean value R1 of the red primary color and the variance value R1 of the red primary color of the screenshot p1 are obtained. Blue primary color mean B1, blue primary color variance Green primary color mean G1 and green primary color variance The calculation method is the same as above and will not be repeated here.

[0081] In summary, we can pre-screenshot p1 for the area where the general logo is located ′ , calculate the mean and variance of the three primary colors in advance, and you can get:

[0082] The mean of the screenshot p1 containing the specific mark obtained from the original image to be verified and the regional pre-screenshot p1 ′ The mean error is within a certain range;

[0083] The mean of the screenshot p1 containing the specific mark obtained from the original image to be verified and the regional pre-screenshot p1 ′ The variance error is within a certain range.

[0084] 5) Device information verification rules:

[0085] By analyzing the metadata of multiple mobile phone manufacturers and different images, we can find that if the metadata of mobile phone screenshots contains device information, the device information is generally consistent with the manufacturer of the operating system installed on the mobile phone. It is known that there is a finite data set S of mobile phone manufacturers and operating system manufacturers such that:

[0086] The device information s filtered out from the mobile phone screenshot belongs to the finite data set S, that is, s∈S.

[0087] Step 5: The verification model summarizes the verification results of the verification rules and outputs the verification model results. If all rules pass the verification, the output verification model result is passed; if there is a rule verification failure, the output verification model result is failed.

[0088] Step 6: Determine whether the original image uploaded by the user is a mobile phone screenshot based on the verification model results. If it passes, output that it is a mobile phone screenshot. If it fails, output that it is not a mobile phone screenshot.

[0089] Another aspect of the present invention is to disclose a system for determining mobile phone screenshots based on image metadata. Figure 2 As shown, it includes an image analysis module, a verification model module and an output module.

[0090] Image Analysis Module: When a user uploads an original image with complete metadata, this module analyzes the original image and retrieves its metadata. This metadata includes, but is not limited to, EXIF ​​standard information, image pixel information, and manufacturer-defined data.

[0091] Verification Model Module: Based on a predefined metadata verification model, this module performs multi-dimensional verification on the metadata information obtained by the image analysis module. The verification rules are divided into five parts: size verification rules, key character verification rules, general identifier pixel distribution verification rules, specific identifier pixel distribution verification rules, and device information verification rules. If the verification model passes, the output is considered a passed verification; if not, the output is considered a failed verification.

[0092] Output module: connected to the verification model module in a communicable manner, outputting the result of the screenshot determination according to the output result of the verification model module.

Claims

1. A method for determining mobile phone screenshots based on image metadata, characterized in that: The following steps are involved: Step 1: Obtain the target image uploaded by the user, where the target image is the original image that has not been compressed or modified; Step 2: Get the metadata information of the target image; Step 3: Determine whether the target image is the original image and the metadata information obtained. If so, proceed to step 4; otherwise, exit the method. Step 4: Input the metadata information into a verification model built based on the image metadata. The verification model is formed by combining multi-dimensional verification rules based on the image metadata information. Only when the metadata information meets the verification rules of all dimensions will the verification model output a verification pass message; otherwise, the verification model outputs a verification fail message. Step 5: If the verification model outputs verification pass information, it is determined that the target image uploaded by the user is a mobile phone screenshot; otherwise, it is determined that the target image uploaded by the user is not a mobile phone screenshot; The multi-dimensional verification rules include size verification rules, key character verification rules, general identification pixel distribution verification rules, specific identification pixel distribution verification rules and device information verification rules, wherein: If the width and height of the target image in the metadata information obtained in step 2 belong to the preset image size finite width dataset and image size finite height dataset, then the size verification rule is met; otherwise, the size verification rule is not met; If the metadata information obtained in step 2 does not contain lens related information, aperture related information, flash related information, shutter related information, GPS related information, and / or thumbnail related information, then the key character verification rule is met; otherwise, the key character verification rule is not met; Whether the general identification pixel distribution verification rule or the specific identification pixel distribution verification rule is met is determined by the following steps: Step 401: Capture the area of ​​the target image that contains the general identifier or the specific identifier to obtain a screenshot of the area; Step 402: Calculate the mean R and variance of the red primary color of the screenshot. Blue primary color mean B, blue primary color variance Green primary color mean G and green primary color variance Where: w and h are the width and height of the screenshot respectively; r i 、b i 、g i are the red, blue, and green primary color values ​​of the i-th pixel in the screenshot respectively; Step 403: If the calculated mean red primary color R, mean blue primary color B and mean green primary color G of the screenshot are within the preset ranges with the preset mean red primary color, mean blue primary color and mean green primary color, respectively, and the calculated variance value of the red primary color of the screenshot is Blue primary color variance And the green primary color variance value If the error with the preset red primary color variance value, blue primary color variance value, and green primary color variance value is within the preset range, then it complies with the general identification pixel distribution verification rule or the specific identification pixel distribution verification rule; otherwise, it does not comply; If the device information in the metadata information obtained in step 2 belongs to the finite data set of the manufacturer of the operating system installed on the mobile phone, then the device information verification rule is met; otherwise, it is not met.

2. The method for determining mobile phone screenshots based on image metadata according to claim 1, wherein: The metadata information includes EXIF ​​standard information specified by the image, image pixel information, and data information customized by the shooting mobile phone manufacturer.

3. The method for realizing mobile phone screenshot determination based on image metadata according to claim 1, characterized in that: Based on different categories of the image metadata information, verification rules corresponding to each dimension are set respectively, and the verification rules of all dimensions are combined to form the verification model.

4. A system for determining mobile phone screenshots based on image metadata, implementing the method according to claim 1, characterized in that: It includes image analysis module, verification model module and output module, among which: Image analysis module: used to obtain the target image uploaded by the user, analyze the target image to obtain the corresponding metadata information, and determine whether the target image is the original image and whether the metadata information is complete; Verification model module: used to verify the original image using metadata information, and output verification pass information or verification fail information based on the verification result; Output module: used to output the final result of whether the target image is a screenshot taken by a mobile phone.

Images