Online photo manipulation detection method and system based on multispectral imaging feature inversion

Abstract

The present application relates to an online photo forgery detection method and system based on multispectral imaging feature inversion, belonging to the technical field of computer vision and image processing; for the identity card, property certificate, business license and other certificate photos submitted by the customers of the power business hall online, the problem that the tampering behaviors such as forgery, alteration, modification and the like cannot be effectively identified, this paper proposes an online photo forgery detection method and system based on multispectral imaging feature inversion, by establishing a standard certificate multispectral feature library, the ordinary RGB photo uploaded by the customer is subjected to spectral inversion, feature reconstruction and authenticity discrimination, realizing non-contact, automatic and high-precision authenticity verification of the online certificate photo, which can be directly embedded in the online power service system for use, and the safety and intelligent level of the power service are improved.

Description

Technical Field

[0001] This invention relates to an online photo forgery detection method and system based on multispectral imaging feature inversion, belonging to the field of computer vision and image processing technology. Background Technology

[0002] In remote services such as power line installation, transfer, and cancellation at the power supply business hall, customers generally complete identity and property rights verification by taking photos with their mobile phones and uploading photos of documents such as ID cards, property ownership certificates, and business licenses.

[0003] However, such photos are easily photoshopped, reproduced, altered, and forged, leading to risks such as fraudulent electricity registration, illegal transfer of ownership, and electricity theft. Traditional online detection relies solely on visible light textures, watermarks, and outlines, failing to identify deeper features such as ink, paper, and anti-counterfeiting zones, resulting in insufficient reliability. Summary of the Invention

[0004] Based on the problems described in the background, the present invention aims to solve the following problem: It provides an online photo forgery detection method and system based on multispectral imaging feature inversion. By establishing a standard document multispectral feature library, it performs spectral inversion, feature reconstruction, and authenticity determination on ordinary RGB photos uploaded by customers, achieving non-contact, automated, and high-precision online document photo authenticity verification. This method can be directly embedded into online power service systems, thereby solving the problem mentioned above where photos of ID cards, property certificates, business licenses, and other documents submitted online by customers at power supply business halls cannot be effectively identified as forged, altered, or tampered with.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for detecting online photo forgery based on multispectral imaging feature inversion, comprising the following steps:

[0006] S1. Obtain the RGB document photo to be detected;

[0007] S2. Preprocess the RGB ID photo;

[0008] S3. Input the preprocessed RGB ID photo into the multispectral feature inversion model to generate pseudo-multispectral features in six bands;

[0009] S4. Compare and match the generated pseudo-multispectral features of the six bands with the standard document multispectral library.

[0010] S5. Calculate the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral anomaly (LSA) after comparison and matching.

[0011] S6. Calculate the overall authenticity score based on the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral anomalies (LSA);

[0012] S7. Based on the comparison result of the comprehensive authenticity judgment score and the preset threshold, output the authenticity, alteration or tampering judgment result of the RGB ID photo.

[0013] S8. Upload the results to the business hall system to complete automatic verification.

[0014] Preferably, the preprocessing in step S2 includes distortion correction, dehazing, enhancement, and RGB ID photo region segmentation.

[0015] Preferably, the six bands in step S3 include the 365nm ultraviolet band, the 450nm blue light band, the 550nm green light band, the 650nm red light band, the 850nm near-infrared band, and the 940nm near-infrared band.

[0016] Preferably, the multispectral feature inversion model in step S3 is a neural network model, and its loss function is defined as:

[0017]

[0018] The reflectance obtained by inversion, The value represents the true multispectral reflectance obtained from professional multispectral acquisition equipment. is the regularization coefficient, and n is the number of bands used, where n=6;

[0019] At the same time, define the band Spectral reflectance at:

[0020]

[0021] in, For target documents in the band Radiance For standard whiteboard in band Radiance.

[0022] Preferably, the formula for calculating the Paper Spectral Difference Index (PSDI) in step S5 is as follows:

[0023]

[0024] in, For the RGB ID photo to be tested Spectral reflectance of the band This represents the spectral reflectance of a standard, authentic document at the corresponding wavelength. The number of bands involved in the calculation.

[0025] Preferably, the formula for calculating the ink spectral matching degree (ISMD) in step S5 is as follows:

[0026]

[0027] in, The spectral reflectance of the RGB ID photo to be tested. This refers to the spectral reflectance of a standard, authentic document.

[0028] Preferably, the definition formula for Local Spectral Anomalies (LSA) in step S5 is as follows:

[0029]

[0030] in, Represents pixels. The inversion spectral reflectance of the current pixel. This represents the average spectral reflectance of the current pixel's neighborhood (3×3 pixel neighborhood). This is the abnormal threshold, with a value range of 0.05 to 0.1, which can be fine-tuned according to the document type.

[0031] Preferably, the comprehensive authenticity discrimination score in step S6 is obtained by weighted fusion of the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral anomaly (LSA):

[0032]

[0033] in, + + = 1, , , This is a weighting coefficient, which can be fine-tuned according to the actual document type. The default value is: =0.3, =0.5, =0.2, This represents the proportion of abnormal pixels to the total number of pixels in the document area.

[0034] This invention also provides an online photo forgery detection system based on multispectral imaging feature inversion, comprising:

[0035] The image acquisition module is used to acquire RGB photos of the document to be inspected.

[0036] The image preprocessing module is used to perform distortion correction, dehazing, enhancement, and RGB ID photo region segmentation on RGB ID photos.

[0037] The spectral inversion module has a built-in neural network model for multispectral feature inversion, which is used to convert RGB ID photos into multi-band pseudo-multispectral reflectance feature maps.

[0038] The authenticity detection module is used to calculate the spectral differences of paper, the matching degree of ink spectrum, and local spectral anomalies, and output a weighted judgment score.

[0039] The result output module is used to feed the judgment results back to the business system.

[0040] The beneficial effects of this invention are:

[0041] 1. This invention utilizes a multispectral feature inversion model to directly recover the spectral features of invisible bands such as ultraviolet and infrared from a single RGB photo taken by a regular mobile phone. It overcomes the limitations of existing technologies that require dedicated multispectral cameras or document scanners for on-site scanning, significantly reducing the procurement and maintenance costs of offline hardware deployment. It is particularly suitable for contactless business scenarios such as power supply service halls where customers need to submit materials remotely online.

[0042] 2. This invention distinguishes genuine documents from counterfeit documents made of ordinary printed paper / inkjet paper at the physical material level by inverting the fixed absorption peak of the paper substrate in the near-infrared band and the infrared reflection characteristics of the anti-counterfeiting ink. Even if the counterfeit is highly similar to the original under naked-eye observation, this invention can still accurately intercept it through the spectral difference index (PSDI) and ink matching degree (ISMD).

[0043] 3. To address common issues with altering names, numbers, and dates on ID photos, or using Photoshop to stitch them together, this invention introduces a local spectral outlier (LSA) detection mechanism based on neighborhood comparison. Because the altered area exhibits abrupt changes in its infrared / ultraviolet spectral characteristics compared to the original anti-counterfeiting area after inversion, this invention can automatically locate and mark these anomalous blocks, solving the problem of existing technologies' difficulty in detecting localized minor tampering.

[0044] 4. This invention can directly identify forgeries, alterations, and tamperings in ordinary ID photos submitted online by customers, without the need for a dedicated multispectral camera or on-site scanning, achieving contactless, automated, fast, and high-precision ID verification. This method can be seamlessly integrated into the power supply line's electricity application system to eliminate fraudulent materials at the source, improving the security and intelligence of power services. Attached Figure Description

[0045] Figure 1 This is a flowchart of the method steps of the present invention. Detailed Implementation

[0046] The embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0047] Example 1

[0048] like Figure 1 As shown, this invention provides an online photo forgery detection method based on multispectral imaging feature inversion, comprising the following steps:

[0049] S1. Obtain the RGB document photo to be detected;

[0050] Typically, the customer takes and uploads an RGB photo of the ID document using their mobile phone.

[0051] S2. Preprocess the RGB ID photo;

[0052] The preprocessing includes distortion correction, dehazing, enhancement, and RGB ID photo region segmentation.

[0053] S3. Input the preprocessed RGB ID photo into the multispectral feature inversion model to generate pseudo-multispectral features in six bands;

[0054] Genuine documents use special paper, anti-counterfeiting ink, fluorescent fibers, laser engraving, and other materials, exhibiting fixed reflectivity, absorptivity, and emissivity in the ultraviolet (UV), blue (B), and near-infrared (NIR) bands. In contrast, counterfeit / altered documents use ordinary paper, printer ink, and PS modifications, resulting in spectral characteristics that differ significantly from genuine documents.

[0055] Define band Spectral reflectance at:

[0056]

[0057] in, For target documents in the band Radiance For standard whiteboard in band The definition of radiance eliminates interference from the lighting environment and ensures the uniformity of spectral data.

[0058] Therefore, multispectral imaging technology utilizes the differences in reflection, absorption, and fluorescence properties of different materials at different wavelengths to accurately distinguish between genuine and counterfeit documents. Customers typically provide standard RGB three-channel images online, lacking crucial ultraviolet and near-infrared spectral data for anti-counterfeiting detection. A lightweight spectral inversion neural network is constructed to establish a mapping relationship between RGB visual features and multispectral physical features, achieving a dimensional upgrade: RGB image → Rpre( 1, 2... 6).

[0059] The six bands include the 365nm ultraviolet band, the 450nm blue light band, the 550nm green light band, the 650nm red light band, the 850nm near-infrared band, and the 940nm near-infrared band.

[0060] The multispectral feature inversion model is a neural network model, and its loss function is defined as:

[0061]

[0062] The reflectance obtained by inversion, The value represents the true multispectral reflectance (obtained by professional multispectral acquisition equipment). is the regularization coefficient used to prevent model overfitting. Its value ranges from 0.001 to 0.01 and can be fine-tuned according to the actual training effect. n is the number of bands used, and here n=6.

[0063] This loss function constrains the error between the predicted spectrum and the true spectrum, ensuring inversion accuracy and guaranteeing that the pseudo-multispectral features can truly reflect the physical characteristics of the document.

[0064] S4. Compare and match the generated pseudo-multispectral features of the six bands with the standard multispectral library.

[0065] Among them, the standard document multispectral feature library is formed by collecting data on standard documents such as ID cards, real estate ownership certificates (property certificates), and business licenses using professional multispectral acquisition equipment (such as ultraviolet 365nm, blue light 450nm, visible light 550nm, red light 650nm, and near-infrared 850 / 940nm) under standard lighting, standard angles, and no wear conditions. After labeling, database construction, and structuring, an authoritative spectral benchmark database is formed, which is used to compare with the inversion spectrum of the document to be tested for counterfeit detection.

[0066] S5. Calculate the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral anomaly (LSA) after comparison and matching.

[0067] Principle of spectral feature discrimination of document paper:

[0068] Genuine document paper (such as PET substrate for ID cards and anti-counterfeiting paper for property certificates) has a fixed absorption peak in the near-infrared band. Therefore, the formula for calculating the spectral difference index (PSDI) of paper is as follows:

[0069]

[0070] in, For the RGB ID photo to be tested Spectral reflectance of the band This represents the spectral reflectance of a standard, authentic document at the corresponding wavelength. The number of bands involved in the calculation.

[0071] Judgment rules:

[0072] When PSDI < 0.1, the paper is genuine document paper; when PSDI ≥ 0.1, the paper is ordinary printing paper, thus it is determined to be counterfeit.

[0073] Principle of Spectral Characteristic Discrimination of Printing Inks:

[0074] Genuine documents use anti-counterfeiting ink, which exhibits a unique response in the ultraviolet / infrared bands, while counterfeit documents use CMYK printer ink, which has extremely high infrared reflectivity. Therefore, the formula for calculating the Ink Spectral Matching Degree (ISMD) is as follows:

[0075]

[0076] in, The spectral reflectance of the RGB ID photo to be tested. This refers to the spectral reflectance of a standard, authentic document.

[0077] Judgment rules:

[0078] When ISMD > 0.9, the ink matches and is judged as genuine; when ISMD ≤ 0.9, the ink does not match and is judged as counterfeit / altered.

[0079] Principle of alteration / PS trace detection:

[0080] Common modifications to online photos, such as erasing, splicing, or wiping, can cause local spectral abrupt changes.

[0081] Therefore, the definition formula for Local Spectral Anomalies (LSA) is:

[0082]

[0083] in, Represents pixels. The inversion spectral reflectance of the current pixel. This represents the average spectral reflectance of the current pixel's neighborhood (3×3 pixel neighborhood). This is the abnormal threshold (the value ranges from 0.05 to 0.1, and can be fine-tuned according to the type of document).

[0084] When LSA(p) > If a pixel is found to be abnormal, it is considered an abnormal pixel. If a continuous block of abnormal pixels (≥5 consecutive pixels) is found, it is considered to be a tampering behavior such as erasure or PS splicing.

[0085] This formula can accurately capture local spectral abrupt changes and effectively identify minor tampering, which meets the detection requirements of this invention.

[0086] S6. Calculate the overall authenticity score based on the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral anomalies (LSA);

[0087] The calculation of the overall authenticity discrimination score is achieved by weighting and fusing the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral outliers (LSA) to obtain the final discrimination score:

[0088]

[0089] in, + + = 1, , , This is a weighting coefficient, which can be fine-tuned according to the actual document type. The default value is: =0.3, =0.5, =0.2, This represents the proportion of abnormal pixels to the total number of pixels in the document area.

[0090] This formula comprehensively considers three core dimensions: paper, ink, and partial tampering. With a reasonable weighting, it can fully reflect the authenticity of the document.

[0091] Judgment rules:

[0092] When Score ≥ a, it is determined to be a genuine document;

[0093] When b≤Score < a, it is judged as suspected alteration;

[0094] When the score is less than b, it is judged as forgery / tampering.

[0095] Where a is 0.88 and b is 0.70.

[0096] S7. Based on the comparison result of the comprehensive authenticity judgment score and the preset threshold, output the authenticity, alteration or tampering judgment result of the RGB ID photo.

[0097] S8. Upload the results to the business hall system to complete automatic verification.

[0098] This invention also provides an online photo forgery detection system based on multispectral imaging feature inversion, comprising:

[0099] The image acquisition module is used to acquire RGB photos of the document to be inspected.

[0100] The image preprocessing module is used to perform distortion correction, dehazing, enhancement, and RGB ID photo region segmentation on RGB ID photos.

[0101] The spectral inversion module has a built-in neural network model for multispectral feature inversion, which is used to convert RGB ID photos into multi-band pseudo-multispectral reflectance feature maps.

[0102] The authenticity detection module is used to calculate the spectral differences of paper, the matching degree of ink spectrum, and local spectral anomalies, and output a weighted judgment score.

[0103] The result output module is used to feed the judgment results back to the business system.

Claims

1. A method for detecting online photo forgery based on multispectral imaging feature inversion, characterized in that, Includes the following steps: S1. Obtain the RGB document photo to be detected; S2. Preprocess the RGB ID photo; S3. Input the preprocessed RGB ID photo into the multispectral feature inversion model to generate pseudo-multispectral features in six bands; S4. Compare and match the generated pseudo-multispectral features of the six bands with the standard document multispectral library. S5. Calculate the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral anomaly (LSA) after comparison and matching. S6. Calculate the overall authenticity score based on the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral anomalies (LSA); S7. Based on the comparison result of the comprehensive authenticity judgment score and the preset threshold, output the authenticity, alteration or tampering judgment result of the RGB ID photo. S8. Upload the results to the business hall system to complete automatic verification.

2. The online photo forgery detection method based on multispectral imaging feature inversion according to claim 1, characterized in that, The preprocessing in step S2 includes distortion correction, dehazing, enhancement, and RGB ID photo region segmentation.

3. The online photo forgery detection method based on multispectral imaging feature inversion according to claim 1, characterized in that, The six bands in step S3 include the 365nm ultraviolet band, the 450nm blue light band, the 550nm green light band, the 650nm red light band, the 850nm near-infrared band, and the 940nm near-infrared band.

4. The online photo forgery detection method based on multispectral imaging feature inversion according to claim 1, characterized in that, In step S3, the multispectral feature inversion model is a neural network model, and its loss function is defined as: The reflectance obtained by inversion, The value represents the true multispectral reflectance obtained from professional multispectral acquisition equipment. Here, n is the regularization coefficient, and n is the number of bands used, where n=6; simultaneously, the bands are defined. Spectral reflectance at: in, For target documents in the band Radiance For standard whiteboard in band Radiance.

5. The online photo forgery detection method based on multispectral imaging feature inversion according to claim 1, characterized in that, The formula for calculating the Paper Spectral Difference Index (PSDI) in step S5 is as follows: in, For the RGB ID photo to be tested Spectral reflectance of the band This represents the spectral reflectance of a standard, authentic document at the corresponding wavelength. The number of bands involved in the calculation.

6. The online photo forgery detection method based on multispectral imaging feature inversion according to claim 1, characterized in that, The formula for calculating the ink spectral matching degree (ISMD) in step S5 is as follows: in, The spectral reflectance of the RGB ID photo to be tested. This refers to the spectral reflectance of a standard, authentic document.

7. The online photo forgery detection method based on multispectral imaging feature inversion according to claim 1, characterized in that, The definition formula for Local Spectral Anomaly (LSA) in step S5 is as follows: in, Represents pixels. The inversion spectral reflectance of the current pixel. This represents the average spectral reflectance of the current pixel's neighborhood (3×3 pixel neighborhood). This is the abnormal threshold, with a value range of 0.05 to 0.1, which can be fine-tuned according to the document type.

8. The online photo forgery detection method based on multispectral imaging feature inversion according to claim 1, characterized in that, In step S6, the overall authenticity discrimination score is calculated by weighted fusion of the paper spectral difference index (PSDI), ink spectral matching degree (ISMD), and local spectral anomaly (LSA). in, + + = 1, , , This is a weighting coefficient, which can be fine-tuned according to the actual document type. The default value is: =0.3, =0.5, =0.2, This represents the proportion of abnormal pixels to the total number of pixels in the document area.

9. An online photo forgery detection system based on multispectral imaging feature inversion, characterized in that, include: The image acquisition module is used to acquire RGB photos of the document to be inspected. The image preprocessing module is used to perform distortion correction, dehazing, enhancement, and RGB ID photo region segmentation on RGB ID photos. The spectral inversion module has a built-in neural network model for multispectral feature inversion, which is used to convert RGB ID photos into multi-band pseudo-multispectral reflectance feature maps. The authenticity detection module is used to calculate the spectral differences of paper, the matching degree of ink spectrum, and local spectral anomalies, and output a weighted judgment score. The result output module is used to feed the judgment results back to the business system.

Images