Power equipment nameplate image tampering detection method based on multi-view and multi-scale supervision
Patent Information
- Application Number
- CN202510568347.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2026-10-30
AI Technical Summary
传统图像篡改检测方法依赖自然场景的显著边缘或纹理特征，难以区分铭牌图像中的真实噪声与篡改痕迹，导致误检和漏检率居高不下
[0037]本发明通过边缘特征与噪声敏感双视图特征融合，结合通道与空间双注意力机制实现篡改痕迹的跨域关联，有效解决了传统方法中噪声干扰下特征混淆及小目标漏检问题；利用多层级监督约束(像素级、边缘级、图像级)强化局部细节与全局语义的一致性，并通过动态噪声注入与JPEG压缩模拟增强模型对复杂攻击的鲁棒性，显著提升了检测精度与泛化能力。此外，该方法支持端到端高效推理，可扩展应用于电力设备铭牌、工业标识及其他高噪声场景的图像真伪鉴别。。
Smart Images
Figure CN123016173A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision. Background Technology
[0002] Images of nameplates for power equipment often suffer from high noise and low contrast due to factors such as glare from the metal material and interference from the shooting environment. Furthermore, the tampered areas are often small text or icons with minimal difference from the real background. Traditional image tampering detection methods rely on the prominent edge or texture features of natural scenes, making it difficult to distinguish between real noise and tampering traces in nameplate images, resulting in high false positive and false negative rates.
[0003] While existing technologies (such as residual networks and dual-stream detection models) perform well in natural images, they are not optimized for nameplate scenes: they lack the ability to detect small targets, have poor robustness to JPEG compression and blur attacks, and single-pixel level supervision is susceptible to noise interference. Summary of the Invention
[0004] The purpose of this invention is to provide a method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision, in order to improve the detection accuracy and generalization ability of power equipment nameplate image tampering.
[0005] The objective of this invention can be achieved through the following technical solutions:
[0006] A method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision, the method comprising the following steps:
[0007] S1. Obtain the nameplate image of the power equipment to be detected as training data;
[0008] S2. The training data is input into the edge feature branch and noise-sensitive branch of the power equipment nameplate image tampering detection network. The outputs of the edge feature branch and noise-sensitive branch are input into the dual attention fusion module of the power equipment nameplate image tampering detection network.
[0009] S3, the dual attention fusion module performs channel attention and spatial attention weighted fusion on the outputs of edge feature branches and noise-sensitive branches to generate fused features;
[0010] S4. A multi-scale supervised decoder that integrates features input to the power equipment nameplate image tampering detection network outputs the tampering region localization result. Based on the loss function and the trained power equipment nameplate image tampering detection network, a power equipment nameplate image tampering detection model is obtained. The loss function includes pixel-level segmentation loss, edge-level detection loss and image-level authenticity loss.
[0011] S5. The power equipment nameplate image tampering detection model outputs actual tampering detection results based on actual power equipment nameplate images.
[0012] Furthermore, after the training data is input into the edge feature branch of the power equipment nameplate image tampering detection network, the following steps are performed in the edge feature branch:
[0013] S21. Perform Sobel operator preprocessing on the training data to generate an initial edge response map;
[0014] S22. Enhance the features of the initial edge response map by using a multi-layer dilated convolutional network to obtain enhanced edge features;
[0015] S23. The enhanced edge features are fused step by step with the deep semantic features output by the U-Net encoder after the training data is input into the U-Net encoder to obtain the edge features.
[0016] Furthermore, the multi-layer dilated convolutional network includes three layers of dilated convolutions with different dilation rates and channels.
[0017] Furthermore, after the training data is input into the noise-sensitive branch of the power equipment nameplate image tampering detection network, the following steps are performed in the noise-sensitive branch:
[0018] S24. Use the BayarConv layer to extract high-frequency noise features from the training data;
[0019] S25. High-frequency noise features are input into an improved ResNet-50 network, which outputs a noise-sensitive feature map.
[0020] Furthermore, the improved ResNet-50 network includes adaptive pooling layers and four-stage convolutional blocks of the ResNet-50 network.
[0021] Furthermore, after inputting high-frequency noise features into the improved ResNet-50 network, multi-level encoding is performed through the four-stage convolutional blocks of the ResNet-50 network. The encoded features are then input into an adaptive pooling layer for normalization to obtain a noise-sensitive feature map.
[0022] Furthermore, the dual-attention fusion module performs channel attention and spatial attention weighted fusion on the outputs of the edge feature branch and the noise-sensitive branch to generate fused features. The specific steps are as follows:
[0023] S31. Apply channel attention weighting to the edge features output by the edge feature branch to generate channel-enhanced features;
[0024] S32. Apply spatial attention weighting to the noise-sensitive feature map output by the noise-sensitive branch to generate spatially enhanced features;
[0025] S33. Add the channel enhancement features and spatial enhancement features element by element, and generate fused features by 1×1 convolution dimensionality reduction.
[0026] Furthermore, the loss function is:
[0027] Loss = α·Loss dice +β·loss ibe +(1-α-β)·Loss bce
[0028] Among them, Loss bce Loss represents the image-level realism loss. dice This represents pixel-level segmentation loss. ibe This indicates the edge-level detection loss;
[0029] The pixel-level segmentation loss is:
[0030]
[0031] in, x Given an input image, W and H represent the width and height of the image, respectively, and G(x) i y is the probability predicted by the model as to whether the i-th pixel has been tampered with, and its value ranges from 0 to 1. i ∈{0,1} is a binary label indicating whether the i-th pixel has been tampered with.
[0032] Furthermore, the edge-level detection loss is the Dice loss calculated at 1 / 4 resolution for tampered edges.
[0033] Furthermore, the image-level realism loss is:
[0034] Loss bce (x)=-(y·logG(x)+(1-y)·log(1-G(x)))
[0035] Where y=1 represents a tampered image, and y=0 represents a real image.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] This invention achieves cross-domain correlation of tampering traces by fusing edge features and noise-sensitive dual-view features, combined with a channel and spatial dual-attention mechanism. This effectively solves the problems of feature confusion under noise interference and missed detection of small targets in traditional methods. Multi-level supervised constraints (pixel-level, edge-level, and image-level) are used to strengthen the consistency between local details and global semantics, and dynamic noise injection and JPEG compression simulation enhance the model's robustness to complex attacks, significantly improving detection accuracy and generalization ability. Furthermore, this method supports efficient end-to-end inference and can be extended to image authentication of power equipment nameplates, industrial markings, and other high-noise scenarios. Attached Figure Description
[0038] Figure 1 This is a diagram of the overall model architecture;
[0039] Figure 2 Flowchart of a method for detecting tampering with nameplate images of power equipment
[0040] Figure 3 This is a diagram of the edge feature branch (EFB) structure.
[0041] Figure 4 This is a diagram of the noise-sensitive branch (NSB) structure.
[0042] Figure 5 This is a schematic diagram of the Dual Attention Fusion Module (DAF).
[0043] Figure 6 This is a comparison chart of tamper detection results in the example, where... Figure 6 (a) is a tampered image. Figure 6 (b) is the test result. Figure 6 (c) represents the tampered area. Detailed Implementation
[0044] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.
[0045] This invention proposes a method for detecting tampering in power equipment nameplate images based on multi-view and multi-scale supervision, the flowchart of which is shown below. Figure 2 As shown, the method includes the following steps:
[0046] S1. Obtain the nameplate image of the power equipment to be detected as training data;
[0047] S2. The training data is input into the edge feature branch and noise-sensitive branch of the power equipment nameplate image tampering detection network. The outputs of the edge feature branch and noise-sensitive branch are input into the dual attention fusion module of the power equipment nameplate image tampering detection network.
[0048] S3, the dual attention fusion module performs channel attention and spatial attention weighted fusion on the outputs of edge feature branches and noise-sensitive branches to generate fused features;
[0049] S4. A multi-scale supervised decoder that integrates features input to the power equipment nameplate image tampering detection network outputs the tampering region localization result. Based on the loss function and the trained power equipment nameplate image tampering detection network, a power equipment nameplate image tampering detection model is obtained. The loss function includes pixel-level segmentation loss, edge-level detection loss and image-level authenticity loss.
[0050] S5. The power equipment nameplate image tampering detection model outputs actual tampering detection results based on actual power equipment nameplate images.
[0051] The edge feature branch (EFB) extraction process in step S2 includes the following sub-steps:
[0052] S21. Perform Sobel operator preprocessing on the input image to generate an initial edge response map;
[0053] S22. Enhance the features of the initial edge response map by using a multi-layer dilated convolutional network to extract multi-scale boundary artifact features;
[0054] S23. The enhanced edge features are fused with the deep semantic features output by the U-Net encoder step by step.
[0055] The specific structure of the multi-layer dilated convolutional network in step S22 is as follows:
[0056] S221. The first layer uses a 3×3 dilated convolution with an inflation rate of 2, and the number of output channels is 64.
[0057] S222, The second layer uses a 3×3 dilated convolution with an inflation rate of 4, and the number of output channels is 128;
[0058] S223, the third layer uses a 3×3 dilated convolution with an inflation rate of 8, and the number of output channels is 256.
[0059] The construction of the noise-sensitive branch (NSB) in step S2 includes the following sub-steps:
[0060] S24. Use the BayarConv layer to extract high-frequency noise features from the input image;
[0061] S25. Noise features are encoded at multiple levels using the ResNet-50 backbone network to generate a noise-sensitive feature map.
[0062] S26. An adaptive pooling layer is introduced in the last layer of ResNet-50 to unify the feature map size to 1 / 16 of the original image.
[0063] The operations of the dual attention fusion module in step S3 include:
[0064] S31. Apply channel attention weighting to the edge feature map output by EFB to generate channel-enhanced features;
[0065] S32. Apply spatial attention weighting to the noise feature map output by NSB to generate spatially enhanced features;
[0066] S33. Add the channel enhancement features and spatial enhancement features element by element, and generate a fused feature map by dimensionality reduction through 1×1 convolution.
[0067] The implementation of the multi-scale supervision strategy in step S4 includes:
[0068] S41. Pixel-level segmentation loss: The Dice loss function is used to optimize the prediction accuracy of the tampered region mask. The calculation formula is as follows:
[0069]
[0070] in x Given an input image, W and H represent the width and height of the image, respectively. G(x) i ) represents the probability predicted by the model of whether the i-th pixel has been tampered with, and its value ranges from 0 to 1. y i ∈{0,1} is a binary label indicating whether the i-th pixel has been tampered with.
[0071] S42, Edge-level detection loss: Calculate the Dice loss for tampered edges at 1 / 4 resolution;
[0072] S43. Image-level Authenticity Loss: The overall image authenticity is determined using binary cross-entropy loss, with the following formula:
[0073] Loss bce (x)=-(y·logG(x)+(1-y)·log(1-G(x)))
[0074] Where y=1 represents a tampered image, and y=0 represents a real image.
[0075] S44. Multi-scale Loss: To effectively combine the three loss functions mentioned above, a weighted combination method is used to fuse pixel-level loss, edge loss, and image scale loss. Specifically, the combined total loss function can be expressed in the following form:
[0076] Loss = α·loss seg +β·lossibe +(1-α-β)·loss edg
[0077] Where α, β∈(0,1) are the weights. It is important to note the edge loss... ibe The calculation is performed only on real images, that is, only unaltered image data is used to calculate the loss term.
[0078] Step S5 also includes robustness enhancement processing:
[0079] S51. During the training phase, apply a JPEG compression attack to the input image, with the compression quality factor randomly selected between 50 and 90.
[0080] S52. Add Gaussian noise to the input image, with the noise variance range set to 0.002-0.01;
[0081] S53. A dynamic weight adjustment strategy is used to balance multi-scale losses. The weight formula is as follows:
[0082]
[0083] Where γ = 0.1, and t is the number of training rounds.
[0084] Figure 5 This is a schematic diagram of the Dual Attention Fusion Module (DAF). Figure 6 This is a comparison chart of tamper detection results in the example.
[0085] The core of this invention includes:
[0086] 1. Dual-view feature extraction: The edge feature branch (EFB) extracts artifacts at the boundary of the tampered region, and the noise-sensitive branch (NSB) captures the noise difference between the tampered and real regions, thus solving the feature confusion problem under noise interference;
[0087] 2. Dual attention fusion mechanism: Employs a weighted fusion of channel attention and spatial attention to multi-view features, enhancing the localization accuracy for small-scale tampering;
[0088] 3. Multi-scale supervision strategy: Combine pixel-level segmentation loss, edge-level detection loss and image-level realism classification loss to constrain the model to learn global and local consistent features;
[0089] 4. Anti-interference training optimization: JPEG compressed noise and Gaussian noise are dynamically injected during training to improve the model's robustness to post-processing attacks.
[0090] Compared with existing technologies, this invention achieves cross-domain correlation of tampering traces by fusing edge features and noise-sensitive dual-view features, combined with channel and spatial dual attention mechanisms. This effectively solves the problems of feature confusion and missed detection of small targets under noise interference in traditional methods. Multi-level supervised constraints (pixel-level, edge-level, and image-level) are used to strengthen the consistency between local details and global semantics, and dynamic noise injection and JPEG compression simulation enhance the model's robustness to complex attacks, significantly improving detection accuracy and generalization ability. Furthermore, this method supports efficient end-to-end inference and can be extended to image authentication of power equipment nameplates, industrial markings, and other high-noise scenarios.
[0091] like Figure 1 As shown, the tampering detection method of the present invention includes an image preprocessing module, an edge feature branch, a noise-sensitive branch, a dual attention fusion module, and a multi-scale supervised decoder.
[0092] ① Image preprocessing module: responsible for distortion correction, size normalization and pixel value standardization of the input image;
[0093] ② Edge feature branch: Composed of Sobel preprocessing layer and dilated convolution module, outputting boundary artifact feature map;
[0094] ③ Noise-sensitive branch: Composed of BayarConv layer and ResNet-50 backbone, outputs noise difference feature map;
[0095] ④ Dual Attention Fusion Module: Located at the intersection of the two channels, it performs channel and spatial attention weighted fusion on EFB and NSB features;
[0096] ⑤ Multi-scale supervised decoder: Based on U-Net structure upsampling, output pixel-level tampering probability map.
[0097] Based on the above structure, this embodiment provides a method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision, such as... Figure 2 As shown, it includes the following steps:
[0098] Step 1: Image Preprocessing and Enhancement
[0099] 1.1 Geometric Correction: After inputting the RGB image of the power equipment nameplate, the geometric distortion of the image is corrected by affine transformation. The rotation angle range is controlled within ±5°, and the scaling ratio is limited to 0.9 to 1.1 times to eliminate the deformation caused by the tilt of the shooting angle or the change of distance.
[0100] 1.2 Size normalization: The input image is uniformly scaled to 512×512 pixels using bilinear interpolation algorithm, and the RGB channels are normalized using Min-Max to linearly map the pixel values to the [0,1] interval;
[0101] 1.3 Dynamic Size Adaptation: For input images with non-fixed resolution, consistent processing is maintained through edge padding and adaptive cropping.
[0102] Step 2: Dual-channel feature extraction
[0103] 2.1 Edge Feature Branch (EFB):
[0104] like Figure 3 Based on the U-Net architecture, it embeds an edge-aware module (EPB) containing Sobel layers and convolutional blocks. A progressive feature fusion strategy is employed, gradually fusing shallow edge features with deep semantic features. The output includes a tampered region segmentation map and an edge detection map.
[0105] 2.2 Noise-Sensitive Branch (NSB):
[0106] like Figure 4 As shown, high-frequency noise residuals are extracted through a BayarConv layer (5×5 convolution kernel, center weight 1, and surrounding weights sum to -1);
[0107] The residual map is input into the improved ResNet-50 backbone network, the global pooling layer is removed, the four-stage convolutional blocks are retained, and the output is a 32×32×256 noise-sensitive feature map.
[0108] Step 3: Dual Attention Feature Fusion
[0109] 3.1 Channel Attention (CA): Global average pooling is performed on the EFB feature map to generate channel weight vectors, using the following formula:
[0110] w c =σ(W2·ReLU(W1·AvgPool(F) efb )))
[0111] Where W1∈R C×C / 4 W2∈R C / 4×C These are learnable parameters;
[0112] 3.2 Spatial Attention (SA): A 7×7 convolution is performed on the NSB feature map to generate a spatial weight matrix.
[0113] w s =σ(Conv 7×7 (F nsb ))
[0114] 3.3 Feature Fusion: The weighted EFB and NSB features are added together, and then the dimensionality is reduced to 512 channels by 1×1 convolution, outputting a 32×32×512 fused feature map.
[0115] Step 4: Multi-scale supervised decoding
[0116] 4.1 By combining pixel-level segmentation loss, edge-level detection loss, and image-level realism classification loss, the model is constrained to learn global and local consistent features;
[0117] 4.2 Decoding the features: The resolution is gradually restored to 512×512 by transposed convolution (kernel size 4×4, stride 2). The upsampled features at each level are concatenated with the corresponding EFB layer features to enhance detail preservation. After Sigmoid activation, the features are binarized (threshold 0.5) and the tampered region mask is output.
[0118] Step 5: Post-processing optimization
[0119] 5.1 Morphological opening operation: A 3×3 core is used to remove isolated noise points;
[0120] 5.2 Connected Component Filtering: Retain candidate regions with an area ≥ 50 pixels and mark them as the final tampered regions.
[0121] This invention relates to the field of image tampering detection technology, specifically proposing a method for detecting tampering in power equipment nameplate images based on multi-view and multi-scale supervision. Addressing the issues of false positives and false negatives caused by material reflection, noise interference, and small-scale text tampering in existing tampering detection algorithms in power equipment nameplate scenarios, this invention proposes a dual-channel network architecture: an edge feature branch (EFB) extracts artifacts at the boundaries of the tampered area, a noise-sensitive branch (NSB) captures inconsistencies between the tampered area and the real area, and a dual attention fusion module integrates multi-view features; combining pixel-level, edge-level, and image-level multi-scale supervision strategies, and optimizing the model using Dice loss and image-level binary cross-entropy loss. Experiments show that this method is significantly more robust than existing technologies under JPEG compression and noise attacks, accurately locating tampered areas of nameplate text and icons, thus ensuring the data security of power systems.
[0122] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A method for detecting image tampering on nameplates of power equipment based on multi-view and multi-scale supervision, characterized in that, The method includes the following steps: S1. Obtain the nameplate image of the power equipment to be detected as training data; S2. The training data is input into the edge feature branch and noise-sensitive branch of the power equipment nameplate image tampering detection network. The outputs of the edge feature branch and noise-sensitive branch are input into the dual attention fusion module of the power equipment nameplate image tampering detection network. S3, the dual attention fusion module performs channel attention and spatial attention weighted fusion on the outputs of edge feature branches and noise-sensitive branches to generate fused features; S4. A multi-scale supervised decoder that integrates features input to the power equipment nameplate image tampering detection network outputs the tampering region localization result. Based on the loss function and the trained power equipment nameplate image tampering detection network, a power equipment nameplate image tampering detection model is obtained. The loss function includes pixel-level segmentation loss, edge-level detection loss and image-level authenticity loss. S5. The power equipment nameplate image tampering detection model outputs actual tampering detection results based on actual power equipment nameplate images.
2. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 1, characterized in that, After the training data is input into the edge feature branch of the power equipment nameplate image tampering detection network, the following steps are performed in the edge feature branch: S21. Perform Sobel operator preprocessing on the training data to generate an initial edge response map; S22. Enhance the features of the initial edge response map by using a multi-layer dilated convolutional network to obtain enhanced edge features; S23. The enhanced edge features are fused step by step with the deep semantic features output by the U-Net encoder after the training data is input into the U-Net encoder to obtain the edge features.
3. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 2, characterized in that, Multilayer dilated convolutional networks consist of three layers of dilated convolutions with different dilation rates and channels.
4. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 1, characterized in that, After the training data is input into the noise-sensitive branch of the power equipment nameplate image tampering detection network, the following steps are performed in the noise-sensitive branch: S24. Use the BayarConv layer to extract high-frequency noise features from the training data; S25. High-frequency noise features are input into an improved ResNet-50 network, which outputs a noise-sensitive feature map.
5. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 4, characterized in that, The improved ResNet-50 network includes adaptive pooling layers and four-stage convolutional blocks of the ResNet-50 network.
6. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 5, characterized in that, After inputting high-frequency noise features into the improved ResNet-50 network, multi-level encoding is performed through the four-stage convolutional blocks of the ResNet-50 network. The encoded features are then input into an adaptive pooling layer for normalization to obtain a noise-sensitive feature map.
7. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 5, characterized in that, The dual-attention fusion module performs channel attention and spatial attention weighted fusion on the outputs of the edge feature branch and the noise-sensitive branch to generate fused features. The specific steps are as follows: S31. Apply channel attention weighting to the edge features output by the edge feature branch to generate channel-enhanced features; S32. Apply spatial attention weighting to the noise-sensitive feature map output by the noise-sensitive branch to generate spatially enhanced features; S33. Add the channel enhancement features and spatial enhancement features element by element, and generate fused features by 1×1 convolution dimensionality reduction.
8. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 1, characterized in that, The loss function is: Loss=a·Loss dice +β·loss ibe +(1-a-b)·Loss bce Among them, Loss bce Loss represents the image-level realism loss. dice This represents pixel-level segmentation loss. ibe This indicates the edge-level detection loss; The pixel-level segmentation loss is: in, x Given an input image, W and H represent the width and height of the image, respectively, and G(x) i y is the probability predicted by the model as to whether the i-th pixel has been tampered with, and its value ranges from 0 to 1. i ∈{0,1} is a binary label indicating whether the i-th pixel has been tampered with.
9. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 1, characterized in that, The edge-level detection loss is the Dice loss calculated at 1 / 4 resolution for tampered edges.
10. The method for detecting tampering of power equipment nameplate images based on multi-view and multi-scale supervision according to claim 1, characterized in that, Image-level realism loss is: Loss bce (x)=-(y·logG(x)+(1-y)·log(1-G(x))) Where y=1 represents a tampered image, and y=0 represents a real image.