Image restoration evidence obtaining method and device based on multi-color space
By extracting and fusing features in RGB, Lab, and HSV color spaces and utilizing a spatially weighted channel attention mechanism, the problem of inaccurate localization of image restoration areas in existing technologies is solved, achieving more efficient image restoration and evidence collection results.
Patent Information
- Application Number
- CN202511000883.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-21
- Publication Date
- 2025-11-14
AI Technical Summary
Existing technologies struggle to accurately locate tampered areas in deep learning-based image restoration algorithms, especially in complex restoration scenarios where the results are poor.
An image restoration and forensic method based on multiple color spaces is adopted. By adaptively fusing the features of RGB, Lab, and HSV color spaces, and utilizing a spatially weighted channel attention mechanism, the image restoration area is extracted and located.
It improves the evidence collection performance and generalization ability in complex restoration scenarios, and can more accurately identify image restoration areas, thereby improving the model's evidence collection efficiency and accuracy.
Smart Images
Figure CN120953126A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image forensics in computer vision, and also belongs to the field of data service technology. More specifically, it relates to an image restoration forensics method and apparatus based on multiple color spaces. Background Technology
[0002] In the digital media era, the emergence of image restoration, a new image editing technology, has posed a serious challenge to digital information security. Editing and tampering with images using image restoration algorithms has become convenient and simple, and the resulting forged images are extremely difficult to detect. The proliferation of tampered images spreading misinformation online not only misleads the public's understanding of the facts but also poses a serious threat to social order and national security.
[0003] Image inpainting presents new challenges to digital image forensics. Image inpainting refers to removing parts of the original information of an image and using restoration methods to complete the image, thereby achieving image tampering. Image inpainting forensics refers to the techniques for identifying and locating the restored areas in an image. Existing inpainting forensics solutions have already achieved a certain level of forensic performance. However, for the now mainstream deep learning-based image inpainting, related forensic research is still relatively limited, and there are problems with inaccurate location of tampered areas; because the content, shape, and size of the restored areas are complex and diverse, most existing solutions also lack the ability to handle complex inpainting scenarios.
[0004] Therefore, there is a need to propose an image forensics method that is oriented towards deep restoration and can adapt to a variety of complex restoration scenarios. Summary of the Invention
[0005] This invention provides an image restoration and forensic method and apparatus based on multiple color spaces, which can achieve superior forensic performance compared with existing technical solutions for restoring tampered images with various complex restoration scenarios.
[0006] In a first aspect, the present invention provides an image restoration and forensics method based on multiple color spaces, comprising: The RGB color space image is converted into a Lab color space version image and an HSV color space version image respectively through color transformation; Features were extracted from the restored tampered image in RGB color space, Lab color space, and HSV color space versions, respectively. By adaptively fusing features from three color spaces using a spatially weighted channel attention mechanism, features suitable for current image forensics are obtained. Based on features suitable for current image forensics, the repaired area is located, thereby determining which area of the image was generated by the image restoration algorithm.
[0007] In some instances, the extraction of features from the restored tampered image in the RGB color space image, the Lab color space version image, and the HSV color space version image respectively includes: The feature extractor extracts features from the restored tampered image in RGB color space, Lab color space, and HSV color space versions respectively. The feature extractor consists of a preliminary feature extraction unit and several cascaded residual blocks. The preliminary feature extraction unit is a convolutional layer used to map the image into features. The residual blocks are used to further extract features and are composed of two cascaded convolutional layers with ReLU activation layers connected in series. The output channel consists of another convolutional layer and a residual operation.
[0008] In some instances, the adaptive fusion of features from the three color spaces using a spatially weighted channel attention mechanism yields features suitable for forensic analysis of the current image, including: The features of the three color spaces are initially fused by stacking them to obtain a feature with 96 channels; The feature channels are adaptively weighted using a spatially weighted channel attention mechanism. Convolutional layers are used to fuse adaptively weighted spatial features.
[0009] In some instances, the spatially weighted channel attention mechanism includes: a preliminary feature extraction part, a spatial location weighting part, and a channel dimension weighting part. The preliminary feature extraction part performs initial feature extraction using convolutional layers and ReLU activation layers. The spatial location weighting part uses cross-channel average pooling and cross-channel max pooling operations to obtain two single-channel feature maps from the initially extracted features. These two single-channel feature maps are stacked and then processed using convolutional layers and a sigmoid activation function layer to obtain a spatial weight map. Simultaneously, convolutional layers and ReLU layers are used to extract features again from the initially extracted features. The spatial weight map is used to perform spatial location weighting on the re-extracted features. The channel dimension weighting part calculates a channel weight map from the spatially weighted feature map using a channel attention structure, and then uses the channel weight map to perform channel weighting on the re-extracted features.
[0010] In some instances, the channel attention structure consists of a global max pooling layer, a ReLU layer inserted between two fully connected layers, and a sigmoid activation function.
[0011] Secondly, the present invention provides an image restoration and forensic device based on multiple color spaces, comprising: The multi-color space conversion unit is used to convert RGB color space images into Lab color space versions and HSV color space versions respectively through color transformation; The multi-color space feature extraction unit is used to extract features from the restored tampered image in RGB color space image, Lab color space version image and HSV color space version image respectively; The multi-color space feature adaptive fusion unit is used to adaptively fuse features in three color spaces through a spatially weighted channel attention mechanism to obtain features suitable for current image forensics. The evidence prediction unit is used to locate the area to be repaired based on features suitable for the current image for evidence collection, thereby determining which area of the image was generated by the image restoration algorithm.
[0012] In some instances, the multi-color space feature extraction unit is used to extract features from the repaired tampered image in the RGB color space image, the Lab color space version image, and the HSV color space version image respectively through a feature extractor. The feature extractor consists of a preliminary feature extraction unit and several cascaded residual blocks. The preliminary feature extraction unit is a convolutional layer used to map the image into features. The residual blocks are used to further extract features and consist of two cascaded convolutional layers with ReLU activation layers connected in series. The output channel consists of another convolutional layer and a residual operation.
[0013] In some instances, the multi-color space feature adaptive fusion unit is used to initially fuse features from three color spaces in a stacked manner to obtain features with 96 channels; adaptive weighting of each channel of the feature is performed through a spatially weighted channel attention mechanism; and convolutional layers are used to fuse the adaptively weighted spatial features.
[0014] In some instances, the spatially weighted channel attention mechanism includes: a preliminary feature extraction part, a spatial location weighting part, and a channel dimension weighting part. The preliminary feature extraction part performs initial feature extraction using convolutional layers and ReLU activation layers. The spatial location weighting part uses cross-channel average pooling and cross-channel max pooling operations to obtain two single-channel feature maps from the initially extracted features. These two single-channel feature maps are stacked and then processed using convolutional layers and a sigmoid activation function layer to obtain a spatial weight map. Simultaneously, convolutional layers and ReLU layers are used to extract features again from the initially extracted features. The spatial weight map is used to perform spatial location weighting on the re-extracted features. The channel dimension weighting part calculates a channel weight map from the spatially weighted feature map using a channel attention structure, and then uses the channel weight map to perform channel weighting on the re-extracted features.
[0015] In some instances, the channel attention structure consists of a global max pooling layer, a ReLU layer inserted between two fully connected layers, and a sigmoid activation function.
[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: Compared with existing technologies, the main advantages are: (1) This invention makes comprehensive use of the features of multiple color spaces, has better evidence collection performance and generalization performance, and can adaptively handle different evidence collection scenarios; (2) The present invention adopts a non-downsampling feature coding structure, which can better preserve the restoration trace information of the image.
[0017] Compared with existing technologies, the main innovation lies in: (1) Existing technologies only extract features in the RGB color space, which is difficult to use for evidence collection in various complex scenarios. This invention makes comprehensive use of the features of the RGB color space, HSV color space, and Lab color space.
[0018] (2) This invention proposes a channel attention mechanism based on spatial weighting, which not only adaptively adjusts the importance of different feature channels, but also adaptively adjusts the importance of different feature spatial positions, so that the model pays more attention to important features and improves performance and efficiency.
[0019] (3) The present invention also proposes a feature adaptive fusion method for multiple color spaces, which further improves the evidence collection capability of the model. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a schematic diagram of an image restoration and forensics method based on multiple color spaces provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall network structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the residual block structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a spatially weighted channel attention structure provided in an embodiment of the present invention. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] In the following description, specific embodiments of the invention will be illustrated with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to several times as being performed by a computer, and computer execution as referred to herein includes operations by a computer processing unit representing electronic signals of data in a structured format. This operation transforms the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise alter the operation of the computer in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of the invention described above are not intended to be limiting, and those skilled in the art will understand that many of the following steps and operations can also be implemented in hardware.
[0024] The terms "module" or "unit" as used herein can be considered as software objects executing on the computing system. Different components, modules, engines, and services described herein can be considered as implementations on the computing system. The apparatus and methods described herein are preferably implemented in software, but can also be implemented in hardware, both of which are within the scope of this invention.
[0025] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.
[0026] like Figure 1 As shown in the figure, an image restoration and forensics method based on multiple color spaces provided by an embodiment of the present invention includes the following steps: (1) Convert the RGB color space image to Lab and HSV color space versions respectively through color transformation; (2) Extract features from the restored tampered image in the RGB, Lab, and HSV color spaces respectively; (3) A channel attention mechanism based on spatial weighting is proposed to adaptively fuse features in three color spaces to obtain features suitable for current image forensics. (4) Based on the above features, the evidence prediction unit is used to locate the repaired area, thereby determining which area of the image was generated by the image repair algorithm.
[0027] Furthermore, the above-mentioned feature extraction for restoring tampered images in RGB color space images, Lab color space versions, and HSV color space versions includes: The feature extractor extracts features from the restored tampered image in RGB color space, Lab color space, and HSV color space versions respectively. The feature extractor consists of a preliminary feature extraction unit and several cascaded residual blocks. The preliminary feature extraction unit is a convolutional layer used to map the image into features. The residual blocks are used to further extract features and are composed of two cascaded convolutional layers with ReLU activation layers connected in series. The output channel consists of another convolutional layer and a residual operation.
[0028] Furthermore, the above-mentioned feature fusion, based on a spatially weighted channel attention mechanism, adaptively fuses features from the three color spaces to obtain features suitable for current image forensics, including: The features of the three color spaces are initially fused by stacking them to obtain a feature with 96 channels; The feature channels are adaptively weighted using a spatially weighted channel attention mechanism. Convolutional layers are used to fuse adaptively weighted spatial features.
[0029] Furthermore, the aforementioned spatially weighted channel attention mechanism includes: a preliminary feature extraction part, a spatial location weighting part, and a channel dimension weighting part. The preliminary feature extraction part performs initial feature extraction using convolutional layers and ReLU activation layers. The spatial location weighting part uses cross-channel average pooling and cross-channel max pooling operations to obtain two single-channel feature maps from the initially extracted features. These two single-channel feature maps are stacked and then processed using convolutional layers and a sigmoid activation function layer to obtain a spatial weight map. Simultaneously, convolutional layers and ReLU layers are used to extract features again from the initially extracted features. The spatial weight map is used to perform spatial location weighting on the re-extracted features. The channel dimension weighting part calculates a channel weight map from the spatially weighted feature map using a channel attention structure, and then uses the channel weight map to perform channel weighting on the re-extracted features.
[0030] Furthermore, the aforementioned channel attention structure consists of one global max pooling layer, one ReLU layer inserted between two fully connected layers, and one Sigmoid activation function.
[0031] This invention extracts features from multiple color spaces and adaptively fuses these features using a spatially weighted channel attention mechanism to obtain features suitable for current image forensics. This enables adaptive processing of images with different content information, different repair area sizes, and different repair area shapes, thus providing an accurate and efficient image restoration forensics solution.
[0032] Another embodiment of the present invention provides an image restoration and forensic device based on multiple color spaces, the overall structure of which is as follows: Figure 2 As shown, it includes four parts: a multi-color space conversion unit, a multi-color space feature extraction unit, a multi-color space feature adaptive fusion unit, and an evidence prediction unit. (1) Multi-color space conversion unit: For the input digital image in RGB color space, the Lab color space version and HSV color space version of the image are obtained by color transformation.
[0033] (2) Multi-color space feature extraction unit: Images in RGB, Lab, and HSV color spaces are input into the corresponding feature extraction units. The feature extractors for the three spaces have the same structure, consisting of a preliminary feature extraction unit and eight cascaded residual blocks. The preliminary feature extraction unit is a convolutional layer with a kernel size of 3×3 and 32 output channels, used to map the image to features. The residual blocks are used for further feature extraction, consisting of two cascaded convolutional layers with kernel sizes of 3×3 and ReLU activation functions, each with 32 output channels, and a residual operation. The residual block structure is as follows: Figure 3As shown. It is worth noting that, considering that forensic tasks are more concerned with the detailed information of images, this invention does not use downsampling operations.
[0034] (3) Multi-color space feature adaptive fusion unit: The features of the three color spaces are initially fused in a stacked manner to obtain a feature with 96 channels; then, a channel attention mechanism based on spatial weighting is proposed to adaptively weight each channel of this feature; finally, a convolutional layer with a kernel size of 1×1 and an output channel of 32 is used to achieve the fusion of multi-color space features. Among them, the channel attention mechanism based on spatial weighting is as follows: Figure 4 As shown, it is divided into 3 parts: 1) Preliminary feature extraction: Preliminary feature extraction is performed using a 3×3 convolutional layer and the ReLU activation function; 2) Spatial location weighting: The initially extracted features are used to obtain two single-channel feature maps by cross-channel average pooling and cross-channel max pooling operations. After stacking, a spatial weight map is obtained by using a 3×3 convolutional layer and a Sigmoid activation function. At the same time, a 3×3 convolutional layer and a ReLU layer are used to extract the features again from the initially extracted features, and then the features are spatially weighted.
[0035] 3) Channel-weighted feature map: The spatially weighted feature map is processed through a channel attention mechanism to obtain a channel weight map, and the features are then weighted by channel. The channel attention structure consists of one global max pooling layer, one ReLU layer inserted between two fully connected layers, and one sigmoid activation function.
[0036] (4) Evidence prediction unit: The fused features are decoded using 8 concatenated residual blocks, and then a convolutional layer with a kernel size of 3×3 and an output channel of 1 is used to convert the features into a probability map of whether the image has been repaired, and the final output result is obtained.
[0037] In another embodiment of the present invention, an image restoration and forensic method based on multiple color spaces is designed. Features are extracted from the image in RGB, Lab, and HSV color spaces respectively. Then, a spatially weighted channel attention mechanism is proposed to adaptively fuse the features from the three color spaces to obtain features suitable for the current image forensics. Finally, based on these features, an forensic prediction unit is used to locate the area to be restored to generate the final forensic result. The specific implementation steps are as follows: Step 1: Obtain the image restoration dataset. Select representative deep inpainting algorithms to construct the image dataset. The image dataset contains several images that have undergone image restoration and their corresponding restoration location labels.
[0038] Step 2: Train the network model using the cross-entropy loss function; Step 3: For the image to be used for evidence collection, use the multi-color space conversion unit to obtain its versions in HSV color space and Lab color space respectively; Step 4: Use the multi-color space feature extraction unit to extract features from images in different color spaces; Step 5: Use the multi-color space feature adaptive fusion unit to adaptively fuse the features of the three color spaces; Step 6: Based on the fused features, use the forensic prediction unit to obtain the forensic results.
[0039] The method proposed in this invention is implemented based on the PyTorch framework, and the model is trained and tested in an Ubuntu environment. The forensic performance of the model is evaluated using an image restoration forensics dataset. Different forensic schemes are tested on the test set, and the forensic metrics F1 and IOU are shown in Table 1.
[0040] Table 1 compares the performance of the proposed algorithm with existing methods in terms of F1 (%) and IoU (%).
[0041] Note: Best results are marked in bold. Metric Explanation: IOU stands for Intersection over Union (IoU). As one of the most important evaluation criteria for object detection and semantic segmentation, IoU refers to the ratio of the intersection and union of the verified tampered region and the real tampered region. F1 score is a comprehensive metric used in statistics to measure the accuracy of binary classification models, taking into account both precision and recall. Both are standard metrics in the field of forensics and can better reflect the quality of forensic results in complex scenarios.
[0042] Table 1 shows the quantitative comparison results of different forensic techniques. The data comparison shows that, compared to existing forensic techniques, this model achieves a 10.85% higher IOU and an 8.14% higher F1 score than the suboptimal solution. Therefore, this invention achieves superior forensic performance compared to other comparative solutions.
[0043] This invention may be implemented in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects as illustrative rather than restrictive, for example: 1) multimedia file types are not limited to images; 2) image restoration algorithms are not limited to those used in the embodiments; 3) the selection of network configuration parameters is not limited to the configurations in the embodiments, such as the number of network branches or loss function parameters. The scope of this invention is indicated by the appended claims rather than the foregoing description. All variations falling within the meaning and scope of equivalent technical solutions of the claims are included within their scope.
[0044] The foregoing has provided a detailed description of an image restoration and forensic method and apparatus based on multiple color spaces provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for image restoration and forensics based on multiple color spaces, characterized in that, include: The RGB color space image is converted into a Lab color space version image and an HSV color space version image respectively through color transformation; Features were extracted from the restored tampered image in RGB color space, Lab color space, and HSV color space versions, respectively. By adaptively fusing features from three color spaces using a spatially weighted channel attention mechanism, features suitable for current image forensics are obtained. Based on features suitable for current image forensics, the repaired area is located, thereby determining which area of the image was generated by the image restoration algorithm.
2. The method according to claim 1, characterized in that, The step of extracting features from the repaired and tampered image in the RGB color space image, Lab color space version image, and HSV color space version image respectively includes: The feature extractor extracts features from the restored tampered image in RGB color space, Lab color space, and HSV color space versions respectively. The feature extractor consists of a preliminary feature extraction unit and several cascaded residual blocks. The preliminary feature extraction unit is a convolutional layer used to map the image into features. The residual blocks are used to further extract features and are composed of two cascaded convolutional layers with ReLU activation layers connected in series. The output channel consists of another convolutional layer and a residual operation.
3. The method according to claim 2, characterized in that, The method involves adaptively fusing features from three color spaces using a spatially weighted channel attention mechanism to obtain features suitable for current image forensics, including: The features of the three color spaces are initially fused by stacking them to obtain a feature with 96 channels; The feature channels are adaptively weighted using a spatially weighted channel attention mechanism. Convolutional layers are used to fuse adaptively weighted spatial features.
4. The method according to claim 3, characterized in that, The spatially weighted channel attention mechanism includes: a preliminary feature extraction part, a spatial location weighting part, and a channel dimension weighting part. The preliminary feature extraction part performs initial feature extraction using convolutional layers and ReLU activation layers. The spatial location weighting part uses cross-channel average pooling and cross-channel max pooling operations to obtain two single-channel feature maps from the initially extracted features. These two single-channel feature maps are stacked and then processed using convolutional layers and a sigmoid activation function layer to obtain a spatial weight map. Simultaneously, convolutional layers and ReLU layers are used to extract features again from the initially extracted features. The spatial weight map is used to perform spatial location weighting on the re-extracted features. The channel dimension weighting part calculates a channel weight map from the spatially weighted feature map using a channel attention structure. The channel weight map is then used to perform channel weighting on the re-extracted features.
5. The method according to claim 4, characterized in that, The channel attention structure consists of one global max pooling layer, one ReLU layer inserted between two fully connected layers, and one Sigmoid activation function.
6. An image restoration and forensic device based on multiple color spaces, characterized in that, include: The multi-color space conversion unit is used to convert RGB color space images into Lab color space versions and HSV color space versions respectively through color transformation; The multi-color space feature extraction unit is used to extract features from the restored tampered image in RGB color space image, Lab color space version image and HSV color space version image respectively; The multi-color space feature adaptive fusion unit is used to adaptively fuse features in three color spaces through a spatially weighted channel attention mechanism to obtain features suitable for current image forensics. The evidence prediction unit is used to locate the area to be repaired based on features suitable for the current image for evidence collection, thereby determining which area of the image was generated by the image restoration algorithm.
7. The apparatus according to claim 6, characterized in that, The multi-color space feature extraction unit is used to extract features from the repaired and tampered image in the RGB color space image, Lab color space version image, and HSV color space version image respectively through a feature extractor. The feature extractor consists of a preliminary feature extraction unit and several cascaded residual blocks. The preliminary feature extraction unit is a convolutional layer used to map the image into features. The residual blocks are used to further extract features and are composed of two cascaded convolutional layers with ReLU activation layers connected in series. The output channel consists of another convolutional layer and a residual operation.
8. The apparatus according to claim 7, characterized in that, The multi-color space feature adaptive fusion unit is used to initially fuse the features of the three color spaces in a stacked manner to obtain features with 96 channels; adaptive weighting of each channel of the feature is performed through a spatial weighted channel attention mechanism; and convolutional layers are used to fuse the adaptively weighted spatial features.
9. The apparatus according to claim 8, characterized in that, The spatially weighted channel attention mechanism includes: a preliminary feature extraction part, a spatial location weighting part, and a channel dimension weighting part. The preliminary feature extraction part performs initial feature extraction using convolutional layers and ReLU activation layers. The spatial location weighting part uses cross-channel average pooling and cross-channel max pooling operations to obtain two single-channel feature maps from the initially extracted features. These two single-channel feature maps are stacked and then processed using convolutional layers and a sigmoid activation function layer to obtain a spatial weight map. Simultaneously, convolutional layers and ReLU layers are used to extract features again from the initially extracted features. The spatial weight map is used to perform spatial location weighting on the re-extracted features. The channel dimension weighting part calculates a channel weight map from the spatially weighted feature map using a channel attention structure. The channel weight map is then used to perform channel weighting on the re-extracted features.
10. The apparatus according to claim 9, characterized in that, The channel attention structure consists of one global max pooling layer, one ReLU layer inserted between two fully connected layers, and one Sigmoid activation function.