An old map restoration method and system for multi-style global degradation

Abstract

The application discloses an old map restoration method and system for multi-style global degradation, taking Swin Transformer layered window self-attention as the main part to construct a HistoMapFormer restoration model, wherein: the encoding end divides the input image into 4*4 non-overlapping image blocks through patch division, and projects to a set dimension through a linear embedding layer; a plurality of HistoMapFormer blocks are stacked for feature transformation, the feature map resolution is gradually reduced through patch merging, and multi-scale layered representation is formed; the decoding end recovers the spatial resolution by gradually upsampling along the U-Net path, and adaptively weights and fuses the main features and skip connection features of the encoding end HistoMapFormer block on the aligned scale by using an SK Fusion module; a mixed attention module is introduced in the HistoMapFormer block to perform window multi-head self-attention and depth separable convolution in parallel, and long-range dependencies and local spatial priors are captured respectively; a Rescale / Rebias gate is introduced on the attention and feedforward residual path to stabilize the model for the Transformer architecture, and the HistoMapFormer model is used to directly output the restored map, thereby solving the problems of poor global degradation restoration effect, weak generalization ability, difficulty in balancing global topology and local details, and inability to meet the GIS application requirements of the prior art.

Description

Technical Field

[0001] This invention relates to the field of digital cultural heritage protection and computer vision interdisciplinary technology, specifically to a method and system for restoring old map images, which can be applied to scenarios such as digital preservation of cultural heritage archives, construction of historical geographic information systems, extraction and vectorization of elements from old maps, and digital display of cultural heritage. Background Technology

[0002] Antique maps are important historical and cultural heritages, possessing irreplaceable value in fields such as historical research and environmental change analysis. Currently, the restoration of antique maps mainly includes two methods: traditional physical restoration and digital image restoration. Traditional physical restoration slows down paper aging through methods such as deacidification, cleaning, and local reinforcement, but it suffers from problems such as high restoration costs, long cycles, and the potential for secondary damage to the original. Furthermore, it cannot completely eliminate global deterioration phenomena caused by long-term paper oxidation, such as yellowing of the base paper, overall fading, color shift, and uneven lighting, nor can it correct geometric distortions caused by paper aging and deformation.

[0003] With the development of digital technology, image-based digital restoration methods have gradually become an important supplement to the preservation of old maps. Existing research has achieved the restoration of local damage and missing parts of old maps (such as the MapMender model), but the following key technical shortcomings still exist: (1) Focus on local repair and ignore global degradation: Existing algorithms are mostly designed for local damage such as tearing and stains, and cannot effectively deal with systemic degradation that affects the whole map. Global degradation is the most common form of damage to old maps.

[0004] (2) Insufficient generalization ability: Old maps vary significantly across eras, media, and coloring techniques. Existing models are difficult to adapt to various cartographic styles such as traditional maps, landscape paintings, mileage maps, and oil paintings. The restoration effect on maps of different regions fluctuates greatly.

[0005] (3) Difficulty in balancing global structure and local details: Traditional convolutional neural networks are difficult to capture long-range spatial dependencies, resulting in distortion of the topological structure of the repaired map; while early Transformer models have high computational complexity, making it difficult to handle high-resolution maps, and are prone to problems such as detail dulling and texture "plastic feel".

[0006] (4) The repair accuracy cannot meet the needs of GIS applications: The maps repaired by the existing methods are insufficient in terms of linear feature continuity, text annotation recognition, and color reproduction, which will introduce systematic errors in subsequent geographic registration, feature vectorization and place name recognition.

[0007] Therefore, there is an urgent need for an old map restoration method that can simultaneously solve the problems of poor global degradation repair effect, weak generalization ability, difficulty in taking into account both global topology and local details, and repair accuracy that cannot meet the needs of GIS applications. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention proposes a method and system for restoring ancient maps with global degradation across multiple styles. This method effectively solves problems such as poor global degradation restoration effects, weak generalization ability, difficulty in taking into account global details, and restoration accuracy that cannot meet the needs of GIS applications.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for restoring ancient maps with multi-style global degradation includes the following steps: Step 1: Obtain old map image data, and after image preprocessing and degradation processing, form an old map image dataset; Step 2: Using the Swing Transformer hierarchical window self-attention as the backbone, construct the HistoMapFormer repair model. The model has a U-shaped encoding / decoding architecture, where: Encoding end: The input image is divided into 4×4 non-overlapping image blocks by patching, and projected to the set dimension through a linear embedding layer; multiple HistoMapFormer blocks are stacked for feature transformation, and the feature map resolution is gradually reduced by merging the patches to form a multi-scale hierarchical representation; Decoding end: Spatial resolution is restored by upsampling along the U-Net path. At the alignment scale, the backbone features and skip connection features of the SKFusion module and the HistoMapFormer block of the encoding end are adaptively weighted and fused. The HistoMapFormer block introduces a hybrid attention module to execute window multi-head self-attention and depthwise separable convolution in parallel, capturing long-range dependencies and local spatial priors respectively; Rescale / Rebias gating is introduced on the attention and feedforward residual paths to stabilize the training process and refine information scheduling. Step 3: Use the old map image dataset obtained in Step 1 to train the HistoMapFormer restoration model; Step 4: After performing geometric correction and resolution unification on the globally degraded old map to be repaired, input it into the trained HistoMapFormer model to directly output the repaired map.

[0010] Furthermore, image preprocessing includes, but is not limited to, geometric correction and image cropping.

[0011] Furthermore, a synthetic global degradation is applied to the ground truth image generated after image preprocessing, including but not limited to yellowing of the background paper, overall fading, color shift, uneven lighting, and global contrast reduction.

[0012] Furthermore, the encoding end is divided into 3 stages, with the specific structure as follows: Stage 1: Stack 16 HistoMapFormer blocks, and output feature map size is 256×256×48; Stage 2: First, reduce the feature map size to 128×128 and increase the feature dimension to 96 by using a patch merging layer, then stack 16 HistoMapFormer blocks; Stage 3: First, reduce the feature map size to 64×64 and increase the feature dimension to 192 by using a patch merging layer, then stack 16 HistoMapFormer blocks; Furthermore, the decoding end is divided into two stages, with the following specific structure: Stage 4: First, the feature map size is increased to 128×128 and the feature dimension is reduced to 96 through the patch separation layer. After being fused with the Stage 2 features of the encoder, 8 HistoMapFormer blocks are stacked. Stage 5: First, the feature map size is increased to 256×256 and the feature dimension is reduced to 48 through the patch separation layer. After being fused with the Stage 1 features of the encoder, 8 HistoMapFormer blocks are stacked. Output layer: The feature map is converted to 256×256×3 through the patch restoration layer, and the final result is output after the residual post-processing module.

[0013] Furthermore, the structure of the HistoMapFormer block is designed as follows: The input features are first processed by the first residual layer for normalization.

[0014] The input features are fed into a hybrid attention module. The output of the hybrid attention module and the output of the first Rescale / Rebias gate are concatenated along the channel dimension and then projected onto the input dimension via a 1×1 convolution. Finally, the residuals of the input features are added to obtain the intermediate features. The intermediate features are normalized by the second residual layer and then fed into the convolutional feedforward module. The output of the second residual layer enters the second Rescale / Rebias gate, and is then added to the output of the convolutional feedforward module to obtain the final output of the block.

[0015] Furthermore, the hybrid attention module adopts a sliding window-based self-attention mechanism, including: the window attention branch uses an 8×8 window size and 8 attention heads; the depthwise separable convolution branch uses a 3×3 convolution kernel.

[0016] Furthermore, the SK Fusion module structure is designed as follows: Perform a 1×1 convolutional linear projection on the skip features to make their dimensions consistent with the backbone features; Perform channel-wise global average pooling on the two features respectively, and then sum them to obtain the global description vector; After sequentially performing 1×1 convolution for dimensionality reduction, ReLU, and 1×1 convolution to increase the dimensionality back to the original dimension, two sets of fused weights are generated. After softmax normalization, the two features are summed channel by channel to obtain the fused features.

[0017] Furthermore, the decoding end uses deconvolution instead of bilinear interpolation for upsampling.

[0018] A global degradation repair system for old maps based on Swin Transformer, comprising: Data preprocessing module: performs geometric correction, image cropping, degradation synthesis, and dataset partitioning of scanned old maps; Model building module: Builds and initializes HistoMapFormer to repair the model; Model training module: performs model training, parameter updates, and saves optimal weights; Inference and Repair Module: Performs batch processing on globally degraded old maps to be repaired and outputs the repair results; Results evaluation module: Quantitatively evaluates the repair effect using indicators such as MSE, MAE, RMSE, and PSNR. Beneficial effects

[0019] Significantly improved restoration accuracy: In the task of global degradation restoration of old maps, the MSE index of this invention reaches 0.00002999, which is 78.71% lower than the existing best Transformer model, and the PSNR is improved to 46.57dB, which can restore the linear features, text annotations and color information of the map with high fidelity.

[0020] Strong generalization ability: It can adapt to various cartographic styles such as traditional maps, landscape paintings, mileage charts, and oil paintings. In cross-style tests, MSE is reduced by 22.35%-30.41% respectively. In the restoration of old maps in different cities such as Guangzhou, Hong Kong, and Tokyo, PSNR is consistently above 44.67dB and SSIM reaches above 0.99.

[0021] High computational efficiency: It adopts a sliding window attention mechanism to achieve near-linear computational complexity, supports ultra-high resolution map inference under standard video memory budget, and the repair time of a single 4K resolution map is less than 10 seconds, meeting the needs of batch processing of large-scale museum collection maps.

[0022] Meeting GIS application requirements: The restored map has significantly improved line network continuity, annotation recognition, and color uniformity, which can effectively reduce systematic errors in subsequent geographic registration, feature vectorization, and place name recognition, providing a high-quality data foundation for historical geography research.

[0023] Non-destructive restoration: The entire process is digitally processed, which completely avoids physical damage to the original old map and can preserve the original map, restoration map and restoration parameters, so as to make the restoration process traceable and verifiable. Attached Figure Description

[0024] Figure 1 is an overall flowchart of the method for global deterioration repair of old maps according to the present invention.

[0025] Figure 2 is an overall architecture diagram of the HistoMapFormer model of this invention.

[0026] Figure 3 is a schematic diagram of the internal structure of the HistoMapFormer block of the present invention.

[0027] Figure 4 is a schematic diagram of the hybrid attention module of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0029] The core of this invention is a method for restoring ancient maps with global degradation across multiple styles, based on the HistoMapFormer restoration model. The specific steps for restoring global degradation in ancient maps are as follows: Step 1: Obtain old map image data, and after image preprocessing and degradation processing, form an old map image dataset.

[0030] 1.1 Collect complete scanned copies of old maps as raw data, and use the control point matching method for geometric correction to eliminate perspective distortion and rotational offset generated during the scanning process.

[0031] 1.2 The corrected images are uniformly cropped into non-overlapping image patches of 256×256 pixels to generate a ground truth dataset; 1.3 Apply synthetic global degradation to each ground truth image, including yellowing of the background paper, overall fading, color shift, and global contrast reduction, to generate paired datasets.

[0032] 1.4 The dataset is randomly divided into training, validation and test sets in a ratio of 8:1:1.

[0033] Step 2: Using the Swin Transformer hierarchical window self-attention mechanism as the backbone, construct the HistoMapFormer repair model. The HistoMapFormer repair model presents a U-shaped encoding / decoding architecture. (Reference) Figure 2 The network structure of the model is as follows: Encoding end: Stage 1: Stack 16 HistoMapFormer blocks, and output feature map size is 256×256×48; Stage 2: First, reduce the feature map size to 128×128 and increase the feature dimension to 96 by using a patch merging layer, then stack 16 HistoMapFormer blocks; Stage 3: First, reduce the feature map size to 64×64 and increase the feature dimension to 192 by using a patch merging layer, then stack 16 HistoMapFormer blocks; Decoding end: Stage 4: First, the feature map size is increased to 128×128 and the feature dimension is reduced to 96 through the patch separation layer. After being fused with the Stage 2 features of the encoder, 8 HistoMapFormer blocks are stacked. Stage 5: First, the feature map size is increased to 256×256 and the feature dimension is reduced to 48 through the patch separation layer. After being fused with the Stage 1 features of the encoder, 8 HistoMapFormer blocks are stacked. Output layer: The feature map is converted to 256×256×3 through the patch restoration layer, and the final result is output after residual post-processing module. Patch merging layer is a step that operates on the patch, not an architecture module, so it is not shown in the diagram, but it exists before each stacked HistoMapFormer.

[0034] More specifically, the HistoMapFormer block employs a "pre-normalization—hybrid attention—convolutional feedforward—residual gating" structure. The hybrid attention module executes windowed multi-head self-attention and depthwise separable convolutions in parallel, capturing long-range dependencies and local spatial priors respectively. Rescale / Rebias gating is introduced on the attention and feedforward residual paths to stabilize the training process and refine information scheduling. (See reference...) Figure 3The HistoMapFormer block has the following structure: The input features are first processed by the first residual layer normalization (RLN).

[0035] Feed into the Hybrid Attention Module: The window attention branch uses an 8×8 window size with 8 attention heads; the depthwise separable convolution (DWConv) branch uses a 3×3 convolution kernel with a stride of 1 and padding of 1.

[0036] The output of the hybrid attention module and the output of the first Rescale / Rebias gate are concatenated along the channel dimension and then projected onto the input dimension via a 1×1 convolution. Finally, the residuals of the concatenated features are added to the input features to obtain the intermediate features.

[0037] The intermediate features are processed by the second residual normalization (RLN) layer and then fed into the convolutional feedforward module (MLP): it consists of two 3×3 convolutional layers with a ReLU activation function inserted in between, and the hidden layer dimension is 4 times the input dimension.

[0038] The output of the second RLN is gated by the second Rescale / Rebias gate and added to the output of the convolutional feedforward module to obtain the final output of the block.

[0039] like Figure 4 As shown, the hybrid attention module employs a sliding window-based self-attention mechanism. Input features are first pre-normalized, then window shifting is performed to break the boundary constraints of fixed blocks, dividing the module into non-overlapping windows of fixed size. Within each window, Q / K / V are generated through linear mapping and split and rearranged in a multi-head manner, calculating correlations only locally within the window to maintain near-linear computational overhead. To explicitly encode geometric priors, a meta-network with relative positional biases is introduced: taking relative coordinates within the window as input, a bias consistent with geometric relationships is generated to modulate content-similarity-based attention weights. After softmax normalization, the V branch is weighted and aggregated to obtain a window-level contextual representation. Finally, window inversion operations are performed on the results of all windows to restore the coordinate system to the full-image coordinate system and align with the shift, enabling cross-window information flow. This mechanism organically combines local modeling, spatial priors, and computational efficiency without changing resolution or tensor shape, resulting in an attention representation that is both expressive and scalable.

[0040] The SK Fusion module is used to adaptively weight and fuse backbone features and skip features, dynamically adjusting the inflow of high-frequency information to enhance the restoration of details such as linear features and text strokes. The structure of the SK Fusion module is as follows: Perform a 1×1 convolutional linear projection on the skip features to make their dimensions consistent with the backbone features; Perform channel-wise global average pooling on the two features respectively, and then sum them to obtain the global description vector; Two sets of fused weights are generated through "1×1 convolution (dimension reduction to 1 / 16) — ReLU — 1×1 convolution (dimension increase to the original dimension)"; After softmax normalization, the two features are summed channel by channel to obtain the fused features; The decoder upsamples step-by-step along the U-Net path to restore spatial resolution, and performs skip-connection fusion with the features from the encoder at the alignment scale to ensure high-resolution detail return and topology consistency; the specific design of the decoder is as follows: Each upsampling stage first uses bilinear interpolation to double the feature map size, and then uses 1×1 convolution to halve the feature dimension; Features corresponding to the scale at the encoding end are fused by the SK Fusion module and then stacked into two HistoMapFormer blocks for feature transformation. Finally, a 1×1 convolution is used to map the feature dimension to 3, outputting the repaired RGB image.

[0041] Step 3: Using the old map image dataset obtained in Step 1, train the HistoMapFormer restoration model constructed in Step 2. 3.1 Hardware Environment: Training was performed using an NVIDIA A800 PCIe 80GB GPU; 3.2 Training parameter settings: initial learning rate is 1e-4, weight decay is 1e-4, batch size is 32, and number of training epochs is 100; 3.3 Optimization Strategy: The AdamW optimizer is used with MSE as the loss function; the ReduceLROnPlateau scheduler is used, and the learning rate is reduced to 0.5 times the original value when the PSNR of the validation set does not improve for 10 consecutive cycles. 3.4 Save the model weight with the highest PSNR on the validation set as the final inference model.

[0042] Step 4: Model Inference After performing geometric correction and resolution unification on the globally degraded old map to be repaired, the repaired map is directly output by inputting it into the trained HistoMapFormer model.

[0043] The difference between this variant and the preferred embodiment is that the patch size is set to 8×8, the output dimension of the linear embedding layer is 192, the window size of the window attention is set to 16×16, and the number of attention heads is 12. This variant is suitable for the restoration of old maps with a resolution of 512×512.

[0044] The difference between this variant and the preferred embodiment is that deconvolution is used instead of bilinear interpolation for upsampling at the decoding end; and a weighted combination of MSE and SSIM losses is used as the loss function. This variant can further improve the visual quality of the restored image.

[0045] To implement the above method, this invention provides a global degradation repair system for old maps based on Swing Transformer, comprising: Data preprocessing module: used to complete geometric correction, image cropping, degradation synthesis, and dataset partitioning of old map scans; Model building module: Used to build and initialize the HistoMapFormer repair model; Model training module: used to perform model training, parameter updates, and save optimal weights; Inference Repair Module: Used to batch process globally degraded and old maps to be repaired and output the repair results; Results evaluation module: Used to quantitatively evaluate the repair effect through indicators such as MSE, MAE, RMSE, PSNR, etc.

[0046] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for restoring ancient maps with global degradation across multiple styles, characterized in that, Includes the following steps: Step 1: Obtain old map image data, and after image preprocessing and degradation processing, form an old map image dataset; Step 2: Using the Swin Transformer architecture as the backbone, construct the HistoMapFormer repair model. The model is a U-shaped encoding / decoding architecture, where: Encoding end: The input image is divided into 4×4 non-overlapping image blocks by patching, and projected to the set dimension through a linear embedding layer; multiple HistoMapFormer blocks are stacked for feature transformation, and the feature map resolution is gradually reduced by merging the patches to form a multi-scale hierarchical representation; Decoding end: Spatial resolution is restored by upsampling along the U-Net path. In terms of alignment scale, the backbone features and skip connection features of the SK Fusion module and the HistoMapFormer block of the encoding end are adaptively weighted and fused. The HistoMapFormer block introduces a hybrid attention module to execute window multi-head self-attention and depthwise separable convolution in parallel, capturing long-range dependencies and local spatial priors respectively; Rescale / Rebias gating is introduced on the attention and feedforward residual paths to stabilize the training process and refine information scheduling. Step 3: Use the old map image dataset obtained in Step 1 to train the HistoMapFormer restoration model; Step 4: After performing geometric correction and resolution unification on the globally degraded old map to be repaired, input it into the trained HistoMapFormer model to directly output the repaired map.

2. The method for restoring ancient maps with global degradation across multiple styles according to claim 1, characterized in that, Image preprocessing includes, but is not limited to, geometric correction and image cropping.

3. The method for restoring ancient maps with global degradation across multiple styles according to claim 1, characterized in that, The generated ground truth image after image preprocessing is subjected to a synthetic global degradation, including but not limited to yellowing of the background paper, overall fading, color shift, uneven lighting, and global contrast reduction.

4. The method for restoring ancient maps with global degradation across multiple styles according to claim 1, characterized in that, The encoding end is divided into 3 stages, with the following specific structure: Stage 1: Stack 16 HistoMapFormer blocks, and output feature map size is 256×256×48; Stage 2: First, reduce the feature map size to 128×128 and increase the feature dimension to 96 by using a patch merging layer, then stack 16 HistoMapFormer blocks; Stage 3: First, reduce the feature map size to 64×64 and increase the feature dimension to 192 by patch merging layer, then stack 16 HistoMapFormer blocks.

5. A method for restoring ancient maps with global degradation across multiple styles according to claim 4, characterized in that, The decoding end is divided into two stages, with the following specific structure: Stage 4: First, the feature map size is increased to 128×128 and the feature dimension is reduced to 96 through the patch separation layer. After being fused with the Stage 2 features of the encoder, 8 HistoMapFormer blocks are stacked. Stage 5: First, the feature map size is increased to 256×256 and the feature dimension is reduced to 48 through the patch separation layer. After being fused with the Stage 1 features of the encoder, 8 HistoMapFormer blocks are stacked. Output layer: The feature map is converted to 256×256×3 through the patch restoration layer, and the final result is output after the residual post-processing module.

6. A method for restoring ancient maps with global degradation across multiple styles according to claim 1, characterized in that, The HistoMapFormer block is structured as follows: The input features are first processed by the first residual layer for normalization. The input features are fed into a hybrid attention module. The output of the hybrid attention module and the output of the first Rescale / Rebias gate are concatenated along the channel dimension and then projected onto the input dimension via a 1×1 convolution. Finally, the residuals of the input features are added to obtain the intermediate features. The intermediate features are normalized by the second residual layer and then fed into the convolutional feedforward module. The output of the second residual layer enters the second Rescale / Rebias gate, and is then added to the output of the convolutional feedforward module to obtain the final output of the block.

7. A method for restoring ancient maps with global degradation across multiple styles according to claim 6, characterized in that, The hybrid attention module adopts a sliding window-based self-attention mechanism, including: the window attention branch uses an 8×8 window size and 8 attention heads; the depthwise separable convolution branch uses a 3×3 convolution kernel with a stride of 1 and padding of 1.

8. A method for restoring ancient maps with global degradation across multiple styles according to claim 1, characterized in that, The SK Fusion module structure is designed as follows: Perform a 1×1 convolutional linear projection on the skip features to make their dimensions consistent with the backbone features; Perform channel-wise global average pooling on the two features respectively, and then sum them to obtain the global description vector; After sequentially performing 1×1 convolution for dimensionality reduction, ReLU, and 1×1 convolution to increase the dimensionality back to the original dimension, two sets of fused weights are generated. After softmax normalization, the two features are summed channel by channel to obtain the fused features.

9. A method for restoring ancient maps with global degradation across multiple styles according to claim 1, characterized in that, The decoding end uses deconvolution instead of bilinear interpolation for upsampling.

10. A global degradation repair system for old maps based on Swing Transformer, characterized in that, include: Data preprocessing module: performs geometric correction, image cropping, degradation synthesis, and dataset partitioning of scanned old maps; Model building module: Builds and initializes HistoMapFormer to repair the model; Model training module: performs model training, parameter updates, and saves optimal weights; Inference and Repair Module: Performs batch processing on globally degraded old maps to be repaired and outputs the repair results; Results evaluation module: Quantitatively evaluates the repair effect using indicators such as MSE, MAE, RMSE, and PSNR.

Images