Ancient mural element detection algorithm based on context semantics and adaptive augmentation

Abstract

The invention discloses an ancient fresco element detection algorithm based on context semantics and adaptive augmentation, and belongs to the cross technical field of computer vision and cultural heritage protection. Comprising the following steps: constructing a Chinese ancient mural element target detection data set; designing an adaptive random erasure image augmentation model, and augmenting the constructed data set; constructing a lightweight multi-scale feature extraction backbone network, and performing feature extraction on the augmented image; constructing a semantic feature enhancement model, and performing semantic enhancement on the extracted features; the obtained enhanced semantic feature map is input into the full convolutional network, a detection result is output, the detection result comprises element categories, bounding box coordinates and confidence coefficients, experiments show that the detection precision of the method on a self-built data set reaches 87.5%, the single-map detection time is only 0.022 seconds, the mural elements can be efficiently and accurately recognized, and the method is suitable for popularization and application. And technical support is provided for ancient mural protection, historical research and digital inheritance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision technology and digital protection of cultural heritage, and particularly relates to an ancient mural element detection algorithm based on context semantics and adaptive augmentation. BACKGROUND

[0002] The ancient murals unearthed in China span over 2200 years from the Qin and Han Dynasties to the Ming and Qing Dynasties, and are widely distributed in more than 500 tombs and sites in 23 provinces and cities such as Henan, Shaanxi and Shanxi, covering royal life, myths and legends, natural landscapes, etc., and are important physical materials for studying ancient society, politics, economy and culture. However, due to factors such as natural weathering and human damage, more than 80% of the ancient murals are damaged to varying degrees. For example, the facial features of some figures in the murals of Mogao Grottoes in Dunhuang are blurred due to wind and sand erosion, and the pigments of the murals in the ancient tombs in Luoyang, Henan, are peeling off due to humidity changes. Traditional mural element research relies on manual recording, and a professional archaeologist can only complete the element annotation of 5-8 murals per day, and the annotation results are influenced by personal experience and lack of consistency.

[0003] With the development of digital technology, the demand for mural virtual restoration and digital exhibition is increasingly urgent, and element detection is a prerequisite for the above applications. For example, in mural disease analysis, the positions of elements such as figures and flowers need to be accurately identified to determine whether the missing area affects the key historical information; in digital exhibition, element detection is needed to realize interactive explanation, such as clicking on the “cloud” element in the mural to pop up the corresponding religious and cultural background of the historical period. Therefore, developing an efficient and accurate mural element detection algorithm has become a core technical requirement for promoting the digital protection of ancient murals.

[0004] Target detection technology based on computer vision has been preliminarily applied in the field of cultural relics. Existing technologies mainly include traditional machine learning methods and deep learning methods. Traditional machine learning methods such as HOG+ SVM and Haar-like features+ Adaboost realize target recognition through hand-designed feature extractors, but are limited by the feature expression ability and have a detection accuracy of less than 50% in complex mural backgrounds and missing scenes, and have gradually been eliminated.

[0005] Deep learning methods have become mainstream due to their automatic feature extraction capabilities, with representative algorithms including the YOLO series, Faster R-CNN, and SSD. In the field of cultural relic inspection, existing research mainly focuses on surface defect detection of three-dimensional artifacts such as bronzes and porcelains, with limited research on ancient murals. For example, the MuralDiff algorithm proposes a method for mural crack detection and virtual restoration based on U-net and a diffusion model, but it only focuses on crack defects and does not address element identification; the multi-scale adaptive convolution algorithm proposed by Wang et al. for virtual restoration of Thangka murals also fails to solve the element detection problem; the DunHuangStitch algorithm constructs a Dunhuang mural splicing dataset, but only achieves image splicing and cannot identify specific elements in the spliced ​​murals.

[0006] Existing object detection datasets such as COCO and VOC are mainly geared towards natural scenes and lack ancient mural samples. A few mural-related datasets, such as the Dunhuang mural dataset, contain fewer than 500 images and only label them with coarse-grained categories like "human figures" and "animals," which cannot meet the needs of fine-grained element detection. Furthermore, existing datasets do not cover the stylistic differences in murals from different historical periods, such as the simple lines of Qin and Han dynasty murals versus the delicate colors of Ming and Qing dynasty murals, resulting in poor generalization ability of models in cross-period mural detection.

[0007] Conventional data augmentation methods, such as random flipping, cropping, and color dithering, can only simulate geometric and color changes in images and cannot simulate the unique peeling and damage scenarios of murals. Existing random erasing methods, such as Random Erasing, randomly select image areas for erasing, which easily erases key features of the target (such as human faces), causing the model to learn incomplete features, resulting in a 15%-20% decrease in accuracy in actual mural damage detection.

[0008] Existing high-precision target detection models, such as Faster R-CNN, have more than 300 million parameters and over 1000 million FLOPs. When running on embedded devices (such as mural on-site detection terminals), the single-image detection time exceeds 0.5 seconds, which cannot meet the requirements of real-time detection. While lightweight models such as YOLOv5-nano reduce the amount of computation, they use single-scale feature extraction and cannot adapt to the size difference between "auspicious clouds" (small targets) and "chariots and horses" (large targets) in murals. The recall rate for small target detection is less than 60%.

[0009] Mural elements exhibit strong semantic relationships; for example, "auspicious clouds" often appear alongside "figures," and "flowers" are frequently found against a "mountain and river" background. Existing models rely solely on local features for detection, failing to utilize the contextual semantic information between elements, leading to a high false detection rate in complex backgrounds. For instance, misdetecting "rock textures" in a mural as "flowers" results in a false detection rate exceeding 25%.

[0010] Ancient murals, as precious world cultural heritage, record key information such as social life, religious beliefs, and artistic styles from different historical periods, serving as important visual materials for the study of ancient civilizations. However, due to their age, most ancient murals suffer from fading, peeling, and damage. Traditional manual identification methods are inefficient, subjective, and prone to causing physical damage. With the development of computer vision technology, deep learning-based object detection algorithms have provided new avenues for mural element detection, but existing technologies still face three core challenges: First, obtaining mural datasets is difficult; existing datasets are small in scale and limited in categories, failing to cover stylistic differences in murals from different historical periods and regions. Second, mural damage leads to incomplete target features; conventional data augmentation methods easily destroy key target features, reducing model generalization ability. Third, mural elements vary greatly in size and have complex backgrounds; existing models are either computationally expensive and difficult to deploy, or suffer from insufficient semantic feature extraction, resulting in low detection accuracy for small and occluded targets. In summary, existing technologies cannot meet the demand for efficient and accurate detection of ancient mural elements. There is an urgent need for innovation in four dimensions: dataset construction, data augmentation, feature extraction, and semantic enhancement, to develop specialized detection algorithms suitable for ancient mural scenarios. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an ancient mural element detection algorithm based on contextual semantics and adaptive augmentation. This algorithm addresses problems such as scarce datasets, poor robustness to missing scenes, high computational load of models, and insufficient utilization of semantic associations in ancient mural element detection. It achieves efficient and accurate detection of six types of elements in murals: male, female, beast, flower, bird, and cloud, providing technical support for the digital preservation and historical research of ancient murals.

[0012] To achieve the above objectives, the present invention provides the following technical solution: an ancient mural element detection algorithm based on contextual semantics and adaptive augmentation, comprising the following steps:

[0013] S1. Construct a dataset for object detection of elements in ancient Chinese murals;

[0014] S2. Design an adaptive random erasure image augmentation model to augment the dataset constructed in step S1;

[0015] S3. Construct a lightweight multi-scale feature extraction backbone network to extract features from the augmented image in step S2;

[0016] S4. Construct a semantic feature enhancement model to semantically enhance the features extracted in step S3;

[0017] S5. Input the enhanced semantic feature map obtained in step S4 into a fully convolutional network and output the detection results, which include element category, bounding box coordinates and confidence score.

[0018] Preferably, step S1 specifically includes the following steps:

[0019] S11. Data Acquisition: Collect images of ancient murals from tombs and sites, covering five historical periods: Qin and Han, Wei, Jin and Northern and Southern Dynasties, Sui and Tang, Song and Yuan, and Ming and Qing Dynasties. A total of 2,358 mural images were collected. After white balance correction and noise reduction of the original images, the image resolution was uniformly adjusted to 1024×1024 pixels.

[0020] S12. Sample annotation: The LabelImg annotation tool was used, and three professionals with archaeological backgrounds annotated the elements in the image. The annotation categories included male, female, beast, flower, bird, and cloud. The annotation format was PASCAL VOC, and the target bounding box coordinates were recorded as xmin, ymin, xmax, and ymax.

[0021] S13. Labeling Verification and Correction: Cross-validation was used to calculate the Kappa coefficient of the labeling results of the three labelers. When the Kappa coefficient was ≥0.85, the labeling results were considered consistent. When the Kappa coefficient was <0.85, a fourth senior archaeological expert was used for arbitration and correction, resulting in 23,852 valid labeled samples.

[0022] Preferably, the constraints of the adaptive random erasure image augmentation model in step S2 include:

[0023] Constraint 1: Define IOUSrand as the ratio of the intersection of the area of ​​the random erasure region and the area of ​​the target mark box to the area of ​​the erasure region, satisfying 0≤IOUSrand≤α, where α∈[0.3, 0.7];

[0024] Constraint 2: Define IOUSbound as the ratio of the intersection of the area of ​​the random erasure region and the area of ​​the target marker box to the area of ​​the marker box, satisfying 0≤IOUSbound≤β, where β∈[0, 0.3];

[0025] Constraint 3: Define Rrand as the aspect ratio of the erased region and Rbound as the aspect ratio of the target bounding box, satisfying Rbound×(1+γ)≤Rrand≤Rbound×(1+δ), where γ∈[-0.1, 0] and δ∈[0, 0.1].

[0026] Preferably, the specific process of generating the random erasure region in step S2 is as follows: randomly select a point P (Px, Py) within the target marking box, and extend it to the left by a distance a, to the right by a distance b, upward by a distance d, and downward by a distance c, forming a rectangular erasure region. Calculate the IOUSrand, IOUSbound, and Rrand of this region. If the three constraints in step S2 are satisfied, the erasure region is retained; otherwise, point P is reselected until an erasure region that satisfies the conditions is generated.

[0027] Preferably, the lightweight multi-scale feature extraction backbone network in step S3 includes a Stem block and a Stage block, wherein: the Stem block includes Stem block-1 and Stage block-2. One path is processed by a 2×2 convolution (stride=1, number of channels=32), and the other path is processed by a 1×1 convolution (stride=1, number of channels=32) and a 3×3 convolution (stride=1, number of channels=32) in sequence. After adjusting the feature maps of the two paths to a uniform size, Eltwise fusion is performed, and then a 1×1 convolution (stride = 1, number of channels=32) is performed.

[0028] Stem block-2 employs a two-way feature extraction and concat fusion approach. Specifically, the output feature map of Stem block-1 is processed through a 3×3 convolution (stride=1, number of channels=32), then divided into two paths. One path undergoes 2×2 max pooling (stride=2, number of channels=32), while the other path undergoes a 1×1 convolution (stride=1, number of channels=32) followed by a 3×3 convolution (stride=2, number of channels=32). The two feature maps are then concat fused and further processed through a 1×1 convolution (stride=1, number of channels=64).

[0029] The stage block contains five cascaded sub-modules. Each sub-module adopts a three-way feature extraction structure, with the number of channels increasing sequentially to 16, 16, 16, and 32, and is residually connected to the input feature map.

[0030] Preferably, the sub-module processing procedure of the Stage block is as follows:

[0031] Taking Stage 1-1 as an example, the input feature map is processed in three ways. Branch 1 goes through a 1×1 convolution (stride=1, number of channels is the same as the input feature map) and a 3×3 convolution (stride=1, number of channels is the same as the input feature map) in sequence. Branch 2 goes through 2×2 max pooling (stride=1, number of channels is the same as the input feature map). Branch 3 goes through a 1×1 convolution (stride=1, number of channels is the same as the input feature map), a 3×3 convolution (stride=1, number of channels is the same as the input feature map), and a 1×1 convolution (stride=1, number of channels is the same as the input feature map) in sequence.

[0032] The three feature maps are fused using Concat, and the number of channels in the input feature map is adjusted to match the number of channels in the fused feature map using 1×1 convolution. Then, residual connections are performed to obtain the output feature map of the submodule.

[0033] Preferably, the semantic feature enhancement model in step S4 includes a context information fusion module and a residual attention module, wherein:

[0034] The context information fusion module processes the output feature maps of the five sub-blocks in the Stage block. The output feature maps of the first three sub-blocks are first subjected to 3×3 max pooling (stride = 2, number of channels unchanged), and then subjected to 1×1 convolution (stride = 1, number of channels = 512).

[0035] The output feature maps of the last two sub-blocks are directly passed through a 1×1 convolution (stride=1, number of channels=512).

[0036] After adjusting the five processed feature maps to a uniform size, they are fused using Concat and then subjected to a 1×1 convolution (stride=1, number of channels=512) to obtain the context semantic feature map.

[0037] The residual attention module processes the feature map output by the backbone network through a residual convolutional block, and then performs a 3×3 convolution (stride=2, number of channels=256) and a 2×2 max pooling (stride = 2, number of channels=256) to obtain the attention weight map.

[0038] The attention weight map is upsampled to the same size as the context semantic feature map through bilinear interpolation. After adjusting the number of channels using 1×1 convolution, it is multiplied element-wise with the context semantic feature map to obtain the enhanced semantic feature map.

[0039] Preferably, the specific process of target detection and prediction in step S5 is as follows:

[0040] A 3×3 convolutional layer (6×1 channels) is used to output the category score, and the category probability is obtained by passing the Softmax function to determine the element category;

[0041] A 3×3 convolutional layer (6×4 channels) is used to output the bounding box offset, and the final bounding box coordinates are calculated by combining 9 preset anchor boxes (3 scales and 3 aspect ratios).

[0042] The predicted bounding boxes are processed by NMS, and the IOU threshold is set to 0.5 to remove overlapping bounding boxes and retain the bounding box with the highest confidence.

[0043] Filter prediction results with a confidence level < 0.5 and output the element category, bounding box coordinates, and confidence level.

[0044] Preferably, in step S2, each original sample generates 5 augmented samples with different erasing regions, expanding the dataset size to 143,112 samples, which are divided into a training set (114,489 samples), a validation set (14,311 samples), and a test set (14,312 samples) in an 8:1:1 ratio.

[0045] Preferably, the training parameters of the lightweight multi-scale feature extraction backbone network are set as follows: using the AdamW optimizer, with an initial learning rate of 1e-4 and a weight decay of 5e-4.

[0046] Cosine annealing learning rate scheduling is used, T_max = 100, eta_min = 1e-6;

[0047] We use a weighted sum of cross-entropy loss and smoothing L1 loss, with a weight ratio of 1:1;

[0048] The batch size is 16;

[0049] The training consists of 100 rounds, employing an early stopping strategy. When the validation set mAP shows no improvement for 10 consecutive rounds, training is stopped and the optimal model is saved.

[0050] Beneficial effects:

[0051] (1) The dataset constructed in this invention covers more than 2,200 years of history and 577 sites, containing 2,358 images and 23,852 labeled samples. It is the largest and most diverse dataset of ancient mural elements to date. Compared with the existing Dunhuang mural dataset (500 images, 2-class annotation), this dataset has 4.7 times more samples and 3 times more categories, and covers mural styles from different historical periods, providing data support for improving the model's generalization ability.

[0052] (2) To address the problem that conventional random erasing can easily damage key features of the target, this invention controls the erasing area through triple constraints (Intersection over Union 1, Intersection over Union 2, and shape similarity) to ensure that the erasing area contains contextual information to improve model robustness without damaging key features of the target. Experiments show that this augmentation method can improve the accuracy of the model in detecting missing murals by 5.9%, which is significantly better than the conventional Random Erasing method (accuracy improvement of 2.1%).

[0053] (3) By adopting a hierarchical structure of Stem blocks and Stage blocks, and combining Eltwise and Concat fusion methods, multi-scale feature extraction is achieved while reducing computational cost. Compared with YOLOv5-nano (1.9M parameters, 4.5G FLOPs), the backbone network of this invention has reduced the number of parameters to 1.1M and the FLOPs to 2.3G, reducing computational cost by 47.1%, and improving multi-scale feature extraction capability. The recall rate for detecting small targets (such as auspicious clouds) has increased from 60% to 78%.

[0054] (4) The context information fusion module and the residual attention module are integrated to improve detection accuracy by utilizing the semantic relationships between elements (such as auspicious clouds and figures appearing together). The context information fusion module integrates features from different levels to capture global semantics; the residual attention module focuses on the target region and suppresses background interference. Experiments show that this module can reduce the false detection rate of the model from 25% to 12% and improve the detection accuracy by 2.3%.

[0055] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of the phrase "embodiment" in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0058] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, "connection" or "joining" in mechanical structures can refer to a physical connection, such as a fixed connection, for example, a connection fixed by fasteners, such as a connection fixed by screws, bolts, or other fasteners; a physical connection can also be a detachable connection, such as a snap-fit ​​or interlocking connection; a physical connection can also be an integral connection, such as a connection formed by welding, bonding, or integral molding. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0059] The experimental environment configuration for this invention is as follows:

[0060] Hardware environment: CPU is Intel Core i9-11900k (3.5GHz), GPU is NVIDIA RTX 4090 (24GB VRAM), memory is 64GB (DDR5 4800MHz), and storage is 2TB SSD;

[0061] Software environment: Operating system is Ubuntu 16.04LTS, deep learning framework is PyTorch 1.12.0, CUDA version is 11.6, CuDNN version is 8.4.0, programming language is Python 3.8, and dependent libraries include OpenCV 4.6.0, NumPy 1.23.5, Matplotlib 3.6.2, and LabelImg 1.8.6.

[0062] Using professional image acquisition equipment (such as a Canon EOS R5 camera with a 24-70mm F2.8 lens), and accompanied by staff from cultural heritage protection units, we photographed the murals at 577 sites. During the shooting, we used a fixed focal length (50mm), a fixed aperture (F8), and a fixed shutter speed (1 / 125s) to ensure consistent image exposure.

[0063] The acquired raw images were preprocessed as follows: white balance correction was performed using Adobe Lightroom to remove the influence of ambient light; noise reduction was performed using OpenCV, and Gaussian filtering (kernel size = 3×3) was used to remove image noise; the image resolution was uniformly adjusted to 1024×1024 pixels, and saved in JPEG format with a compression quality factor of 95.

[0064] Assemble an annotation team: Annotators must have a background in archaeology or art history and undergo a week of training to be familiar with the definitions and annotation standards of the six categories of elements (e.g., male is defined as a figure in a mural that is clearly male, including emperors, soldiers, and commoners).

[0065] Label tool configuration: Set the label format of the LabelImg tool to PASCAL VOC, and retain the bounding box coordinate precision to integer pixels;

[0066] Labeling process: Each labeler independently labeled 1 / 3 of the image. After labeling, a Python script was used to calculate the Kappa coefficient of the three labelers. For images with a Kappa coefficient < 0.85, a senior archaeological expert (with more than 10 years of experience in ancient mural research) arbitrated the data and corrected the bounding box coordinates or category labels. Finally, 23,852 valid labeled samples were obtained, including 5,236 males, 4,892 females, 3,784 beasts, 3,256 flowers, 2,988 birds, and 3,696 clouds.

[0067] The erasure region generation algorithm is implemented using Python. It reads the dataset constructed in step S1 and iterates through the image and annotation file of each sample.

[0068] For each sample, the generate_erase_region function is called to generate 5 different erasure regions;

[0069] For each erased area, set the pixel values ​​within the area to black (RGB = (0, 0, 0)) to generate an augmented image;

[0070] Save the augmented images and their corresponding annotation files (the annotation files are the same as the original samples, since only the augmented images are used and the target locations remain unchanged), and finally obtain 143,112 augmented samples. The dataset is divided into a training set (80%, 114,489 samples), a validation set (10%, 14,311 samples), and a test set (10%, 14,312 samples).

[0071] The backbone network is implemented using the PyTorch framework. The optimizer is AdamW, with an initial learning rate of 1e-4 and a weight decay of 5e-4.

[0072] Learning rate scheduling: Cosine annealing learning rate scheduling is used, T_max = 100, eta_min = 1e-6;

[0073] Loss function: The weighted sum of cross-entropy loss (classification loss) and smoothed L1 loss (regression loss) is used, with a weight ratio of 1:1;

[0074] Batch size: 16 (supports 24GB GPU memory);

[0075] Training rounds: 100 rounds, using an early stopping strategy: training stops when the validation set mAP shows no improvement for 10 consecutive rounds.

[0076] The semantically enhanced feature map is then input into a fully convolutional network. The specific steps are as follows:

[0077] Classification prediction: A 3×3 convolutional layer (number of channels = 6×1, where 6 is the number of categories) is used to output category scores, and the category probabilities are obtained by passing the Softmax function;

[0078] Bounding box regression: A 3×3 convolutional layer (number of channels = 6×4, where 4 is the bounding box coordinate offset) is used to output the bounding box offset, and the final bounding box coordinates are calculated by combining the anchor boxes (3 preset sizes and 3 aspect ratios, for a total of 9).

[0079] Non-maximum suppression (NMS): The predicted bounding boxes are processed by NMS with an IOU threshold of 0.5 to remove overlapping bounding boxes and retain the bounding box with the highest confidence.

[0080] Output results: Output the category (male / female / beast / flower / bird / cloud), bounding box coordinates (xmin, ymin, xmax, ymax), and confidence score (0 - 1) for each target.

[0081] Experimental dataset: The target detection dataset of ancient Chinese mural elements constructed using this invention, the test set contains 14312 samples;

[0082] Comparison Algorithms: Five mainstream object detection algorithms were selected for comparison, including YOLOv5-nano, YOLOv5-s, Faster R-CNN, SSD, and EfficientDet-D0;

[0083] Evaluation metrics: Precision, Recall, F1 score, mean precision (mAP), Time per image, and computational cost (FLOPs) are used as evaluation metrics.

[0084] To verify the effectiveness of each module of this invention, an ablation experiment was conducted, and the results are shown in Table 1:

[0085] Model configuration Precision (%) Recall (%) F1 - score mAP (%) FLOPs (G) Single image detection time (s) Base model (without any module) 78.2 76.5 0.773 75.8 4.5 0.035 Base model + adaptive random erasing 82.5 81.3 0.819 81.7 4.5 0.036 Base model + lightweight backbone network 80.1 79.2 0.796 78.9 2.3 0.025 Base model + semantic feature enhancement 83.6 82.1 0.828 82.4 5.1 0.042 Invention (full module) 89.3 87.6 0.884 87.5 2.3 0.022

[0086] Table 1

[0087] As shown in Table 1, the adaptive random erasure module improves mAP by 5.9% (75.8%→81.7%), indicating that this module can effectively simulate mural damage scenarios and improve model robustness; the lightweight backbone network reduces FLOPs by 47.1% (4.5G→2.3G), shortens single-image detection time by 0.01s (0.035s→0.025s), and mAP only decreases by 2.1% (75.8%→78.9%), achieving a balance between computational load and accuracy; the semantic feature enhancement module improves mAP by 6.6% (75.8%→82.4%), indicating that this module can effectively utilize contextual semantic information and improve detection accuracy; when all modules are configured, the present invention achieves mAP of 87.5%, F1 score of 0.884, FLOPs of only 2.3G, and single-image detection time of 0.022s, demonstrating the best overall performance.

[0088] The experimental results comparing this invention with mainstream algorithms are shown in Table 2:

[0089] Algorithm Precision (%) Recall (%) F1 - score mAP (%) FLOPs (G) YOLOv5-nano 80.5 78.3 0.794 78.5 4.5 YOLOv5-s 84.2 82.5 0.833 82.8 17.0 Faster R-CNN 85.6 83.1 0.843 83.5 28.0 SSD 79.3 77.8 0.785 78.2 12.0 EfficientDet-D0 83.8 81.9 0.828 82.3 3.9 Invention 89.3 87.6 0.884 87.5 2.3

[0090] Table 2

[0091] As shown in Table 2:

[0092] This invention ranks first in all four accuracy metrics: precision, recall, F1 score, and mAP. Its mAP is 4.0% higher than that of the second-ranked Faster R-CNN (87.5% → 83.5%), indicating that the detection accuracy of this invention is significantly better than that of mainstream algorithms.

[0093] In terms of computational load and detection speed, the present invention has a FLOPs of only 2.3G, which is 48.9% lower than YOLOv5 - nano (4.5G). The single image detection time is 0.022s, which is 26.7% shorter than EfficientDet - D0 (0.030s), indicating that the present invention has excellent real-time performance while ensuring accuracy.

[0094] The detection results of the six elements by this invention are shown in Table 3:

[0095] Element category Precision (%) Recall (%) F1 - score mAP (%) Male 92.5 91.8 0.921 92.2 Female 91.3 90.5 0.909 90.8 Beast 88.6 87.2 0.879 87.8 Flower 85.2 83.6 0.844 84.2 Bird 84.8 82.9 0.838 83.6 Cloud 86.1 84.3 0.852 84.8

[0096] Table 3

[0097] As shown in Table 3:

[0098] The detection accuracy was highest for the male and female categories (mAP of 92.2% and 90.8%, respectively), because these two categories had a large number of samples (5236 and 4892, respectively) and small differences in intra-class features (such as relatively fixed facial features).

[0099] The detection accuracy for the flower and bird categories was low (mAP 84.2% and 83.6%, respectively), because the number of samples in these two categories was small (3256 and 2988, respectively), and the intra-class features were very different (e.g., the painting styles of flowers in different dynasties were significantly different).

[0100] Overall, the mAP of all six elements exceeded 83%, indicating that the present invention has good detection capabilities for various mural elements.

[0101] To verify the detection capability of this invention in complex scenarios (such as dense elements and severe defects), three typical complex scenarios were selected for testing. Scenario 1 (dense elements): murals from an Eastern Han Dynasty tomb in Luoyang, Henan Province, containing 12 figures, 8 auspicious clouds, and 5 flowers. This invention successfully detected all elements with a confidence level of ≥0.75, with no missed detections or false detections.

[0102] Scenario 2 (Severe damage): A mural in a Tang Dynasty tomb in Xi'an, Shaanxi Province, shows that 40% of the faces of the figures are missing and 30% of the auspicious cloud area is missing. This invention can still accurately detect the figures and auspicious clouds with confidence levels of 0.68 and 0.72, respectively, and a false detection rate of 0.

[0103] Scenario 3 (Dense Small Targets): Northern Wei Dynasty Grottoes murals in Datong, Shanxi Province, contain 20 small birds (size < 50×50 pixels). This invention detected 18 of them, with a recall rate of 90%, which is significantly better than YOLOv5-nano (recall rate 65%).

[0104] The above results demonstrate that the present invention still exhibits excellent detection performance in complex scenarios and can meet the actual needs for detecting elements in ancient murals.

[0105] This invention can be used for automated element statistics and analysis of ancient murals, for example:

[0106] The frequency of the "auspicious cloud" element in murals from different historical periods was statistically analyzed to determine its connection with the religious beliefs of the time.

[0107] By comparing the painting styles of "animal" elements in the murals of different tombs at the same site, the social status of the tomb owners can be determined.

[0108] Compared to manual statistics (an archaeologist counts an average of 5-8 murals per day), this invention can count 1000+ murals per day, increasing efficiency by more than 125 times, and the consistency of the statistical results is over 95%.

[0109] This invention can combine test results to analyze the impact of mural damage on key elements, for example:

[0110] Detect the missing areas of the "figures" elements in the mural and determine whether the missing areas affect key historical information such as the figures' clothing and posture;

[0111] The proportion of missing "floral" elements was statistically analyzed to assess the preservation status of the murals and provide a basis for prioritizing the restoration of damaged areas.

[0112] In a pilot application at the Mogao Grottoes in Dunhuang, this invention helped cultural relics conservationists quickly identify 30 missing areas that affected key historical information, saving two months of manual investigation time for the restoration work.

[0113] This invention can be used for interactive digital exhibitions of ancient murals, for example:

[0114] In the museum's digital mural display case, visitors can click on the detected "chariot and horse" element to see information such as the use and manufacturing process of that type of chariot and horse in the corresponding historical period.

[0115] An online digital exhibition hall for murals has been developed. The detection results of this invention enable the element retrieval function. Viewers can enter "auspicious cloud" to view all mural fragments containing the auspicious cloud element.

[0116] In the "Digital Exhibition of Ancient Murals" at the Henan Museum, the application of this invention increased the number of times visitors could interact by 3 times and increased the average visitor stay time by 15 minutes.

[0117] The detection results of this invention can serve as the basis for virtual restoration of murals, for example:

[0118] Based on the bounding boxes and semantic information of the detected "flower" elements, the system assists restorers in determining the outline and color of the restoration area.

[0119] By combining the characteristics of similar elements from different historical periods, we can provide style references for restorers and ensure that the restoration results conform to the original historical appearance.

[0120] In the restoration project of Tang Dynasty murals at the Shaanxi History Museum, the test results of this invention provided the restorers with accurate element outlines and style references, shortening the restoration cycle by 15% and achieving a 92% satisfaction rate for the restoration results.

[0121] This invention proposes an ancient mural element detection algorithm based on contextual semantics and adaptive augmentation. By constructing a large-scale dataset, designing an adaptive augmentation model, a lightweight backbone network, and a semantic enhancement model, it solves the problems of scarce datasets, poor robustness to damaged scenes, high model computational cost, and insufficient utilization of semantic associations in ancient mural element detection. Experiments show that this invention achieves an mAP of 87.5% on a self-built dataset, with a single image detection time of 0.022s and FLOPs of only 2.3G, significantly outperforming mainstream object detection algorithms. In pilot applications in archaeological research, mural damage analysis, digital exhibitions, and cultural relic conservation and restoration, this invention demonstrates excellent practicality, providing key technical support for the digital conservation of ancient murals.

[0122] Future research directions will mainly include the following three aspects:

[0123] Dataset Expansion: Further expand the dataset size, add more element categories such as "carriages and horses", "architecture", and "musical instruments", cover more murals from more regions (such as Xinjiang and Gansu), and improve the model's generalization ability;

[0124] Virtual restoration integration: Combining the detection results of this invention, a virtual restoration algorithm for murals based on a diffusion model is developed to achieve integrated "detection-restoration" and provide more comprehensive technical support for cultural relic protection;

[0125] Mobile Deployment: By optimizing the model structure and employing techniques such as model quantization and pruning, this invention can be deployed to mobile devices (such as mobile phones and tablets), enabling cultural relic conservation personnel to conduct real-time monitoring at the mural site and further improve work efficiency.

[0126] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An ancient mural element detection algorithm based on contextual semantics and adaptive augmentation, characterized in that, Includes the following steps: S1. Construct a dataset for object detection of elements in ancient Chinese murals; S2. Design an adaptive random erasure image augmentation model to augment the dataset constructed in step S1; S3. Construct a lightweight multi-scale feature extraction backbone network to extract features from the augmented image in step S2; S4. Construct a semantic feature enhancement model to semantically enhance the features extracted in step S3; S5. Input the enhanced semantic feature map obtained in step S4 into a fully convolutional network and output the detection results, which include element category, bounding box coordinates and confidence score.

2. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11. Data Acquisition: Collect images of ancient murals from tombs and sites, covering five historical periods: Qin and Han, Wei, Jin and Northern and Southern Dynasties, Sui and Tang, Song and Yuan, and Ming and Qing Dynasties. A total of 2,358 mural images were collected. After white balance correction and noise reduction of the original images, the image resolution was uniformly adjusted to 1024×1024 pixels. S12. Sample annotation: The LabelImg annotation tool was used, and three professionals with archaeological backgrounds annotated the elements in the image. The annotation categories included male, female, beast, flower, bird, and cloud. The annotation format was PASCAL VOC, and the target bounding box coordinates were recorded as xmin, ymin, xmax, and ymax. S13. Labeling Verification and Correction: Cross-validation was used to calculate the Kappa coefficient of the labeling results of the three labelers. When the Kappa coefficient was ≥0.85, the labeling results were considered consistent. When the Kappa coefficient was <0.85, a fourth senior archaeological expert was used for arbitration and correction, resulting in 23,852 valid labeled samples.

3. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 1, characterized in that: The constraints of the adaptive random erasure image augmentation model in step S2 include: Constraint 1: Define IOUSrand as the ratio of the intersection of the area of ​​the random erasure region and the area of ​​the target mark box to the area of ​​the erasure region, satisfying 0≤IOUSrand≤α, where α∈[0.3, 0.7]; Constraint 2: Define IOUSbound as the ratio of the intersection of the area of ​​the random erasure region and the area of ​​the target marker box to the area of ​​the marker box, satisfying 0≤IOUSbound≤β, where β∈[0, 0.3]; Constraint 3: Define Rrand as the aspect ratio of the erased region and Rbound as the aspect ratio of the target bounding box, satisfying Rbound×(1+γ)≤Rrand≤Rbound×(1+δ), where γ∈[-0.1, 0] and δ∈[0, 0.1].

4. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 3, characterized in that: The specific process of generating a random erasure region in step S2 is as follows: Randomly select a point P (Px, Py) within the target marking box. With P as the center, extend a distance to the left, a distance to the right, a distance to the up, and a distance to the down to form a rectangular erasure region. Calculate the IOUSrand, IOUSbound, and Rrand of this region. If the three constraints in step S2 are satisfied, the erasure region is retained; otherwise, point P is reselected until an erasure region that satisfies the conditions is generated.

5. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 1, characterized in that: The lightweight multi-scale feature extraction backbone network in step S3 includes a Stem block and a Stage block. The Stem block includes Stem block-1 and Stage block. One path goes through a 2×2 convolution (stride=1, number of channels=32), and the other path goes through a 1×1 convolution (stride=1, number of channels=32) and a 3×3 convolution (stride=1, number of channels=32) in sequence. After adjusting the feature maps of the two paths to a uniform size, they are fused using Eltwise, and then fused again by a 1×1 convolution (stride = 1, number of channels = 32). Stem block-2 employs a two-way feature extraction and concat fusion approach. Specifically, the output feature map of Stem block-1 is processed through a 3×3 convolution (stride=1, number of channels=32), then divided into two paths. One path undergoes 2×2 max pooling (stride=2, number of channels=32), while the other path undergoes a 1×1 convolution (stride=1, number of channels=32) followed by a 3×3 convolution (stride=2, number of channels=32). The two feature maps are then concat fused and further processed through a 1×1 convolution (stride=1, number of channels=64). The stage block contains five cascaded sub-modules. Each sub-module adopts a three-way feature extraction structure, with the number of channels increasing sequentially to 16, 16, 16, and 32, and is residually connected to the input feature map.

6. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 5, characterized in that: The sub-module processing procedure of the Stage block is as follows: Taking Stage 1-1 as an example, the input feature map is processed in three ways. Branch 1 goes through a 1×1 convolution (stride=1, number of channels is the same as the input feature map) and a 3×3 convolution (stride=1, number of channels is the same as the input feature map) in sequence. Branch 2 goes through 2×2 max pooling (stride=1, number of channels is the same as the input feature map). Branch 3 goes through a 1×1 convolution (stride=1, number of channels is the same as the input feature map), a 3×3 convolution (stride=1, number of channels is the same as the input feature map), and a 1×1 convolution (stride=1, number of channels is the same as the input feature map) in sequence. The three feature maps are fused using Concat, and the number of channels in the input feature map is adjusted to match the number of channels in the fused feature map using 1×1 convolution. Then, residual connections are performed to obtain the output feature map of the submodule.

7. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 1, characterized in that: The semantic feature enhancement model in step S4 includes a context information fusion module and a residual attention module, wherein: The context information fusion module processes the output feature maps of the five sub-blocks in the Stage block. The output feature maps of the first three sub-blocks are first subjected to 3×3 max pooling (stride = 2, number of channels unchanged), and then subjected to 1×1 convolution (stride = 1, number of channels = 512). The output feature maps of the last two sub-blocks are directly passed through a 1×1 convolution (stride=1, number of channels=512). After adjusting the five processed feature maps to a uniform size, they are fused using Concat and then subjected to a 1×1 convolution (stride=1, number of channels=512) to obtain the context semantic feature map. The residual attention module processes the feature map output by the backbone network through a residual convolutional block, and then performs a 3×3 convolution (stride=2, number of channels=256) and a 2×2 max pooling (stride = 2, number of channels=256) to obtain the attention weight map. The attention weight map is upsampled to the same size as the context semantic feature map through bilinear interpolation. After adjusting the number of channels using 1×1 convolution, it is multiplied element-wise with the context semantic feature map to obtain the enhanced semantic feature map.

8. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 1, characterized in that: The specific process of target detection and prediction in step S5 is as follows: A 3×3 convolutional layer (6×1 channels) is used to output the category score, and the category probability is obtained by passing the Softmax function to determine the element category; A 3×3 convolutional layer (6×4 channels) is used to output the bounding box offset, and the final bounding box coordinates are calculated by combining 9 preset anchor boxes (3 scales and 3 aspect ratios). The predicted bounding boxes are processed by NMS, and the IOU threshold is set to 0.5 to remove overlapping bounding boxes and retain the bounding box with the highest confidence. Filter prediction results with a confidence level < 0.5 and output the element category, bounding box coordinates, and confidence level.

9. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 1, characterized in that: In step S2, each original sample generates 5 augmented samples with different erasing regions, expanding the dataset to 143,112 samples, which are divided into a training set (114,489 samples), a validation set (14,311 samples), and a test set (14,312 samples) in an 8:1:1 ratio.

10. The ancient mural element detection algorithm based on contextual semantics and adaptive augmentation according to claim 1, characterized in that: The training parameters of the lightweight multi-scale feature extraction backbone network are set as follows: AdamW optimizer is used, the initial learning rate is 1e-4, and the weight decay is 5e-4. Cosine annealing learning rate scheduling is used, T_max = 100, eta_min = 1e-6; We use a weighted sum of cross-entropy loss and smoothing L1 loss, with a weight ratio of 1:1; The batch size is 16; The training consists of 100 rounds, employing an early stopping strategy. When the validation set mAP shows no improvement for 10 consecutive rounds, training is stopped and the optimal model is saved.