Mural disease detection method and system based on multi-stage network and dynamic loss

Abstract

The invention relates to the technical field of image semantic segmentation and defect detection, in particular to a mural disease detection method and system based on a multi-stage network and dynamic loss, and the method comprises the steps: constructing a double-branch multi-stage detection network which comprises a coarse segmentation branch network and a fine segmentation branch network; performing fast down-sampling processing on the input mural image by using the coarse segmentation branch network to generate a low-resolution coarse disease thermodynamic diagram; performing fine segmentation processing on the input wall painting image by using the fine segmentation branch network, and outputting a high-precision disease segmentation image with the same size as the original image; performing weighted fusion on the rough disease thermodynamic diagram output by the rough segmentation branch network and the high-precision disease segmentation diagram output by the fine segmentation branch network to obtain a final binary disease segmentation mask; and training the double-branch multi-stage detection network by adopting a dynamic loss function of damage perception. According to the method, the accuracy and recall rate of mural disease detection are remarkably improved, and particularly, the method has excellent performance in complex background and small target disease detection.

Description

Technical Field

[0001] This invention relates to the field of image semantic segmentation and defect detection technology, and in particular to a method and system for detecting mural defects based on multi-stage networks and dynamic loss, which aims to detect and segment damaged areas (such as paint loss, cracks, mold spots, etc.) in ancient mural images. Background Technology

[0002] Currently, the detection of mural defects mainly falls into two categories: traditional image processing methods and deep learning methods. Traditional methods include threshold segmentation, region growing, and multidimensional gradient detection. For example, some studies have used improved threshold segmentation to detect peeling areas in murals by analyzing the color and brightness characteristics of the peeling areas, but this requires repeated manual trials to select the threshold range. Similarly, using multi-scale morphological gradients and guided filtering to detect cracks and peeling in murals also requires manual parameter setting. These methods rely on manual feature extraction, are highly targeted, and are difficult to adapt to diverse mural defect scenarios. In terms of deep learning, convolutional neural networks (CNNs), especially U-Net, have become the mainstream technology for mural defect segmentation. Some improved methods, such as using ConvNeXt backbone networks to enhance feature extraction or introducing spatial pyramid pooling in the encoder to strengthen the detection of peeling edges, have also been introduced. The Transformer mechanism has also been introduced to improve the accuracy of crack segmentation in Tang Dynasty murals. While these methods have improved mural damage detection performance to some extent, they still have significant shortcomings: First, publicly available mural damage datasets are extremely scarce and imbalanced, making it difficult for existing solutions to accurately and effectively identify various types of mural damage, especially in dealing with extreme imbalances in the damage data (some murals are severely damaged while others only have minor cracks and spots). Second, traditional segmentation networks often exhibit incomplete predictions, making it difficult for the model to simultaneously understand the global semantics and local details of an image. In the detection of small target defects (such as hairline cracks), conventional loss functions can cause the model to favor the background class, ignoring sparse foreground defect regions—for example, for images with very few defect pixels, using ordinary binary cross-entropy loss, the model is more likely to classify pixels as "background" to reduce the error rate, resulting in the neglect of rare but crucial small cracks. In summary, existing technologies are ineffective in handling small defect regions in complex backgrounds and in achieving high detection rates for small targets with high background ratios, failing to meet the needs of refined damage detection for cultural relics and murals. Summary of the Invention

[0003] To address the problems of extreme data imbalance, difficulty in identifying small target cracks, and difficulty in taking into account both global and local information in existing technologies, this invention proposes a mural disease detection method and system based on multi-stage networks and dynamic loss. By using a dual-branch multi-stage detection network and a dynamic loss function for damage perception, the accuracy of mural disease detection and the detection rate of small targets are significantly improved, meeting the requirements for high-precision and high-efficiency detection.

[0004] To achieve the above objectives, the technical solution adopted is:

[0005] This invention provides a method for detecting mural defects based on multi-stage networks and dynamic loss, comprising the following steps:

[0006] A dual-branch, multi-stage detection network is constructed, which includes a coarse segmentation branch network and a fine segmentation branch network.

[0007] The coarse segmentation branch network is used to perform fast downsampling processing on the input mural image to generate a low-resolution coarse heat map of the damage.

[0008] The fine segmentation branch network is used to perform fine segmentation processing on the input mural image, and output a high-precision disease segmentation map of the same size as the original image. The fine segmentation branch network includes an encoder, an edge enhancement module, a Transformer encoder, a query decoder, a dynamic feature modulation module, and a decoder.

[0009] The coarse disease heat map output by the coarse segmentation branch network and the high-precision disease segmentation map output by the fine segmentation branch network are weighted and fused to obtain the final binary disease segmentation mask.

[0010] A damage-aware dynamic loss function is used to train a two-branch multi-stage detection network.

[0011] According to the mural disease detection method based on multi-stage network and dynamic loss of the present invention, the coarse segmentation branch network uses a pre-trained ResNet-18 as the backbone feature extraction network to extract multi-scale hierarchical features of the input mural image, and compresses the high-dimensional features into a single-channel output through a lightweight decoder to obtain a coarse branch disease mask with an input size of 1 / 32; then, bilinear interpolation is used to upsample the coarse mask back to the original image size to obtain the approximate location distribution of the disease area.

[0012] According to the mural disease detection method based on multi-stage network and dynamic loss of the present invention, the processing procedure of the finely segmented branch network further includes:

[0013] Multi-scale features are extracted by an encoder, and edge information is fused by an edge enhancement module. Global dependencies are modeled by a Transformer encoder. Damage embedding vectors are extracted by a query decoder. The feature map output by the Transformer encoder is modulated in the channel dimension by the damage embedding vectors using a dynamic feature modulation module. The modulated features are then upsampled and decoded to output a high-precision lesion segmentation map.

[0014] According to the mural disease detection method based on multi-stage network and dynamic loss of the present invention, the encoder of the fine segmentation branch network is based on U-Net structure, which includes four convolutional blocks. Each convolutional block is followed by max pooling for downsampling, and a channel-spatial attention module (CSA) is introduced in the high-level features.

[0015] According to the mural disease detection method based on multi-stage network and dynamic loss of the present invention, the edge enhancement module further extracts multi-scale edge feature maps of the mural image using a pre-trained shallow ResNet-7 network; projects the edge features of the highest layer in the multi-scale edge feature map so that its number of channels is the same as the number of channels of the highest layer output features of the encoder; and adds and fuses the projected edge features with the highest layer output features of the encoder to obtain a fused feature map that integrates high-level semantics and edge contour information.

[0016] The Transformer encoder flattens the fused feature map output by the edge enhancement module into N feature tokens, adds position encoding, and stacks multiple layers of multi-head self-attention and feedforward networks to output a feature map that fuses global semantics.

[0017] According to the mural disease detection method based on multi-stage network and dynamic loss of the present invention, the query decoder further includes a set of learnable query vectors, which are cross-attentioned with the global feature sequence output by the Transformer encoder to obtain a set of embedding vectors for different types of diseases.

[0018] According to the mural disease detection method based on multi-stage network and dynamic loss of the present invention, the dynamic feature modulation module further includes: performing average pooling on multiple damage embedding vectors output by the query decoder to obtain a global semantic vector; mapping the global semantic vector into a scaling coefficient vector and an offset coefficient vector through a two-layer fully connected network; and using the scaling coefficient vector and the offset coefficient vector to dynamically modulate the feature map output by the Transformer encoder in the channel dimension.

[0019] The decoder of the fine segmentation branch network corresponds to each layer of the encoder. Each level sequentially performs bilinear upsampling, concatenation with the corresponding scale encoder features, convolutional block fusion, and applies channel-spatial attention again, finally outputting the complete segmentation result of the fine branch.

[0020] According to the mural disease detection method based on multi-stage network and dynamic loss of the present invention, the calculation formula of the weighted fusion is further as follows:

[0021] M final =λ·P fine +(1–λ)·P coarse

[0022] Among them, M final For the final binary disease segmentation mask, P fine P is a high-precision disease segmentation map output by a finely segmented branch network. coarse This is a rough disease heatmap upsampled to the original image size, where λ is an adjustable weighting coefficient used to balance segmentation accuracy and disease recall.

[0023] According to the mural disease detection method based on multi-stage network and dynamic loss of the present invention, the dynamic loss function is a combination of dynamically weighted binary cross-entropy loss and Dice loss; the dynamic loss function dynamically adjusts the contribution weight of the image in the overall loss according to the true proportion of diseased pixels in each training image, and assigns additional loss weights to sparse foreground diseased pixels in the image.

[0024] Furthermore, the present invention also provides a mural disease detection system based on multi-stage networks and dynamic loss, used to implement the above-mentioned mural disease detection method based on multi-stage networks and dynamic loss, the system comprising:

[0025] The dual-branch network construction module is used to build a dual-branch multi-stage detection network, which includes a coarse segmentation branch network and a fine segmentation branch network.

[0026] The coarse segmentation processing module is used to perform rapid downsampling processing on the input mural image using the coarse segmentation branch network to generate a low-resolution coarse heat map of the damage.

[0027] The fine segmentation processing module is used to perform fine segmentation processing on the input mural image using the fine segmentation branch network, and output a high-precision disease segmentation map of the same size as the original image. The fine segmentation branch network includes an encoder, an edge enhancement module, a Transformer encoder, a query decoder, a dynamic feature modulation module, and a decoder.

[0028] The result fusion module is used to perform weighted fusion of the coarse disease heat map output by the coarse segmentation branch network and the high-precision disease segmentation map output by the fine segmentation branch network to obtain the final binary disease segmentation mask.

[0029] The network training module is used to train a two-branch multi-stage detection network using an injury-aware dynamic loss function.

[0030] The beneficial effects achieved by adopting the above technical solution are:

[0031] This invention focuses on rare samples through a dynamic loss function, enabling the model to successfully segment hairline cracks and tiny spots that are easily missed by previous methods, achieving a high recall rate with near-zero false negatives. Simultaneously, with the edge enhancement module, the output disease mask contour more closely matches the actual damage shape, with fine boundaries and a high degree of overlap with manually labeled areas. This indicates that this invention is not only suitable for detecting significant diseases such as large-area peeling, but can also accurately locate extremely fine cracks. While ensuring accuracy, this network structure is streamlined and efficient, with inference speed meeting real-time requirements, making it suitable for large-scale mural disease surveys and conservation projects.

[0032] Experiments have verified that the network model of this invention achieves significant performance improvements in mural disease detection. Compared to existing state-of-the-art methods, the average intersection-over-union (IoU) is improved by up to approximately 8.8% on various benchmark datasets, and the Dice coefficient is improved by approximately 6.3%, such as... Figure 2 As shown in the figure. At the same time, classification metrics such as accuracy and recall have also been significantly improved. The model's sensitivity to diseased pixels (recall) and its ability to discriminate against the background (specificity) have both been enhanced, and the overall accuracy has surpassed the existing level. In particular, it has shown better robust recognition capabilities for small diseased areas in complex mural images. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. The drawings are merely illustrative of some embodiments of the present invention and are not intended to limit the scope of the present invention to all embodiments.

[0034] Figure 1 This is an overall framework diagram of the mural disease detection method based on multi-stage network and dynamic loss according to an embodiment of the present invention;

[0035] Figure 2 This is a comparison of various metrics between the present invention and the baseline algorithm;

[0036] Figure 3 This is a structural diagram of the finely segmented branch network according to an embodiment of the present invention;

[0037] Figure 4 This is a structural diagram of the query decoder according to an embodiment of the present invention;

[0038] Figure 5 This is a structural diagram of the dynamic feature modulation module according to an embodiment of the present invention;

[0039] Figure 6 This is a comparison of the test results of the embodiments of the present invention on the mural dataset with Ground Truth;

[0040] Figure 7 This is a comparison result of the embodiments of the present invention with advanced baseline algorithms on a pavement crack dataset;

[0041] Figure 8 This is a comparison result between the embodiment of the present invention and Ground Truth on the wall damage dataset;

[0042] Figure 9 This is a comparison result between the embodiments of the present invention and Ground Truth on the skin disease and sky cloud datasets. Detailed Implementation

[0043] The exemplary solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art.

[0044] Existing technologies suffer from drawbacks such as scarce and imbalanced mural disease datasets, difficulty in balancing global and local information in models, and the tendency to miss small-target diseases. This invention addresses these shortcomings by achieving accurate and automatic segmentation and detection of diseased areas in complex mural images using only image data, without relying on additional textual descriptions. This invention improves the detection rate and segmentation accuracy for diseases such as micro-cracks and peeling, while maintaining accurate localization of large-scale damaged areas, reducing the burden of manually setting hyperparameters. By fusing global semantic and local detail information and dynamically adjusting loss weights, this invention aims to significantly improve the robustness and accuracy of automatic mural disease detection.

[0045] To achieve the aforementioned objectives, this invention discloses a method for detecting mural defects based on multi-stage networks and dynamic loss, the overall framework of which is as follows: Figure 1 As shown, it specifically includes the following content:

[0046] Step S101: Construct a dual-branch multi-stage detection network (MuHP-DLNet), which includes a coarse branch network and a fine branch network.

[0047] Step S102: Use the coarse segmentation branch network to perform fast downsampling processing on the input mural image to generate a low-resolution coarse disease heat map (coarse segmentation map).

[0048] Specifically, the coarse segmentation branch network uses a pre-trained ResNet-18 as the backbone feature extraction network to extract multi-scale hierarchical features (stage0–stage4) from the input mural image. A lightweight decoder (consisting of three convolutional layers + BatchNorm + ReLU + Dropout) compresses the high-dimensional features into a single-channel output, resulting in a coarse branch lesion mask with an input size of 1 / 32. Then, bilinear interpolation is used to upsample this coarse mask back to the original image size to obtain the approximate location distribution of the lesion areas. The coarse branch structure is simple and computationally efficient, providing a global spatial prior in the early stages of model training to guide the fine branches in focusing on potential lesion areas, while also providing stable robustness support for the model in the later stages of training.

[0049] Step S103: Use the fine segmentation branch network to perform fine segmentation processing on the input mural image and output a high-precision lesion segmentation map of the same size as the original image. The fine segmentation branch network includes an encoder, an edge enhancement module, a Transformer encoder, a query decoder, a dynamic feature modulation module, and a decoder.

[0050] The processing steps of the fine segmentation branch network include: extracting multi-scale features through the encoder and introducing an edge enhancement module to fuse edge information; modeling global dependencies through the Transformer encoder; extracting damage embedding vectors through the query decoder; modulating the feature map output by the Transformer encoder using the damage embedding vectors through the dynamic feature modulation module; upsampling and decoding the modulated features to output a high-precision lesion segmentation map.

[0051] The following section provides a detailed introduction to each module of the fine-segmented branch network.

[0052] (1) Encoder

[0053] The encoder of the fine-segmentation branch network is based on the U-Net structure and extracts multi-scale features from the input mural image through progressive downsampling. It includes four convolutional blocks (Conv1–Conv4), each followed by max pooling for a 2x downsampling. In the high-level features (outputs of Conv3 and Conv4 layers), a Channel-Spatial Attention (CSA) module is introduced to dynamically recalibrate the feature channel weights and emphasize spatially relevant areas of disease, thereby highlighting disease clues such as cracks and edges.

[0054] (2) Edge Enhancer Module

[0055] To compensate for the loss of boundary details caused by encoder downsampling, a fine branch introduces an Edge Enhancer to acquire multi-scale edge information from the mural image. Specifically, the Edge Enhancer uses a pre-trained shallow ResNet-7 network to extract edge feature map E (including edge information at different scales). It takes the 1024-channel edge features from the highest layer, projects them through a 1×1 convolution onto the same 256-channel output as the encoder's Conv4, and then adds and fuses them with the Conv4 output features to obtain a feature map F that integrates high-level semantics and edge contour information. fused This fusion feature incorporates both semantic information about the diseased area and preserves object boundary details, making subsequent processing more sensitive to the damage contour. The formula is as follows: F fused =F enc4 +E proj E proj The feature map after 1×1 convolution projection of the deep edge features extracted by Edge Enhancer.

[0056] (3) Transformer Encoder

[0057] After obtaining the high-order features of the fusion edges, the fine branch further introduces a Transformer encoder to model global dependencies. The Transformer encoder will then... fused The feature map is flattened into N feature tokens. By incorporating positional encoding and stacking multiple layers of multi-head self-attention and feedforward networks, the features at each location can globally "see" other regions of the image. This global self-attention mechanism allows the model to capture the correlation between distant pixels, enabling damage patterns scattered across different locations to reference each other, thus more accurately distinguishing between damaged and intact areas. After L layers of multi-head self-attention processing, the output is a feature map F that incorporates global semantics. trans .

[0058] (4) Query Decoder

[0059] The global features output by the Transformer Encoder are further fed into the Query Decoder, such as... Figure 4 As shown, this is used to extract semantics of various potential damage patterns. The query decoder contains a set of learnable query embeddings, which are obtained by combining the global feature sequence output by the Transformer encoder (with respect to feature map F). trans The flattened sequence undergoes cross-attention interaction to obtain a set of embedding vectors Q for different types of diseases. outThese query vectors are equivalent to extracting several possible damage feature descriptions from the image. In other words, each query vector is like a "task" that aims to query and aggregate a specific semantic information related to the disease from the global features (for example, one may specifically learn "cracks", and another may learn "peeling").

[0060] Since the types and quantities of damage in the actual image cannot be determined in advance, the Query Decoder pre-sets a scale of Q queries, from which it adaptively learns the main damage patterns. Each query vector encodes a global damage semantic, and the query set output by the Query Decoder aggregates the semantic information of different damage patterns (such as cracks, peeling, mold, etc.) in the entire image.

[0061] (5) Dynamic Feature Modulation (DFM) module

[0062] To efficiently fuse the global impairment semantics extracted by the Query Decoder back to pixel-level prediction, this invention designs a dynamic feature modulation module, such as... Figure 5 As shown. The purpose of this modulation module is to use global semantic information (query vector) as guidance to dynamically and selectively enhance or suppress each channel of the feature map in order to optimize the subsequent decoding process.

[0063] Specifically, the Q damage embedding vectors obtained from the Query Decoder are first averaged using pooling to obtain a global semantic vector q. glob (Dimension C) represents a summary of the damage distribution across the entire graph. Then, a two-layer fully connected network (MLP) is used to process q... glob The mapping is performed using a scaling factor vector γ and an offset factor vector β with the same dimensions as the number of channels C; finally, the feature map F output by the Transformer Encoder is... trans Dynamic modulation is performed at the channel dimension: F mod [c,h,w]=F trans [c,h,w]×(1+γ c )+β c F trans[c,h,w] represents the feature value of the original feature map at position (h,w) in the c-th channel. In other words, by utilizing the global semantic information obtained from the query, corresponding scaling and offset are applied to the features of each channel. This is equivalent to providing a "dynamic filter" for subsequent decoding, enhancing or suppressing the feature responses of specific channels based on the overall image's damage distribution. Through this dynamic fusion, pixel-level decoding decisions implicitly consider the overall image's damage distribution: enhancing features in channels that help identify damage, and weakening irrelevant or even interfering features, thus effectively combining local details and global semantics to prepare for final decoding.

[0064] (6) Decoder

[0065] The decoder structure for the fine-branch is similar to U-Net, progressively upsampling the modulated features to restore the original image resolution. The decoder consists of four levels, corresponding to the encoder layers Conv1–Conv4. Each level first performs bilinear upsampling of the feature map, doubling its size, then concatenates it with encoder features of the corresponding scale (skip connections), fuses them through convolutional blocks, and finally applies CSA channel spatial attention to further refine the segmentation result. Each level of the decoder outputs an auxiliary segmentation result (corresponding to the resolution of that level). During training, the loss is also calculated on these intermediate outputs to achieve depth supervision, assisting gradient propagation and convergence. Finally, the highest layer of the decoder outputs the complete segmentation result P of the fine branch. fine The same size as the input image.

[0066] Step S104: The coarse disease heat map output by the coarse segmentation branch network and the high-precision disease segmentation map output by the fine segmentation branch network are weighted and fused to obtain the final binary disease segmentation mask.

[0067] Coarse mask P of coarse branch output coarse With fine branch fine mask P fine The final binary segmentation output mask is generated through weighted fusion. The fusion method uses a pixel-wise weighted average:

[0068] M final =λ·P fine +(1–λ)·P coarse

[0069] Among them, M final For the final binary disease segmentation mask, P fine P is a high-precision disease segmentation map output by a finely segmented branch network. coarseThis is a coarse disease heatmap upsampled to the original image size. λ is an adjustable weighting coefficient used to balance segmentation precision and disease recall. λ can be manually set based on validation set performance or determined through learning. A larger λ value relies more on fine branch results to improve segmentation precision; a smaller λ value increases the contribution of coarse branches to improve recall of minor diseases. Through fusion, the high-precision localization of fine branches and the global detection capability of coarse branches can be combined to improve the completeness and accuracy of the final mask.

[0070] Step S105: Train the dual-branch multi-stage detection network using a damage-aware dynamic loss function.

[0071] Dynamic Loss Function Design: This invention proposes a dynamic loss function for damage perception to address the challenges of class imbalance and small target detection. The loss function combines Binary Cross-Entropy (BCE) loss and Dice loss (region overlap loss based on Intersection over Union, IoU) for optimization. In traditional approaches, the weights of these two losses are usually fixed and require manual adjustment to balance overall accuracy and sensitivity to small targets. This invention adaptively adjusts the weights of BCE and Dice terms based on the proportion of damaged pixels and the segmentation difficulty of each training image. Specifically, the true damage ratio of each training image is pre-calculated. When the damaged area in an image is very small (extremely low foreground pixel ratio), it indicates that the sample belongs to an extremely imbalanced and difficult case. This invention increases the weight of such samples in the total loss, increasing their contribution to network parameter updates, thereby forcing the model to pay more attention to those sparse, small-area damages. Simultaneously, within the image, sparse foreground pixels are assigned additional loss weights, for example, by adding a fixed weighting coefficient α, increasing the contribution of the prediction error of these pixels to the loss. Thus, when the model fails to correctly segment these few foreground pixels, the loss increases significantly, prompting the model to "remember" these difficult-to-identify areas and correct biases caused by data imbalance. Conversely, for samples with a large proportion of disease, the model can relatively easily learn large foreground regions; for such samples, a lower relative weight is used (e.g., the weight is restored to 1) to avoid overemphasizing easily segmented parts. Through this mechanism, adaptive loss weighting for different images is achieved, without manual adjustment of hyperparameters, ensuring accurate segmentation of large-scale disease while improving the detection rate of extremely small crack spots. Furthermore, each auxiliary output uses the same dynamically weighted loss as the main output to jointly optimize and improve the model's discriminative ability across different scales. In summary, the dynamic loss function automatically adapts to different types and degrees of damage in mural images during training, significantly mitigating the negative impact of extreme imbalance between positive and negative samples on model training.

[0072] Input / output and training method of dual-branch multi-stage detection network:

[0073] The input to the MuHP-DLNet model can be a color mural image (the size is adjustable, such as 512×512 pixels used in the experiment). The output is a binary mask of the same size as the input image, where "1" represents the detected disease area and "0" represents the intact background area. During network training, a large number of mural images with labeled disease masks are used as the training set input. A phased training approach is preferred: the coarse branch can be trained separately first to quickly learn the approximate disease distribution; then, the coarse and fine branches are trained jointly. In the early stages of fine branch training, the weight of the coarse branch is fixed or a small learning rate is given to stably provide global priors. During training, the dynamically weighted BCE and Dice combined loss are used to jointly supervise and optimize the main output and each auxiliary output. The optimization algorithm can use adaptive gradient methods such as Adam. The initial learning rate and training epochs can be set according to the dataset size (e.g., 1e-4, 100 training epochs). Once training converges, the model can perform inference and prediction on new images. During inference, the image of the mural to be detected is input into a pre-trained MuHP-DLNet model. Coarse and fine branches are computed in parallel to obtain their respective masks, and then fused according to the set λ value to output the final defect detection result. The entire inference process is real-time and efficient, and can meet the requirements of large-scale processing in actual cultural relic surveys while maintaining high accuracy.

[0074] Corresponding to the above method, embodiments of the present invention also disclose a mural disease detection system based on multi-stage networks and dynamic losses, comprising:

[0075] The dual-branch network construction module is used to build a dual-branch multi-stage detection network, which includes a coarse segmentation branch network and a fine segmentation branch network.

[0076] The coarse segmentation processing module is used to perform rapid downsampling processing on the input mural image using the coarse segmentation branch network to generate a low-resolution coarse heat map of the damage.

[0077] The fine segmentation processing module is used to perform fine segmentation processing on the input mural image using the fine segmentation branch network, and output a high-precision disease segmentation map of the same size as the original image. The fine segmentation branch network includes an encoder, an edge enhancement module, a Transformer encoder, a query decoder, a dynamic feature modulation module, and a decoder.

[0078] The result fusion module is used to perform weighted fusion of the coarse disease heat map output by the coarse segmentation branch network and the high-precision disease segmentation map output by the fine segmentation branch network to obtain the final binary disease segmentation mask.

[0079] The network training module is used to train a two-branch multi-stage detection network using an injury-aware dynamic loss function.

[0080] To verify the effectiveness of this scheme, further explanations and illustrations are provided below based on experimental data.

[0081] Experiment 1: Training and testing of a mural disease detection model based on the MuralDD dataset. Results are as follows: Figure 6 As shown.

[0082] This experiment trained the model on multiple publicly available and self-built datasets and validated its performance on a test set. The MuralDD dataset, derived from images of murals in the Dunhuang Grottoes, covers various types of mural damage (such as large-area pigment loss, small-scale cracks and spots), and was carefully organized to ensure a relatively balanced distribution of samples at different damage levels. Before training, the images and corresponding damage masks were preprocessed, including uniformly adjusting the resolution (e.g., 512×512) and data augmentation (random rotation, brightness adjustment, etc.) to improve the model's generalization ability. Then, the network structure was built: the coarse branch uses a ResNet-18 backbone, and the fine branches include an encoder (5 layers of convolutional downsampling with a CSA module), an Edge Enhancer (based on ResNet-7 to extract edge features), a Transformer Encoder (2 layers, with 8 self-attention heads), a Query Decoder (query count Q = 5, dimension = number of encoder output channels), dynamic feature modulation, and a U-Net-style decoder. The loss function uses the dynamically weighted BCE+Dice combination described above. The pixel weighting coefficient α can be, for example, 2.0 to emphasize the sparse foreground. The weights of the BCE and Dice loss terms are dynamically adjusted for each image. The Adam optimizer is used to train the model for 100 epochs with an initial learning rate of 1e-4, and the learning rate is decayed every 20 epochs to fine-tune the model. During training, the fusion weight λ of the coarse and fine branches is initially set to 0.5 to ensure a balance between accuracy and recall. After the model stabilizes, λ can be further optimized based on the validation set (in this embodiment, λ≈0.7 is finally selected to slightly favor the results of the fine branch). After training, the model is used for prediction on the MuralDD test set. For each unseen mural image, the model first generates a disease heatmap quickly by the coarse branch, roughly marking the suspected damage areas; the fine branch then performs a detailed analysis of the image to identify fine cracks and edge details. The coarse and fine results are fused to obtain the final disease mask. Test results show that in mural images containing complex backgrounds and multi-scale diseases, the model of this invention can accurately detect all damage areas of different sizes in the image. For example, in a mural image where there are both large areas of paint peeling and a few small cracks, the model successfully segmented the complete outline of the large peeling area while also capturing hairline cracks that are difficult to detect with the naked eye. In contrast, the traditional U-Net model showed gaps at the edges of the large peeling area in the same image and almost failed to identify small cracks. The mask output by the model accurately marked both large and small defects, which is basically consistent with manual annotation. This embodiment verifies the excellent detection capability of the present invention for multi-scale defects in complex backgrounds: whether it is a mural background with complex colors or defect targets with large differences in size, the model can take into account both global and local information and achieve high-precision and high-recall segmentation results.

[0083] Experiment 2: Detection of Small-Target Diseases Due to Extreme Imbalance

[0084] This experiment demonstrates the application effect of the present invention in small-sample disease scenarios (extremely low proportion of small targets). Images from various datasets were selected, such as... Figure 7-9 As shown, the image contains only a very small number of minute cracks (accounting for less than 1% of the image area), while the vast majority of the remaining areas are intact. In such scenarios, ordinary segmentation models often tend to output the entire background due to the scarcity of foreground pixels, resulting in missed detections of small defects. Based on the model trained in Experiment 1, the model of this invention performs inference on this image: First, the coarse heatmap output by the coarse branch may only give a very weak response due to the fine cracks, but the dynamic loss function has already taught the model to pay attention to such "small damage" samples during training, so the fine branch will not ignore abnormal textures in the image. The global semantics provided by Transformer enable the model to identify abnormal areas that do not belong to normal patterns, even when faced with highly complex mural content (such as the texture of clothing, which is similar in grayscale to cracks); Edge Enhancer ensures that even if the cracks are as fine as hair, their edge signals are still reflected in high-level features. Finally, the fine branch outputs clear crack segmentation results, and the final mask obtained after fusing the coarse branch accurately marks the locations of several cracks that are difficult to detect with the naked eye, without incorrectly marking other background textures as false positives. Compared to models that do not use dynamic weighted loss (which output almost entirely black results), this model successfully detected minute defects in nearly defect-free images, demonstrating its superior detection capability for extremely imbalanced, small-target defects. In this embodiment, the output mask of the model perfectly matches the manually marked locations of fine cracks, achieving near 100% recall and precision even with extremely low foreground proportions. This further illustrates that the present invention, through a multi-branch architecture combined with dynamic loss, significantly outperforms existing technologies in small-sample defect detection tasks. In summary, the two experiments demonstrate that the MuHP-DLNet of this invention achieved excellent performance in mural defect detection of varying complexity, validating the effectiveness and versatility of its technical solution.

[0085] This invention innovatively employs a two-stage architecture of coarse-branch + fine-branch detection network, performing coarse localization followed by fine segmentation, integrating global priors and local details to improve the efficiency and accuracy of disease detection. A dynamic loss function for damage perception is also designed, adjusting the loss weights in real time based on the disease distribution of each image, adaptively balancing BCE and Dice losses, significantly improving the training effect of extremely imbalanced samples, and ensuring that small foreground targets are no longer ignored by the loss function. By introducing a query decoder and a dynamic feature modulation module, the query decoder extracts the global semantic query vector and fuses it back into pixel-level features through dynamic feature modulation, achieving an effective combination of global semantics and local details. This mechanism enables the network to perform targeted detection of disease features of different types and scales, enhancing its adaptability to diverse diseases. In summary, this invention provides a high-precision segmentation technology solution specifically for disease detection in ancient murals, maintaining excellent detection performance even when the foreground-background ratio is significantly different and the disease targets are minute.

[0086] Unless otherwise specifically stated, the components, steps, numerical expressions, and values ​​described in these embodiments do not limit the scope of the invention.

[0087] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0088] The units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations are not considered to be beyond the scope of this invention.

[0089] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This invention is not limited to any particular combination of hardware and software.

[0090] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features within the technical scope disclosed in the present invention; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting mural defects based on multi-stage networks and dynamic loss, characterized in that, Includes the following steps: A dual-branch, multi-stage detection network is constructed, which includes a coarse segmentation branch network and a fine segmentation branch network. The coarse segmentation branch network is used to perform fast downsampling processing on the input mural image to generate a low-resolution coarse heat map of the damage. The fine segmentation branch network is used to perform fine segmentation processing on the input mural image, and output a high-precision disease segmentation map of the same size as the original image. The fine segmentation branch network includes an encoder, an edge enhancement module, a Transformer encoder, a query decoder, a dynamic feature modulation module, and a decoder. The coarse disease heat map output by the coarse segmentation branch network and the high-precision disease segmentation map output by the fine segmentation branch network are weighted and fused to obtain the final binary disease segmentation mask. A damage-aware dynamic loss function is used to train a two-branch multi-stage detection network.

2. The method for detecting mural defects based on multi-stage networks and dynamic losses according to claim 1, characterized in that, The coarse segmentation branch network uses a pre-trained ResNet-18 as the backbone feature extraction network to extract multi-scale hierarchical features from the input mural image. A lightweight decoder compresses the high-dimensional features into a single-channel output to obtain a coarse branch disease mask with an input size of 1 / 32. Then, bilinear interpolation is used to upsample the coarse mask back to the original image size to obtain the approximate location distribution of the disease area.

3. The method for detecting mural defects based on multi-stage networks and dynamic losses according to claim 1, characterized in that, The processing steps of the fine-segmented branch network include: Multi-scale features are extracted by an encoder, and edge information is fused by an edge enhancement module. Global dependencies are modeled by a Transformer encoder. Damage embedding vectors are extracted by a query decoder. The feature map output by the Transformer encoder is modulated in the channel dimension by the damage embedding vectors using a dynamic feature modulation module. The modulated features are then upsampled and decoded to output a high-precision lesion segmentation map.

4. The mural disease detection method based on multi-stage network and dynamic loss according to claim 3, characterized in that, The encoder of the fine segmentation branch network is based on the U-Net structure and contains four convolutional blocks. Each convolutional block is followed by max pooling for downsampling, and a channel-spatial attention module (CSA) is introduced into the high-level features.

5. The method for detecting mural defects based on multi-stage networks and dynamic losses according to claim 3, characterized in that, The edge enhancement module uses a pre-trained shallow ResNet-7 network to extract multi-scale edge feature maps from the mural image; it projects the highest-level edge features in the multi-scale edge feature map so that the number of channels is the same as the number of channels in the highest-level output features of the encoder; it then adds and fuses the projected edge features with the highest-level output features of the encoder to obtain a fused feature map that integrates high-level semantics and edge contour information. The Transformer encoder flattens the fused feature map output by the edge enhancement module into N feature tokens, adds position encoding, and stacks multiple layers of multi-head self-attention and feedforward networks to output a feature map that fuses global semantics.

6. The mural disease detection method based on multi-stage network and dynamic loss according to claim 3, characterized in that, The query decoder contains a set of learnable query vectors, which interact with the global feature sequence output by the Transformer encoder through cross-attention to obtain a set of embedding vectors for different types of diseases.

7. The method for detecting mural defects based on multi-stage networks and dynamic losses according to claim 3, characterized in that, The dynamic feature modulation module specifically includes: performing average pooling on multiple damage embedding vectors output by the query decoder to obtain a global semantic vector; mapping the global semantic vector into a scaling factor vector and an offset factor vector through a two-layer fully connected network; and using the scaling factor vector and the offset factor vector to dynamically modulate the feature map output by the Transformer encoder in the channel dimension. The decoder of the fine segmentation branch network corresponds to each layer of the encoder. Each level sequentially performs bilinear upsampling, concatenation with the corresponding scale encoder features, convolutional block fusion, and applies channel-spatial attention again, finally outputting the complete segmentation result of the fine branch.

8. The method for detecting mural defects based on multi-stage networks and dynamic losses according to claim 1, characterized in that, The formula for calculating the weighted fusion is as follows: M final =λ·P fine +(1–λ)·P coarse Among them, M final For the final binary disease segmentation mask, P fine P is a high-precision disease segmentation map output by a finely segmented branch network. coarse This is a rough disease heatmap upsampled to the original image size, where λ is an adjustable weighting coefficient used to balance segmentation accuracy and disease recall.

9. The method for detecting mural defects based on multi-stage networks and dynamic losses according to claim 1, characterized in that, The dynamic loss function is a combination of dynamically weighted binary cross-entropy loss and Dice loss. The dynamic loss function dynamically adjusts the contribution weight of each image to the overall loss based on the true proportion of diseased pixels in each training image, and assigns additional loss weights to sparse foreground diseased pixels in the image.

10. A mural disease detection system based on multi-stage network and dynamic loss, characterized in that, For implementing the mural disease detection method based on multi-stage network and dynamic loss as described in any one of claims 1-9, the system comprises: The dual-branch network construction module is used to build a dual-branch multi-stage detection network, which includes a coarse segmentation branch network and a fine segmentation branch network. The coarse segmentation processing module is used to perform rapid downsampling processing on the input mural image using the coarse segmentation branch network to generate a low-resolution coarse heat map of the damage. The fine segmentation processing module is used to perform fine segmentation processing on the input mural image using the fine segmentation branch network, and output a high-precision disease segmentation map of the same size as the original image. The fine segmentation branch network includes an encoder, an edge enhancement module, a Transformer encoder, a query decoder, a dynamic feature modulation module, and a decoder. The result fusion module is used to perform weighted fusion of the coarse disease heat map output by the coarse segmentation branch network and the high-precision disease segmentation map output by the fine segmentation branch network to obtain the final binary disease segmentation mask. The network training module is used to train a two-branch multi-stage detection network using an injury-aware dynamic loss function.

Images