A method and device for detecting the growth of mold on the surface of paper

Abstract

The application relates to the technical field of computer vision and image processing, and particularly discloses a paper surface mold spot growth dynamic detection method and device; the detection method comprises the following steps: S1, acquiring a surface visible light image of a paper sample to be detected; S2, pre-processing the surface visible light image; S3, inputting the pre-processed image into a mold spot segmentation model pre-trained to obtain a pixel-level segmentation result image, wherein the segmentation result image at least contains a mold spot area category and a paper healthy area category; and S4, based on the segmentation result image, calculating one or more quantitative indexes for representing mold spot growth states; the detection device comprises an image acquisition module, a preprocessing module, a segmentation module and an analysis module; and the application is beneficial to realizing high-precision segmentation of a paper surface mold spot area and dynamic quantitative analysis of a mold growth process.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and more specifically, to a method and apparatus for dynamic detection of mold growth on paper surface based on an improved deep learning network. Background Technology

[0002] Paper artifacts and archives, such as ancient books, paintings, and documents, are mainly composed of plant fibers and have porous and hydrophilic physical properties. In high-temperature, high-humidity, and poorly ventilated storage environments, they are highly susceptible to infection by airborne mold spores. When mold colonizes and grows on the surface of paper, forming mold spots, it not only causes chemical degradation of paper fibers such as hydrolysis and oxidation, resulting in a decrease in mechanical strength, but also causes discoloration and adhesion of the paper. In severe cases, mold spots can cover textual information, causing irreversible loss of historical and cultural information. In addition, the spores or metabolites produced by some molds during their metabolism may pose a threat to human health, becoming a significant hidden danger to the safety management of collections in museums and institutions.

[0003] Existing technologies for detecting and assessing mold diseases on the surface of paper artifacts mainly include: 1. Microscopic observation method: Direct observation through a stereomicroscope or biological microscope. This method relies on the experience of professionals and is difficult to achieve rapid, large-area quantitative assessment.

[0004] II. Spectroscopic analysis: Fourier transform infrared spectroscopy (FT-IR), Raman spectroscopy and other techniques are used to analyze the changes in paper chemical composition caused by mold metabolism; this method usually requires sampling, is a destructive or minimally destructive analysis, and is difficult to provide spatial distribution information of mold spots.

[0005] III. Hyperspectral imaging technology: It can simultaneously acquire spatial and spectral information of the target and has a good effect on identifying moldy areas. However, the equipment is expensive and the data acquisition and processing process is complex, making it difficult to popularize in grassroots cultural heritage protection units.

[0006] IV. Electronic nose and thermal imaging technology: This technology indirectly determines mold growth by detecting volatile organic compounds or changes in surface temperature field, but it is easily affected by environmental interference and has low spatial positioning accuracy.

[0007] In recent years, deep learning and semantic segmentation technologies have made breakthrough progress in the field of image recognition and have been successfully applied in medical imaging, remote sensing monitoring, and industrial quality inspection. Semantic segmentation aims to classify each pixel in an image, providing a new technical path for pixel-level localization and quantitative analysis of mold spots. The U-Net network, with its unique encoder-decoder structure and skip connections, has performed well in tasks such as medical cell segmentation and has become one of the basic networks for semantic segmentation.

[0008] However, directly applying the classic U-Net model to mold spot segmentation on paper surfaces also presents several obstacles: First, the contrast between mold spots and the old paper background may be low, with similar colors and textures, resulting in blurred boundaries; second, mold spots vary in shape and size and often coexist with interference such as paper fibers and stains; third, mold spots typically occupy a small proportion of pixels in the image, leading to severe class imbalance. Existing general-purpose segmentation models (such as FCN, DeepLab series, PSPNet, etc.) often suffer from inaccurate boundary segmentation, missed detection of small-area mold spots, and sensitivity to noise when dealing with such complex, weak, and small target segmentation tasks, making it difficult to meet the needs of high-precision quantitative detection and dynamic monitoring of mold spots on paper artifacts.

[0009] The aforementioned technical problems urgently need to be solved. Summary of the Invention

[0010] To address the shortcomings of existing technologies, the present invention aims to provide a method and apparatus for dynamic detection of mold growth on paper surfaces. This method utilizes acquired visible light images of the surface and an improved semantic segmentation model specifically designed for paper mold characteristics to achieve high-precision pixel-level automatic segmentation of mold areas. Based on the segmentation results, quantitative indicators characterizing the growth state of mold are calculated, thereby providing objective and quantitative technical basis for disease assessment, risk warning, and preventive protection decisions for paper cultural relics.

[0011] To achieve the above objectives, the present invention provides a method for detecting the dynamic growth of mold spots on paper surfaces, comprising the following steps: S1. Obtain a visible light image of the surface of the paper sample to be tested; S2. Preprocess the visible light image of the surface; S3. Input the preprocessed image into the pre-trained mold spot segmentation model to obtain a pixel-level segmentation result map, wherein the segmentation result map includes at least the mold spot region category and the paper health region category; S4. Based on the segmentation result image, calculate one or more quantitative indicators to characterize the growth status of mold spots.

[0012] As a further improvement to the technical solution of the present invention, in step S3, the mold spot segmentation model is a semantic segmentation model based on the U-Net architecture, and its encoder and decoder structures have skip connections.

[0013] As a further improvement to the technical solution of the present invention, the encoder part of the improved semantic segmentation model uses a pre-trained VGG16 network as a feature extraction network to extract low-level texture features and high-level semantic features of the image.

[0014] As a further improvement to the technical solution of the present invention, the improved semantic segmentation model also integrates an attention mechanism module; the attention mechanism module is arranged at least at one of the outputs of each level of the encoder, the bottleneck layer of the model, and the skip connection stage of the decoder, in order to enhance the model’s attention to the moldy area and suppress background interference.

[0015] As a further improvement to the technical solution of the present invention, the attention mechanism module is a convolutional attention module, which includes a channel attention submodule and a spatial attention submodule in sequence; wherein, the channel attention submodule is used to recalibrate the weights of the channel dimensions of the feature map to highlight the feature channels related to mold spots; the spatial attention submodule is used to recalibrate the weights of the spatial dimensions of the feature map to focus on the image region where the mold spots are located.

[0016] As a further improvement to the technical solution of the present invention, the bottleneck layer of the improved semantic segmentation model also includes a multi-scale contextual feature extraction module. The module expands the receptive field of the model through parallel dilated convolutional layers with different dilation rates, so as to simultaneously capture the local details and global contextual information of the mold spots.

[0017] As a further improvement to the technical solution of the present invention, the multi-scale context feature extraction module is a spatial pyramid dilated convolution module, which includes at least three parallel dilated convolution branches with dilation rates of 6, 12 and 18 respectively.

[0018] As a further improvement to the technical solution of the present invention, the skip connection is implemented through a differential sensing skip fusion module; the differential sensing skip fusion module is configured to: calculate the difference between the upsampled features of the decoder and the corresponding skip features of the encoder, generate a difference enhancement feature based on the difference, and fuse the difference enhancement feature with the skip feature to strengthen the mold boundary information and suppress noise in the fusion process.

[0019] As a further improvement to the technical solution of the present invention, in step S4, the quantitative index includes the mold coverage rate S. P and mold growth rate (GR); Mold coverage S P The calculation formula is: ; in, This represents the pixel area of ​​the mold patch region in the segmentation result image. This represents the pixel area of ​​the healthy region of the paper in the segmentation result image; The formula for calculating the growth rate (GR) of mold spots is: ; in, S represents the mold coverage rate at two adjacent time points. P The difference, The time interval between two adjacent time points is denoted as .

[0020] On the other hand, the present invention also provides a device for detecting the dynamic growth of mold spots on paper surfaces, comprising: The image acquisition module is used to acquire visible light images of the surface of the paper sample to be tested. The preprocessing module is used to preprocess the visible light image of the surface; The segmentation module, equipped with a pre-trained mold spot segmentation model, processes the pre-processed image and outputs pixel-level segmentation results; and The analysis module is used to calculate one or more quantitative indicators to characterize the growth status of mold spots based on the segmentation result map.

[0021] Compared with the prior art, the method and apparatus for detecting the dynamic growth of mold spots on paper surface provided by the present invention can have, but are not limited to, the following beneficial technical effects: First, it only requires the use of conventional industrial cameras to acquire visible light images, without the need for expensive hyperspectral or spectral equipment, and does not cause any damage to the cultural relics themselves, making it suitable for rapid and frequent inspections in on-site environments such as museums and archives.

[0022] Second, the CBAM-VGG-Unet model proposed in this invention forms an optimized combination for the characteristics of paper mold spots by enhancing feature extraction with VGG16, achieving adaptive attention focusing with CBAM, fusing multi-scale context with ASPP, and optimizing boundary feature fusion with DASF. This model can effectively cope with the challenges of low contrast, complex backgrounds, small targets, and varied shapes. It achieved an average intersection-union ratio of 94.60% on a self-built dataset, and its recognition accuracy for mold spot regions is significantly higher than that of mainstream models such as DeepLabV3, PSPNet, and FCN.

[0023] Third, this invention realizes the transformation from image to objective data by defining and calculating the mold coverage S. P The growth rate (GR) can accurately quantify the severity and speed of mold disease; through continuous monitoring, mold growth curves can be plotted to achieve dynamic assessment and early warning of disease development, providing precise data support for preventive protection.

[0024] Fourth, the entire process, from image input to quantitative indicator output, is automated, which reduces reliance on professionals and ensures the objectivity and consistency of the evaluation results.

[0025] Of course, this method is not only applicable to paper artifacts, but can also be extended to the detection of microbial diseases in porous organic cultural heritage such as textiles and wood, and has broad application prospects in the field of cultural heritage protection.

[0026] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0027] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0028] Figure 1 A flowchart illustrating a method for detecting the dynamic growth of mold spots on paper surface, provided in an embodiment of the present invention; Figure 2 This is a diagram illustrating the composition of a paper surface mold growth dynamic detection device provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the overall structure of the CBAM-VGG-Unet network in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the convolutional attention module used in an embodiment of the present invention; Figure 5 This is a schematic diagram of the spatial pyramid dilated convolution module used in an embodiment of the present invention; Figure 6 Images of three types of mold growth on paper surfaces and their segmentation results obtained based on the method of this invention; Figure 7 This is a growth curve diagram of three types of molds on the surface of paper obtained based on the method of the present invention; Figure 8 A visual comparison of the segmentation results of three types of mold using different semantic segmentation models. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Example 1

[0030] This embodiment provides a method for detecting the dynamic growth of mold spots on paper surfaces. The steps of this method can be described in detail below.

[0031] 1. Dataset Preparation and Preprocessing 1.1 Paper Samples and Mold Culture Red Star brand pure bark Xuan paper (60% Qingtan bark, 40% straw) was selected as the mold culture medium and cut into 5cm×5cm samples; common dominant contaminant species of paper cultural relics: Aspergillus niger, Penicillium citrinum, and Paecilomyces lilacinus were selected as experimental subjects.

[0032] Prepare potato dextrose agar medium, sterilize it, and pour it into petri dishes; inoculate activated mold spores into the medium and incubate in a constant temperature and humidity incubator until the stationary phase; scrape spores with a sterile inoculation loop, place them in sterile water and shake to prepare a spore suspension, count them using a hemocytometer, and adjust the spore concentration to 1×10⁻⁶ using nutrient solution. 2 CFU / mL; Immerse the Xuan paper sample in the spore suspension, ensuring full saturation, then remove it, place it in a sterile petri dish, and incubate it in an incubator with the set temperature and humidity.

[0033] 1.2 Image Acquisition and Annotation Using a Hikvision industrial camera with a constant light source, visible light images of the paper sample surface are taken periodically (e.g., every 2 hours) during the cultivation period, with a resolution of 1024×1024. The acquired visible light images need to cover the lag phase, logarithmic phase, and stationary phase of mold growth.

[0034] The collected images were annotated pixel by pixel using the Labelme annotation tool; five categories were defined: 0-background, 1-healthy paper, 2-Aspergillus niger, 3-Penicillium citrinum, and 4-Penicillium lilacinus; a total of 1169 annotated original images were obtained.

[0035] 1.3 Data Augmentation and Dataset Partitioning To increase data diversity and prevent overfitting, online random data augmentation was performed on the training set images, including: random horizontal / vertical flipping (probability 0.5), random rotation (-15° to +15°), random scaling (0.8 to 1.2 times), random brightness / contrast adjustment (coefficient 0.8 to 1.2), addition of random Gaussian noise (mean 0, variance 0.001), and slight Gaussian blur; after data augmentation, the number of images was expanded to 3315.

[0036] The augmented dataset was randomly divided into a training set (2653 images), a validation set (331 images), and a test set (331 images) in an 8:1:1 ratio. The training set was used for model parameter learning, the validation set was used for hyperparameter tuning and early stopping, and the test set was used for final performance evaluation.

[0037] 2. Construction and training of the mold spot segmentation model (CBAM-VGG-Unet) 2.1 Overall Model Structure like Figure 3 As shown, the CBAM-VGG-Unet model has a symmetrical U-shaped structure, with the encoder (feature extraction path) on the left, the decoder (feature restoration path) on the right, and the bottleneck layer in the middle. The left and right symmetrical layers are connected by a differentially perceptual skip fusion module (DASF).

[0038] 2.2 Encoder The encoder employs the convolutional portion of a pre-trained VGG16 network (a classic deep convolutional neural network, where "16" represents the number of layers with trainable weights); specifically, it uses the first 13 convolutional layers (containing 5 convolutional blocks, each followed by a max-pooling layer) as the backbone for feature extraction; the input image size is [H, W, 3]. After passing through the encoder, feature maps are obtained in five different stages, with their spatial size gradually reduced to 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input, while the number of channels increases sequentially to 64, 128, 256, 512, and 512; VGG16's powerful hierarchical feature extraction capability provides the model with rich feature representations from shallow texture to deep semantics.

[0039] 2.3 Convolutional Attention Module (CBAM) like Figure 4 As shown, CBAM is a lightweight attention module that applies channel attention and spatial attention in sequence.

[0040] Channel attention: For the input feature map F, global average pooling and global max pooling are performed respectively to obtain two description vectors [1,1,C]. These two vectors are fed into a shared two-layer multilayer perceptron. The first layer has C / r neurons (r is the reduction ratio, usually 16) and ReLU activation is used. The second layer has C neurons. The two vectors output by the MLP are added element by element and then activated by the Sigmoid activation function to generate channel attention weights Mc with the shape [1,1,C]. Finally, Mc is multiplied by the input feature map F channel by channel to obtain the channel attention refined feature F'.

[0041] Spatial attention: Taking F' as input, global average pooling and global max pooling are first performed in the channel dimension to obtain two feature maps [H,W,1]. These two feature maps are concatenated in the channel dimension to form a feature map [H,W,2]. Then, a 7×7 convolutional layer is used to reduce its dimension to [H,W,1]. After passing through the Sigmoid activation function, spatial attention weights Ms are generated. Finally, Ms is multiplied by F' pixel by pixel to obtain the final dual attention weighted feature.

[0042] In this model, CBAM is embedded after the output of each stage of the encoder and after the output of the bottleneck layer for adaptive enhancement before feature propagation.

[0043] 2.4 Bottleneck Layer and Spatial Pyramid Dentated Convolution Module The feature map downsampled to 1 / 32 of the encoder output enters the bottleneck layer; such as Figure 5As shown, in the bottleneck layer, conventional convolution operations are performed first, followed by the ASPP module (Dental Spatial Pyramid Pooling Module, a module for capturing multi-scale contextual information. It captures local details and global semantic information of an image simultaneously through multiple parallel dilated convolutional layers with different dilation rates, thereby helping the model better understand objects of different sizes).

[0044] The ASPP module processes the input feature map F in parallel: Branch 1: A 1×1 convolutional layer.

[0045] Branch 2: A 3×3 convolutional layer with an inflation rate d1=6.

[0046] Branch 3: A 3×3 convolutional layer with an inflation rate d2=12.

[0047] Branch 4: A 3×3 convolutional layer with an inflation rate of d3=18.

[0048] Branch 5: A global average pooling layer pools the feature map to [1,1,C], then adjusts the number of channels through a 1×1 convolution, and finally upsamples it back to the original spatial size through bilinear interpolation.

[0049] The output feature maps of the five branches are concatenated along the channel dimension, and then fused and dimensionality-reduced through a 1×1 convolutional layer to obtain the bottleneck layer output feature F rich in multi-scale contextual information. ASPP The calculation process is expressed as follows:

[0050] Where Conv represents convolution operation, the superscript d represents dilation rate, GAP represents global average pooling, Upsample represents upsampling, and [·] represents channel concatenation.

[0051] 2.5 Decoder and Differential Sensing Skip Fusion Module The decoder consists of multiple upsampling-convolutional blocks that progressively restore the spatial resolution of the feature maps. The key point is that the input of each layer of the decoder comes not only from the upsampling of the feature map of the next layer, but also from the feature map of the corresponding layer of the encoder. However, the skip connections here are replaced by the DASF module (Differential Aware Skip Fusion module, which selectively fuses features by calculating the differences between the corresponding feature maps of the encoder and the decoder, aiming to enhance boundary information and suppress noise).

[0052] The operation flow of the DASF module is as follows: Assume the upsampled features from the decoder are F up The corresponding jump feature from the encoder is F skip .

[0053] First, adjust F using a 1×1 convolution. up The number of channels, making it consistent with F skip The same applies, and bilinear interpolation is used to ensure consistent spatial dimensions.

[0054] Calculate F up With F skip absolute difference F diff =|F up -F skip The difference map typically highlights the boundaries and areas of texture variation.

[0055] The difference feature F diff With F up The data is concatenated along the channel dimension, then passed through a 3×3 convolutional layer, followed by batch normalization and ReLU activation to extract the differential enhancement feature F. diffenhanced .

[0056] Finally, F diffenhanced With the original F skip Perform element-wise addition and fusion to obtain the fused feature F that is passed to the current layer of the decoder. fuse .

[0057] This process allows the model to focus more on regions where the encoded and decoded features are inconsistent (usually the target boundary) during fusion, thereby improving edge segmentation quality and reducing interference from redundant background information.

[0058] 2.6 Model Output and Training The last layer of the decoder maps the number of channels to the number of categories K (K=5 in this example) through a 1×1 convolutional layer. Then, the Softmax function outputs the probability of each pixel belonging to each category. The index of the highest probability is taken as the predicted category of the pixel, and finally a segmentation result map of the same size as the input image is generated.

[0059] The model was trained using the cross-entropy loss function, and the Adam optimizer was used for parameter updates; the initial learning rate was set to 1e. -4 The cosine annealing strategy was used to dynamically adjust the batch size; the batch size was set to 8; training was performed on an NVIDIA GeForce RTX3090 GPU; an early stopping strategy was used to prevent overfitting; training was stopped when the validation set loss no longer decreased within 20 consecutive epochs.

[0060] 3. Quantitative Indicator Calculation and Dynamic Analysis 3.1 Calculation of mold coverage The model performs inference on the test image to obtain the segmentation result image; for a single image, it statistically predicts the total number S of pixels that belong to the mold category (which can distinguish different fungal species or be calculated together). M And the total number of pixels predicted as healthy paper, SA According to the formula Calculate the mold coverage of the image; average the results of multiple images of the same type of mold to obtain the average coverage of that type of mold.

[0061] 3.2 Coverage Error Analysis To evaluate the accuracy of the model segmentation results for area calculation, the mean relative error (MRE) is defined as the evaluation metric:

[0062] Where N is the number of test images; c is the predicted area of ​​mold spots in the i-th image by the model; i The actual pixel area of ​​the mold spots is manually labeled; For a very small constant (e.g., 1e) -7 (), to prevent the denominator from being zero.

[0063] 3.3 Quantification of Mold Growth Dynamics Paper samples inoculated with a specific mold were selected, and images were acquired every 2 hours within the critical growth window of approximately 52 to 78 hours post-inoculation, resulting in image sequences at a total of 12 time points (e.g., ...). Figure 6 As shown, ac represents paper samples with Aspergillus niger, Penicillium citrinum, and Paecilomyces lilacinus growing on them, respectively, demonstrating the typical growth states of the three types of molds before and after the lag phase, logarithmic growth phase, and stationary phase; df represents the segmentation results of Aspergillus niger, Penicillium citrinum, and Paecilomyces lilacinus, respectively.

[0064] The trained model is used to segment each image, and the mold coverage S at each time point t is calculated. P (t).

[0065] Calculate the instantaneous growth rate GR(t) between adjacent time points = [S P (t+Δt)-S P (t)] / Δt, where Δt = 2 hours.

[0066] With time t as the x-axis, coverage S P (t) is the ordinate, plot the mold growth curve (e.g.) Figure 7 As shown, a - coverage rate, b - coverage rate of change); by fitting growth models such as logistic functions to the data points, growth parameters can be quantified and used to predict the coverage rate at a future point in time or to determine the growth stage.

[0067] 4. Experimental verification and result analysis 4.1 Evaluation Indicators The average intersection-over-union (IoU) ratio is used as the primary evaluation metric. For the i-th class, its IoU is calculated as follows:

[0068] Among them, TP i A true positive (a pixel correctly predicted as class i), FP i For false positives (pixels that are not of class i but are predicted as class i), FN i A false negative (a pixel in class i is predicted as another class). The average IoU across all classes is used to obtain mIoU.

[0069] 4.2 Ablation Experiment Results Ablation experiments were conducted on a self-built dataset to verify the effectiveness of each improved module. The results are shown in Table 1.

[0070] Table 1. Quantitative analysis of mold spot segmentation and ablation experiments on paper surface

[0071] The results show that the VGG16 encoder and CBAM attention module bring the most significant performance improvement (1.18% and 0.74% mIoU improvement, respectively). The addition of ASPP and DASF modules further optimizes the model, bringing the final mIoU to 94.60%.

[0072] 4.3 Comparison of experimental results The proposed CBAM-VGG-Unet model is compared with mainstream semantic segmentation models such as DeepLabV3 (a well-known semantic segmentation model series), HRNet (high-resolution network), PSPNet (pyramid scene parsing network), FCN (fully convolutional network), and SENet (squeeze-excitation network). The quantitative results are shown in Table 2 below.

[0073] Table 2. Quantitative Analysis of Comparative Experiment on Mold Spot Region Segmentation on Paper Surface In terms of visual contrast (such as) Figure 6 As shown in the figure, DeepLabV3 and HRNet have obvious small target omissions and boundary blurring; PSPNet and FCN have relatively complete segmentation on large patches, but the boundaries are distorted; SENet performs well, but the boundary smoothness is not as good as the model shown in this embodiment; this model shows the most complete region coverage and the clearest boundary contour in the segmentation of various types of molds.

[0074] As shown in Table 3 below, the MRE of this model for Aspergillus niger, Penicillium citrinum, and Paecilomyces lilacinus are 2.56%, 2.08%, and 2.32%, respectively, all lower than the comparative model, proving its accuracy in quantitative area calculation.

[0075] Table 3. Coverage error of each model

[0076] The method shown in this embodiment achieves significantly better technical results than other mainstream models (such as DeeplabV3, FCN, PSPNet, etc.) in the paper mold spot segmentation task. The fundamental reason is that it has carried out a set of systematic and collaborative model improvements for the core difficulties of this specific task. These improvements are not a simple stacking of modules, but an optimization of the entire process from feature extraction, feature selection, context understanding to detailed reconstruction, forming a powerful synergy.

[0077] Specifically, this method uses VGG16 pre-trained on a large dataset (ImageNet) as the encoder backbone. The deep convolutional structure of VGG16 can extract multi-level and more discriminative features from low-level edges / textures to high-level semantics, providing higher-quality raw materials for all subsequent processing. Convolutional attention modules are embedded before the output of each layer of the encoder, the bottleneck layer, and the skip connections. Important feature channels are first screened, and then important regions are located within these channels, achieving accurate feature recalibration. This allows the network's computing resources to be concentrated on analyzing suspicious regions, greatly improving the signal-to-noise ratio and target perception ability in complex backgrounds. A spatial pyramid dilated convolution module is introduced in the bottleneck layer, using dilated convolutions in parallel with dilation rates of (6, 12, 18) to obtain a large receptive field. At the same time, features are observed with different visual ranges, thus achieving stable segmentation performance for mold spots of different sizes. A differential perceptual skip fusion module is used instead of direct stitching. The selective fusion mechanism can effectively filter out background noise in the encoder features that is inconsistent with the semantics of the current decoding stage, making the reconstructed segmentation map boundaries clearer and smoother, and the interior of the region purer.

[0078] The independent contributions of the above improvements in the ablation experiment (VGG +1.18%, CBAM +0.74%, ASPP + DASF further optimized) demonstrate their effectiveness. In the comparative experiment, this scheme leads in all aspects with a mIoU of 94.60%, and performs best in IoU (89.39%, 89.83%, 90.30%) and coverage calculation error MRE (2.56%, 2.08%, 2.32%) in the mold spot region. This directly confirms that the systematic improvement can accurately solve the core challenges of paper mold spot segmentation in terms of weak features, complex background, variable scale, and blurred boundaries, thus achieving better technical results. Example 2

[0079] This embodiment provides a dynamic detection device for mold growth on paper surface that implements the method described in Embodiment 1. The device mainly includes two parts: a hardware acquisition unit and a software processing system, which work together through a data bus and a communication interface.

[0080] 1. Hardware acquisition unit This unit, known as the image acquisition module 101, is responsible for acquiring high-quality images of the paper surface under controlled conditions. Its core is a programmable industrial area array camera (e.g., a Hikvision series camera). The camera is connected to the main control computer via a USB 3.0 or GigE interface. A ring-shaped LED shadowless light source is provided in front of the camera to provide uniform and stable visible light illumination for the sample, minimizing shadows and reflections. The module's workflow is as follows: Under software control, the camera is triggered at a preset resolution (e.g., 1024×1024) and shooting parameters (e.g., exposure time, gain) to capture visible light images of the paper sample surface and transmits the digital image data to the software system in real time.

[0081] 2. Software processing system The system is deployed on a host computer (such as a high-performance workstation equipped with a GPU) and includes the following sequentially connected modules: preprocessing module 102, segmentation module 103, analysis module 104, and user interaction and display module 105.

[0082] The preprocessing module 102 is responsible for normalizing the raw visible light images input from the image acquisition module, preparing input data for the segmentation model. Its specific functions include: data management: storing and organizing the raw images, corresponding annotation files (if any), and metadata; annotation support: providing interfaces or integrated tools (such as Labelme) for users to perform pixel-level annotations on training set images, defining categories such as background, healthy paper, Aspergillus niger, Penicillium citrinum, and Paecilomyces lilacinus; data augmentation: during model training, dynamically applying random geometric and color transformations to the training set images, including rotation, translation, scaling, cropping, brightness / contrast adjustment, adding Gaussian noise, and blurring, to expand dataset diversity and improve model generalization ability; dataset partitioning: automatically dividing the labeled dataset into training, validation, and test sets according to a preset ratio (e.g., 8:1:1).

[0083] The segmentation module 103 is the core analysis unit of the device. This module carries and runs a pre-trained mold spot segmentation model, namely the CBAM-VGG-Unet model. Its functions include: Model loading: Upon module startup, it loads the trained model parameter file (format can be PyTorch .pt, TensorFlow .pb, or ONNX, etc.) from the storage medium; Inference computation: After receiving the standardized image from the preprocessing module, it performs forward propagation computation; Model execution sequence: Multi-level features are extracted through the VGG16 encoder, features are recalibrated using the CBAM attention mechanism, multi-scale context is fused at the bottleneck layer through the ASPP module, and encoder skip features are fused through the DASF module during decoding, ultimately outputting the class probability of each pixel; Result generation: ArgMax operation is applied to the output probability map to generate a pixel-level segmentation result map with the same resolution as the input image, where each pixel value represents its predicted class (e.g., 0-background, 1-healthy paper, 2-Aspergillus niger, etc.).

[0084] The analysis module 104 is connected to the segmentation module 103, receives the segmentation result image output by the segmentation module, and performs quantitative calculation tasks. This analysis module specifically includes the following two calculation units: Coverage Calculation Unit: This unit is configured to perform the following operations: 1) Scan the input segmentation result image and count the total number S of pixels predicted as mold spots. M And the total number of pixels S predicted as healthy areas of the paper A 2) The embedded algorithm is invoked to automatically calculate the coverage rate S of mold spots in the current image according to the formula. P .

[0085] Growth rate calculation unit: This unit is configured to process time series image analysis tasks. Its workflow is as follows: 1) Receive the same paper sample at multiple consecutive time points (denoted as t1, t2, ..., t... n A series of coverage rates S obtained after segmentation and calculation P (t1),S P (t2),...,S P (t n ); 2) Based on this, calculate the mold growth rate GR between adjacent time points, where ΔS P =S P (t i+1 )-S P (t i ), ΔT=t i+1 -t i ;3) Optionally, calculate S for all time points. P Perform curve fitting to plot the dynamic curve of mold growth.

[0086] User interaction and display module 105, whose functions include: Graphical User Interface: Provides a visual operation interface that allows users to configure camera parameters, select analysis models, start / stop the detection process, and view historical data.

[0087] Results visualization: The original visible light image, the pseudo-color image of the segmentation result, and the coverage S are displayed synchronously or in split-screen mode on the interface. P Numerical values, growth rate (GR) values, and growth trend curves.

[0088] Report generation: It can automatically generate detection reports containing key images, data tables and curves, and supports export.

[0089] The device shown in this embodiment realizes an automated closed-loop processing from image acquisition to growth dynamics quantification report. Its workflow is as follows: The user places the paper sample on the stage of the hardware acquisition unit and starts the task through the software interface; the image acquisition module acquires the image and transmits it to the preprocessing module for necessary standardization; subsequently, the image is sent to the segmentation module, and the CBAM-VGG-Unet model outputs the segmentation map; the analysis module then processes the segmentation map, and the coverage calculation unit gives the current mold coverage degree. If it is a time series analysis, the growth rate calculation unit will further give the dynamic change index; all results are finally clearly presented on the user interface.

[0090] Finally, it should be noted that this article uses specific examples to illustrate the principles and implementation methods of the present invention. The above description of the embodiments is only for the purpose of helping to understand the core ideas of the present invention. Without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A method for detecting the dynamic growth of mold spots on paper surface, characterized in that, Includes the following steps: S1. Obtain a visible light image of the surface of the paper sample to be tested; S2. Preprocess the visible light image of the surface; S3. Input the preprocessed image into the pre-trained mold spot segmentation model to obtain a pixel-level segmentation result map, wherein the segmentation result map includes at least the mold spot region category and the paper health region category; S4. Based on the segmentation result image, calculate one or more quantitative indicators to characterize the growth status of mold spots.

2. The method according to claim 1, characterized in that: In step S3, the mold spot segmentation model is a semantic segmentation model based on the improved U-Net architecture, and its encoder and decoder structures have skip connections.

3. The method according to claim 2, characterized in that: The encoder part of the improved semantic segmentation model uses a pre-trained VGG16 network as a feature extraction network to extract low-level texture features and high-level semantic features of the image.

4. The method according to claim 3, characterized in that: The improved semantic segmentation model also integrates an attention mechanism module; the attention mechanism module is placed at least at one of the outputs of each level of the encoder, the bottleneck layer of the model, and the skip connection stage of the decoder, to enhance the model's attention to the moldy area and suppress background interference.

5. The method according to claim 4, characterized in that: The attention mechanism module is a convolutional attention module, which includes a channel attention submodule and a spatial attention submodule. The channel attention submodule is used to recalibrate the weights of the channel dimensions of the feature map to highlight the feature channels related to mold spots. The spatial attention submodule is used to recalibrate the weights of the spatial dimensions of the feature map to focus on the image region where the mold spots are located.

6. The method according to any one of claims 3-5, characterized in that: The bottleneck layer of the improved semantic segmentation model also includes a multi-scale contextual feature extraction module. This module expands the receptive field of the model through parallel dilated convolutional layers with different dilation rates to simultaneously capture local details and global contextual information of mold spots.

7. The method according to claim 6, characterized in that: The multi-scale context feature extraction module is a spatial pyramid dilated convolution module, which includes at least three parallel dilated convolution branches with dilation rates of 6, 12, and 18, respectively.

8. The method according to any one of claims 2-7, characterized in that: The skip connection is implemented through a differential sensing skip fusion module; the differential sensing skip fusion module is configured to: calculate the difference between the upsampled features of the decoder and the corresponding skip features of the encoder, generate a difference enhancement feature based on the difference, and fuse the difference enhancement feature with the skip features to strengthen the mold boundary information and suppress noise in the fusion process.

9. The method according to claim 1, characterized in that: In step S4, the quantitative indicators include mold coverage rate SP and mold growth rate GR; The formula for calculating the mold coverage rate (SP) is: ; in, This represents the pixel area of ​​the mold patch region in the segmentation result image. This represents the pixel area of ​​the healthy region of the paper in the segmentation result image; The formula for calculating the growth rate (GR) of mold spots is: ; Where S represents the mold coverage rate at two adjacent time points. P The difference is the time interval between the two adjacent time points.

10. A device for detecting the dynamic growth of mold spots on paper surfaces, characterized in that, include: The image acquisition module is used to acquire visible light images of the surface of the paper sample to be tested. The preprocessing module is used to preprocess the visible light image of the surface; The segmentation module is equipped with a pre-trained mold spot segmentation model, which is used to process the pre-processed image and output pixel-level segmentation results. as well as The analysis module is used to calculate one or more quantitative indicators to characterize the growth status of mold spots based on the segmentation result map.

Images