Deep learning-based ancient book paper fiber image classification method and electronic device

Abstract

The present invention relates to the technical field of image classification, and in particular, provides a method for classifying ancient book paper fiber images based on deep learning and an electronic device. The method includes constructing a paper fiber image data set, preprocessing and data augmentation on the paper fiber image data set to obtain a processed data set; using the MobileViT model as the basic model to extract the feature information of the fibers from the processed data set; according to the extracted feature information of the fibers, improving the structure of the MobileViT model, designing a median-enhanced fusion attention module to obtain an improved model; designing an adaptive contrast loss function to train the improved model to enhance the model's ability to identify the feature differences between different categories of fibers. This method realizes the rapid and accurate classification of paper fibers, ensures the success of the restoration work, and improves the preservation quality of the paper after restoration.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular, to a method for classifying ancient book paper fiber images based on deep learning and an electronic device. Background Art

[0002] As the main carrier of ancient books, paper will inevitably age over time. In addition, natural disasters and improper storage often cause damage to the paper of ancient books, resulting in problems such as blurred handwriting and page detachment. These damages not only affect the readability and integrity of ancient books, but also lead to the irreversible loss of valuable historical information. Therefore, the restoration of ancient books is particularly important, which not only helps to restore the original appearance of ancient books, but also can continue their cultural value. During the process of ancient book restoration, the classification of paper types is crucial. By accurately classifying the types of ancient book paper, restoration experts can select appropriate materials and techniques to ensure the success of the restoration work, maximize the retention of the original characteristics and value of ancient books, and improve the preservation quality of the restored ancient book paper.

[0003] The existing methods for classifying ancient book paper have the following problems: experts usually classify by observing the color and texture of the paper, but the aging process of the paper will change its characteristics, resulting in low accuracy of this method; mass spectrometry can provide detailed chemical components of the paper, thus improving the accuracy of classification, but this method will cause physical damage to the paper of ancient books; although spectral analysis has high classification accuracy, the equipment cost is high and it cannot be widely applied. In this context, the use of deep learning technology for automatic classification of ancient book paper fibers provides an efficient and low-cost solution. Use a fiber image analyzer to collect images and perform statistical analysis on the collected image features. The results show that different fiber types have significant distinguishing features, laying a foundation for the application of deep learning in paper fiber classification.

[0004] However, applying deep learning technology to the identification of ancient book paper fibers has the following problems: since deep learning technology is first applied to the field of ancient book paper fiber identification, there is a lack of high-quality paper fiber datasets; noise often accompanies the production process of paper fiber specimens, and the fiber morphology is diverse and easy to overlap, resulting in difficulties for the model to extract key features; the differences between different types of fibers are tiny, usually only reflected in local details, and it is difficult for the model to capture these subtle class differences. Summary of the Invention

[0005] In view of this, the present invention provides a method for classifying ancient book paper fiber images based on deep learning and an electronic device, so as to achieve rapid and accurate classification of paper fibers, ensure the success of the restoration work, and improve the preservation quality of the restored paper.

[0006] In a first aspect, the present invention provides a method for classifying ancient book paper fiber images based on deep learning, and the method includes:

[0007] Step 1: Collect fiber images, perform data preprocessing on the fiber images, divide them into a training set and a test set, and perform online data augmentation on the training set to obtain a paper fiber data set based on the test set and the augmented training set;

[0008] Step 2: Use the MobileViT model as the basic model to extract the feature information of the fibers from the paper fiber data set to obtain the final output features;

[0009] Step 3: According to the final output features, improve the structure of the MobileViT model, design a median-enhanced fusion attention module to obtain the improved model;

[0010] Step 4: Design an adaptive contrast loss function to train the improved model and output the final classification result.

[0011] Optionally, the step 1 includes:

[0012] Step 11: Observe the fiber slide through a fiber image analyzer to confirm the characteristics of each fiber; after screening, the fiber image data set includes 3350 ancient book paper fiber images with a resolution of 4088×3072 and 23 different categories of paper fibers;

[0013] Step 12: Perform data preprocessing on the collected fiber images to uniformly adjust all images to a resolution of 224×224;

[0014] Step 13: Use an online image enhancement strategy to simulate different microscopic imaging conditions by horizontally flipping, randomly cropping, vertically flipping, randomly rotating within 30 degrees, and adjusting brightness, contrast, saturation, and hue of the images.

[0015] Optionally, the step 2 includes:

[0016] Step 21: Given an input feature map , where H and W respectively represent the height and width of the feature map, is the number of input channels; use a 3×3 convolutional layer to extract the initial local information, with a stride of 2 and the spatial size halved. The expression of the initial local information is:

[0017] ;

[0018] Among them, represents the 3×3 convolution operation;

[0019] Step 22: Input Lightweight local feature extraction is performed through several MV2 modules. First, the number of channels is expanded to through 1×1 pointwise convolution, and the ReLU6 activation function is applied for non-linear mapping. Subsequently, 3×3 depthwise separable convolution is used to extract spatial features while keeping the number of channels unchanged. The ReLU6 activation function is applied again for non-linear mapping, and its expression is:

[0020] ;

[0021] Among them, represents the 1×1 pointwise convolution operation, and represents the 3×3 depthwise separable convolution operation;

[0022] Step 23: Compress the number of channels from back to through 1×1 convolution, and output through the linear layer Linear. Its expression is:

[0023] ;

[0024] Step 24: If the input and output channels are the same and the stride is 1, then add a residual connection:

[0025] ;

[0026] When the stride is 2, directly output ;

[0027] Output feature , ;

[0028] Step 25: The input feature extracts finer-grained local features through 3×3 convolution. Its expression is:

[0029] ;

[0030] Among them, C represents the number of channels after extracting local features;

[0031] Step 26: To process local features and capture richer spatial relationships, the local feature is divided into N non-overlapping image patches to form an input sequence . Its expression is:

[0032] ;

[0033] Among them, d represents the feature dimension of each image patch; Unfold represents the unfolding operation, which converts the feature map into a series of small patches, and each image patch is converted into a vector sequence after unfolding, providing input for global feature learning;

[0034] Step 27: Process the sequence of image patches through a Transformer encoder , capturing global relationships. After passing through L layers of the Transformer, the model can learn the global context information of the image and obtain encoded features , and its expression is:

[0035] ;

[0036] Among them, L represents the number of layers of the Transformer encoder stack;

[0037] Step 28: Refold the encoded features back into the same spatial dimension H×W as the input feature map to ensure the integrity of information, and its expression is:

[0038] ;

[0039] Among them, is the number of global feature channels, and the Fold operation restores the encoded features to the same spatial structure as the input feature map;

[0040] Step 29: Combine the local features and the global features through a fusion module to obtain an enhanced feature representation , and its expression is:

[0041] ;

[0042] Among them, represents the number of output channels of the fused features;

[0043] Step 30: Adjust the fused feature channels through a 3 ×3 convolution to obtain the final output feature Y, and its expression is:

[0044] .

[0045] Optionally, step 3 includes:

[0046] Design a median-enhanced fusion attention module for the final output feature and integrate it into the MobileViT model; the fusion attention module removes image noise through median pooling operations and combines the attention mechanism to extract important features of the fiber image;

[0047] Step 31: For the channel attention mechanism, given a feature map , first, perform global max pooling, global average pooling, and global median pooling operations on it respectively. Global max pooling is used to capture the most representative activation regions in the feature map and extract significant features; global average pooling is used to capture the overall information of the channels and reflect the global feature distribution; global median pooling is used to remove noise in the image; the size of each pooling result is ; secondly, input the three pooling results into a shared multi-layer perceptron to generate three corresponding attention maps, where the shared multi-layer perceptron includes two convolutional layers, a Leaky ReLU activation function, and a sigmoid function; add the three attention maps element-wise to obtain the final channel attention map ; finally, multiply the channel attention map element-wise with the original input feature map F to obtain the output feature map , and its process calculation expression is:

[0048] ;

[0049] ;

[0050] where, represents the operation of the shared multi-layer perceptron, and are global max pooling, global average pooling, and global median pooling respectively, represents element-wise multiplication;

[0051] Step 32. In the spatial attention mechanism, first, pass the feature map through a 5×5 convolutional layer to capture spatial information and generate a preliminary spatial feature map ; subsequently, in view of the characteristics that the paper fibers are long and of different lengths, design a multi-scale convolutional layer to refine the feature expression; integrate the multi-scale feature information by adding the outputs of convolutional layers with different depths element-wise to generate a fused feature map ; finally, convert into the final spatial attention map through a 1×1 convolutional layer; multiply the spatial attention map element-wise with the channel-weighted feature map to obtain the final output feature map, that is, the final classification result, and its calculation process expression is:

[0052] ;

[0053] ;

[0054] ;

[0055] where, represents depth convolution operations of different sizes, and n represents the number of depth convolutions. and represent 5×5 and 1×1 convolution operations respectively.

[0056] Optionally, the adaptive contrast loss function in step 4 has the following expression:

[0057] ;

[0058] ;

[0059] ;

[0060] ;

[0061] where N represents the batch size. and are preprocessed through L2 regularization. is and is the similarity metric of. is the dynamic threshold. is the fixed threshold. is the weight parameter. represents the adaptive threshold; represents the adaptive contrast loss value; z represents the sample; represents the cross-entropy loss value; y represents the true label of the sample. represents the predicted label of the sample; represents the overall loss value.

[0062] In a second aspect, an embodiment of the present invention provides an electronic device, including: one or more processors; a memory; and one or more computer programs, where the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to execute the method for classifying ancient book paper fiber images based on deep learning in the first aspect or any possible implementation manner of the first aspect.

[0063] In the technical solution provided by the present invention, the method includes constructing a paper fiber image dataset, preprocessing and data augmentation on the paper fiber image dataset to obtain a processed dataset; using the MobileViT model as the basic model to extract the feature information of the fibers from the processed dataset; according to the extracted feature information of the fibers, improving the structure of the MobileViT model, designing a median-enhanced fusion attention module to obtain an improved model; designing an adaptive contrast loss function to train the improved model to enhance the model's ability to recognize the feature differences between different categories of fibers. This method realizes the rapid and accurate classification of paper fibers, ensures the success of the restoration work, and improves the preservation quality of the restored paper. Description of the Drawings

[0064] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0065] Figure 1 It is a flowchart of the method for classifying ancient book paper fiber images provided by the embodiments of the present invention;

[0066] Figure 2 It is a flowchart of obtaining the paper fiber dataset provided by the embodiments of the present invention;

[0067] Figure 3 It is a structural diagram of the MobileViT model provided by the embodiments of the present invention;

[0068] Figure 4 It is a structural diagram of the improved MobileViT model provided by the embodiments of the present invention;

[0069] Figure 5 It is a diagram of the median-enhanced fusion attention module provided by the embodiments of the present invention;

[0070] Figure 6 It is a confusion matrix diagram of the classification results of the test set provided by the embodiments of the present invention;

[0071] Figure 7 It is a schematic diagram of an electronic device provided by the embodiments of the present invention. Detailed Embodiments

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0073] It should be clear that the described embodiments are only some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0074] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the", and "said" used in the embodiments of the present invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0075] It should be understood that the term " / and" used herein is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A / and B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after.

[0076] Depending on the context, the word "if" as used herein may be interpreted as "when", "while", "in response to determining", or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detected (stated condition or event)" may be interpreted as "when determined", "in response to determining", "when detected (stated condition or event)", or "in response to detecting (stated condition or event)".

[0077] Figure 1 The flowchart of the ancient book paper fiber image classification method provided for the embodiments of the present invention is as Figure 1 shown, and the method includes:

[0078] Step 1: Collect fiber images, perform data preprocessing on the fiber images, divide them into a training set and a test set, and perform online data augmentation on the training set to obtain a paper fiber data set based on the test set and the augmented training set.

[0079] In the embodiments of the present invention, as Figure 2 shown, Step 1 includes:

[0080] Step 11: Observe the fiber glass slide through a fiber image analyzer to confirm the characteristics of each type of fiber. After screening, the fiber image dataset includes 3,350 ancient book paper fiber images with a resolution of 4088×3072 and 23 different types of paper fibers. The number of samples for each type of fiber is shown in Table 1.

[0081] Table 1 Category names and quantities of fibers

[0082] 。

[0083] The fiber glass slide is prepared by experienced experts strictly according to a standardized process to ensure the consistency and accuracy of the images and minimize human errors to the greatest extent. All images are taken with a fiber image analyzer. To ensure that every feature of the fiber is captured, the resolution under the objective scale used during shooting is 16μm. Considering issues such as optical distortion and excessive impurities that may be caused by factors such as the environment, equipment, and film preparation process, each image is strictly screened, labeled, discarded, and cleaned after collection to ensure the comprehensiveness and high quality of the image data.

[0084] Step 12: Perform data preprocessing on the collected fiber images and uniformly adjust all images to a resolution of 224×224;

[0085] To meet the input requirements of the deep learning model and improve the model processing efficiency, data preprocessing is performed on the collected fiber images.

[0086] Step 13: Use an online image enhancement strategy to simulate different microscopic imaging conditions by horizontally flipping the image, randomly cropping, vertically flipping, randomly rotating within 30 degrees, and adjusting brightness, contrast, saturation, and hue.

[0087] In the embodiment of the present invention, the adjustment amplitude is 0.2. Simulating different microscopic imaging conditions increases data diversity, reduces the risk of model overfitting, and improves the classification accuracy and robustness of the model.

[0088] Step 2: Use the MobileViT model as the basic model to extract the feature information of the fibers from the paper fiber dataset and obtain the final output features.

[0089] In the embodiment of the present invention, as Figure 3 shown, Step 2 includes:

[0090] Use the MobileViT model as the basic model for feature extraction. It is composed of 3 parts as a whole, namely the convolutional layer, the MV2 module, and the MobileViT module. The MV2 module is the inverted residual structure in the MobileNet V2 module. Among them, the MobileViT module and the MV2 module are the core modules of the MobileViT model;

[0091] Step 21: Given the input feature map , where H and W represent the height and width of the feature map respectively, is the number of input channels; Use a 3×3 convolutional layer to extract the initial local information, with a stride of 2, and the spatial size is halved. The expression for the initial local information is:

[0092] ;

[0093] Where represents the 3×3 convolution operation;

[0094] Step 22: Input is subjected to lightweight local feature extraction through a number of MV2 modules. First, the channels are expanded to through a 1×1 pointwise convolution, and the ReLU6 activation function is applied for non-linear mapping; Subsequently, the spatial features are extracted through a 3×3 depthwise separable convolution, and the number of channels remains unchanged; The ReLU6 activation function is applied again for non-linear mapping, and its expression is:

[0095] ;

[0096] Where represents the 1×1 pointwise convolution operation, represents the 3×3 depthwise separable convolution operation;

[0097] Step 23: Compress the channels from back to through a 1×1 convolution, and output through the linear layer Linear. Its expression is:

[0098] ;

[0099] Step 24: If the input and output channels are the same and the stride is 1, then add a residual connection:

[0100] ;

[0101] When the stride is 2, directly output ;

[0102] Output feature , ;

[0103] Step 25: The input feature extracts finer-grained local features through a 3×3 convolution. Its expression is:

[0104] ;

[0105] Among them, C represents the channel after extracting local features;

[0106] Step 26: To process the local features and capture richer spatial relationships, the local features are divided into N non-overlapping image patches to form an input sequence , and its expression is:

[0107] ;

[0108] Among them, d represents the feature dimension of each image patch; Unfold represents the unfolding operation, which converts the feature map into a series of small patches, and each image patch is converted into a vector sequence after unfolding, providing input for global feature learning;

[0109] Step 27: Process the image patch sequence through the Transformer encoder , capture the global relationship. After passing through L layers of Transformer, the model can learn the global context information of the image and obtain the encoded features , and its expression is:

[0110] ;

[0111] Among them, L represents the number of layers of the Transformer encoder stack;

[0112] Step 28: Refold the encoded features back into the same spatial dimension H×W as the input feature map to ensure the integrity of information, and its expression is:

[0113] ;

[0114] Among them, is the number of global feature channels, and the Fold operation restores the encoded features to the same spatial structure as the input feature map;

[0115] Step 29: Combine the local features and the global features through the fusion module to obtain an enhanced feature representation , and its expression is:

[0116] ;

[0117] Among them, represents the output channel number of the fused features;

[0118] Step 30: Adjust the fused feature channels through 3 3 convolution to obtain the final output feature Y, and its expression is:

[0119] 。

[0120] Step 3: According to the final output features, improve the structure of the MobileViT model, design a median-enhanced fusion attention module, and obtain the improved model.

[0121] In the classification of paper fiber images, aiming at the problems of diverse and easily overlapping paper fiber morphologies, and irregular noises often accompanying in the images, to improve the model's ability to identify paper fiber features and reduce the interference of noises on the model performance, the structure of the model is improved, as Figure 4 shown, the improved paper fiber classification model structure includes multiple key modules: First is the 3x3 convolutional layer, followed by the first layer composed of MV2 modules with a stride of 2; the second layer consists of MV2 modules with strides of 2 and 1; the third layer combines the MV2 module with a stride of 2 and the MobileViT module; the fourth and fifth layers continue to use the module configuration of the third layer. In addition, a median-enhanced fusion attention module (MEFA) is designed to improve the model's ability to extract key features of paper fibers. Finally, the model compresses features through a 1x1 convolutional layer and generates the final output through a classifier.

[0122] In the embodiment of the present invention, as Figure 5 shown, Step 3 includes:

[0123] For the final output features, design a median-enhanced fusion attention module and integrate it into the MobileViT model; the fusion attention module removes image noises through median pooling operations, and at the same time combines the attention mechanism to extract important features of the fiber image, significantly improving the model's recognition ability and robustness for key features of the fiber image;

[0124] Step 31: For the channel attention mechanism, given the feature map , first perform global max pooling, global average pooling, and global median pooling operations on it respectively. Global max pooling is used to capture the most representative activation regions in the feature map and extract significant features; global average pooling is used to capture the overall information of the channels and reflect the global feature distribution; global median pooling is used to remove noises in the image; the size of each pooling result is ; secondly, input the three pooling results into a shared multi-layer perceptron to generate corresponding three attention maps, where the shared multi-layer perceptron includes two convolutional layers, a Leaky ReLU activation function, and a sigmoid function; perform element-wise addition on these three attention maps to obtain the final channel attention map ; finally, use the channel attention map Multiply element - by - element with the original input feature map F to obtain the output feature map , and the calculation expression of the process is:

[0125] ;

[0126] ;

[0127] Among them, represents the operation of a shared multi - layer perceptron, and are global max - pooling, global average - pooling, and global median - pooling respectively, represents element - by - element multiplication;

[0128] Step 32. In the spatial attention mechanism, first, pass the feature map through a 5×5 convolutional layer to capture spatial information and generate a preliminary spatial feature map ; Subsequently, aiming at the characteristics that paper fibers are long and of different lengths, design a multi - scale convolutional layer to refine the feature expression; integrate multi - scale feature information by element - by - element adding the outputs of convolutional layers with different depths to generate a fused feature map ; Finally, convert to the final spatial attention map through a 1×1 convolutional layer; multiply the spatial attention map and the channel - weighted feature map element - by - element to obtain the final output feature map, that is, the final classification result, and its calculation process expression is:

[0129] ;

[0130] ;

[0131] ;

[0132] Among them, represents depth - convolutional operations of different sizes, n represents the number of depth - convolutional operations, and represent 5×5 and 1×1 convolutional operations respectively.

[0133] In the embodiments of the present invention, the multi - scale convolutional layer includes convolutional kernels of different sizes, such as 1×7, 7×1, 1×11, 11×1, 1×21, 21×1. Through multi - scale convolution, the model can capture features at different scales more finely, thereby improving the expression ability of paper fiber features.

[0134] The traditional MobileViT model uses the cross - entropy loss function. However, this loss function cannot guide the model to identify the subtle differences between different categories. Therefore, an adaptive contrast loss function is proposed in the present invention to enhance the model's sensitivity to the subtle feature differences between different categories of fibers.

[0135] Step 4: Design an adaptive contrast loss function to train the improved model and output the final classification result.

[0136] In the embodiment of the present invention, the adaptive contrast loss function in Step 4 has the following expression:

[0137] ;

[0138] ;

[0139] ;

[0140] ;

[0141] where N represents the batch size, and are pre - processed through L2 regularization, is and 's similarity measure, is the dynamic threshold, is the fixed threshold, is the weight parameter, represents the adaptive threshold; represents the adaptive contrast loss value; z represents the sample; represents the cross - entropy loss value; y represents the true label of the sample, represents the predicted label of the sample; represents the overall loss value.

[0142] In the embodiment of the present invention, the designed adaptive contrast loss function aims to minimize the similarity between samples of different categories and maximize the similarity between samples of the same category. For samples of the same category, the adaptive contrast loss function guides the model to maximize the similarity between them through ; for samples of different categories, in order to prevent the loss from being dominated by negative samples with small similarities, an adaptive threshold is introduced. This threshold consists of a dynamic threshold and a fixed threshold. The dynamic threshold adjusts flexibly according to the similarity of samples within the batch, which helps to capture the subtle differences between different categories; the fixed threshold effectively controls the fluctuation range of the loss, reducing the training instability caused by excessive fluctuation of the dynamic threshold, thereby enhancing the robustness of the training process. Only when the similarity of the negative sample pair exceeds will the contrast loss be An effective gradient update will only be generated to prompt the model to learn the distinctions between different categories. By dynamically adjusting the threshold, the excessive influence of irrelevant samples on the loss function can be avoided, ensuring that the model can focus on the negative samples with significant distinctiveness during training, thereby improving the efficiency and stability of training.

[0143] The existing methods for classifying ancient book papers have the following problems: Experts usually classify by observing the color and texture of the papers, but the aging process of the papers will change their characteristics, resulting in low accuracy of this method. Mass spectrometry can provide the detailed chemical composition of the papers, thus improving the classification accuracy, but this method will cause physical damage to the ancient book papers. Although spectroscopic analysis has a high classification accuracy, the equipment cost is high and it cannot be widely applied. Against this background, using deep learning technology to automatically classify the fibers of ancient book papers provides an efficient and low-cost solution. However, applying deep learning technology to the field of identifying the fibers of ancient book papers faces many challenges, including the lack of a publicly available image dataset of ancient book paper fibers, the complex structure of paper fibers, and the small differences in fiber characteristics between different categories. Therefore, it is necessary to construct a high-quality paper fiber dataset and develop a deep learning algorithm specifically for the characteristics of paper fibers to achieve fast and accurate classification of paper fibers.

[0144] In this invention patent, a high-quality paper fiber dataset is constructed. The first 80% of the dataset is used for model training, that is, the training set, and the remaining 20% is used for testing, that is, the test set, and the class distribution is kept balanced to reduce classification bias. The present invention is compared with MobileViT, Resnet 50, MobileNet V3, ConvNeXt, and ViT in terms of performance. The model performance is evaluated by accuracy, precision, and F1 score.

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] where c is the number of classes, the number of samples of the i-th class correctly classified as the i-th class, represents the number of samples other than the i-th class correctly classified as non-i class, represents the number of samples of other classes misclassified as the i-th class, Indicates the number of samples of the i-th class misclassified as other classes.

[0150] Compare the accuracy, precision, and F1 score of the improved model of the present invention with other models. The experimental results are shown in Table 2. The model of the present invention is compared with MobileViT, Resnet50, MobileNet v3, ConvNeXt, and ViT-B / 16. From the experimental data, it can be seen that the performance of the improved model of the present invention is better than that of other comparison models in all metrics; as Figure 6 shown, the confusion matrix of the improved model of the present invention for classifying paper fibers is presented, indicating that the model can well distinguish the characteristics of different fibers on the test set.

[0151] Table 2 Recognition results of paper fibers by different algorithms

[0152] 。

[0153] Each step of the embodiments of the present invention can be executed by an electronic device. Among them, the electronic device includes but is not limited to mobile phones, tablet computers, portable PCs, desktop computers, etc.

[0154] In the technical solution provided by the present invention, the method includes constructing a paper fiber image dataset, preprocessing and data augmentation of the paper fiber image dataset to obtain a processed dataset; using the MobileViT model as the basic model to extract the feature information of the fibers from the processed dataset; according to the extracted feature information of the fibers, improving the structure of the MobileViT model, designing a median-enhanced fusion attention module to obtain an improved model; designing an adaptive contrast loss function to train the improved model to enhance the model's ability to recognize the feature differences between different classes of fibers. This method realizes the fast and accurate classification of paper fibers, ensures the success of the restoration work, and improves the preservation quality of the restored paper.

[0155] The embodiments of the present invention provide a computer-readable storage medium. The computer-readable storage medium includes a stored program. Among them, when the program runs, it controls the electronic device where the computer-readable storage medium is located to execute the embodiments of the above-mentioned method for classifying ancient book paper fiber images based on deep learning.

[0156] Figure 7 Schematic diagram of an electronic device provided by an embodiment of the present invention, as Figure 7 shown, the electronic device 21 includes: a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, it implements the method for classifying ancient book paper fiber images based on deep learning in the embodiments. To avoid repetition, it will not be elaborated here one by one.

[0157] The electronic device 21 includes, but is not limited to, a processor 211 and a memory 212. Those skilled in the art can understand that Figure 7 These are merely examples of the electronic device 21 and do not constitute a limitation on the electronic device 21. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, a bus, etc.

[0158] The so-called processor 211 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0159] The memory 212 may be an internal storage unit of the electronic device 21, such as the hard disk or memory of the electronic device 21. The memory 212 may also be an external storage device of the electronic device 21, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 21. Further, the memory 212 may also include both the internal storage unit and the external storage device of the electronic device 21. The memory 212 is used to store computer programs and other programs and data required by the network device. The memory 212 may also be used to temporarily store data that has been output or is to be output.

[0160] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system, device, and unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0161] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for classifying ancient book paper fiber images based on deep learning, characterized in that The method includes: Step 1: Collect fiber images, perform data preprocessing on the fiber images, divide them into a training set and a test set, and perform online data augmentation on the training set to obtain a paper fiber dataset based on the test set and the augmented training set; Step 2: Use the MobileViT model as the basic model to extract the feature information of the fibers from the paper fiber dataset to obtain the final output features; Step 3: According to the final output features, improve the structure of the MobileViT model, design a median-enhanced fusion attention module to obtain the improved model; Step 4: Design an adaptive contrast loss function to train the improved model and output the final classification result; The said Step 2 includes: Step 21: Given an input feature map , where H and W represent the height and width of the feature map respectively, is the number of input channels; Use a 3×3 convolutional layer to extract initial local information, with a stride of 2, and the spatial size is halved. The expression for the initial local information is: ； Among them, represents a 3×3 convolution operation; Step 22: Input Perform lightweight local feature extraction through a number of MV2 modules. First, expand the channels to through 1×1 pointwise convolution, and apply the ReLU6 activation function for non-linear mapping; subsequently, extract spatial features through 3×3 depthwise separable convolution while keeping the number of channels unchanged; apply the ReLU6 activation function for non-linear mapping again, and its expression is: ； Among them, represents a 1×1 pointwise convolution operation, represents a 3×3 depthwise separable convolution operation; Step 23: Compress the number of channels from back to through a 1×1 convolution, and output through a linear layer Linear. Its expression is: ； Step 24: If the input and output channels are the same and the stride is 1, add a residual connection: ； When the step size is 2, directly output ; Output feature , ; Step 25, input features Extract finer-grained local features through a 3×3 convolution, and its expression is: ； where C represents the channels after extracting local features; Step 26. To process local features and capture richer spatial relationships, the local features are segmented into N non-overlapping image patches to form an input sequence , and its expression is: ； where d represents the feature dimension of each image patch; Unfold represents the unfolding operation, which converts the feature map into a series of small patches, and each image patch is converted into a vector sequence after unfolding, providing input for global feature learning; Step 27: Process the sequence of image patches through a Transformer encoder , capture global relationships. After passing through L layers of the Transformer, the model can learn the global context information of the image and obtain encoded features , and its expression is: ； where L represents the number of layers of the Transformer encoder stack; Step 28, Fold the encoded feature back into the same spatial dimension H×W as the input feature map to ensure the integrity of the information, and its expression is: ； Among them, is the number of global feature channels, and the Fold operation restores the encoded features to the same spatial structure as the input feature map; Step 29: Combine the local feature and the global feature through a fusion module to obtain an enhanced feature representation , and its expression is: ； Among them, represents the output channel number of the fused features; Step 30: Adjust the fused feature channels through 3 3 convolutions to obtain the final output feature Y, and its expression is: ； The adaptive contrast loss function in the said Step 4 has the following expression: ； ； ； ； where N represents the batch size, and are preprocessed by L2 regularization, is and the similarity metric of, is the dynamic threshold, is the fixed threshold, is the weight parameter, represents the adaptive threshold; represents the adaptive contrast loss value; z represents the sample; represents the cross-entropy loss value; y represents the true label of the sample, represents the predicted label of the sample; represents the overall loss value.

2. The method according to claim 1, characterized in that, The said Step 1 includes: Step 11: Observe the fiber slides through a fiber image analyzer to confirm the characteristics of each fiber; after screening, the fiber image dataset includes 3350 ancient book paper fiber images with a resolution of 4088×3072 and 23 different categories of paper fibers; Step 12: Perform data preprocessing on the collected fiber images to uniformly adjust all images to a resolution of 224×224; Step 13: Use an online image enhancement strategy to simulate different microscopic imaging conditions by horizontally flipping, randomly cropping, vertically flipping, randomly rotating within 30 degrees, and adjusting brightness, contrast, saturation, and hue of the images.

3. The method according to claim 1, wherein The said Step 3 includes: For the final output features, design a median-enhanced fusion attention module and integrate it into the MobileViT model; the fusion attention module removes image noise through median pooling operations and simultaneously extracts important features of the fiber images by combining the attention mechanism; Step 31. For the channel attention mechanism, given the feature map , first, perform global max pooling, global average pooling, and global median pooling operations on it respectively. Global max pooling is used to capture the most representative activation regions in the feature map and extract significant features; global average pooling is used to capture the overall information of the channels and reflect the global feature distribution; global median pooling is used to remove noise in the image; the size of each pooling result is ; secondly, input the three pooling results into a shared multi-layer perceptron to generate three corresponding attention maps, where the shared multi-layer perceptron includes two convolutional layers, a Leaky ReLU activation function, and a sigmoid function; add these three attention maps element-wise to obtain the final channel attention map ; finally, multiply the channel attention map element-wise with the original input feature map F to obtain the output feature map , and its process calculation expression is: ； ； Among them, represents a shared multi-layer perceptron operation, and are global max pooling, global average pooling, and global median pooling respectively, represents element-wise multiplication; Step 32. In the spatial attention mechanism, first, the feature map passes through a 5×5 convolutional layer to capture spatial information and generate a preliminary spatial feature map ; Subsequently, considering the characteristics that paper fibers are long and vary in length, a multi-scale convolutional layer is designed to refine the feature representation; by element-wise adding the outputs of convolutional layers with different depths, multi-scale feature information is integrated to generate a fused feature map ; Finally, through a 1×1 convolutional layer, is converted into the final spatial attention map; the spatial attention map and the channel-weighted feature map are multiplied element-wise to obtain the final output feature map, which is the final classification result. The expression of its calculation process is: ； ； ； Among them, represents depth convolution operations of different sizes, n represents the number of depth convolutions, and represent 5×5 and 1×1 convolution operations respectively.

4. An electronic device, characterized in that, including: One or more processors; A memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to execute the deep learning-based ancient book paper fiber image classification method according to any one of claims 1 to 3.

Images