Ancient book paper fiber image classification method based on deep learning and electronic equipment

Abstract

The invention relates to the technical field of image classification, and particularly provides a deep learning-based ancient book paper fiber image classification method and electronic equipment. The method comprises the following steps: constructing a paper fiber image data set, and carrying out preprocessing and data enhancement on the paper fiber image data set to obtain a processed data set; a MobileViT model is used as a basic model, and feature information of fibers is extracted from the processed data set; improving the structure of the MobileViT model according to the feature information of the extracted fibers, and designing a mid-value enhanced fusion attention module to obtain an improved model; according to the method, the self-adaptive comparison loss function is designed, the improved model is trained to enhance the recognition capability of the model on the feature difference between different types of fibers, and the method realizes rapid and accurate classification of paper fibers, ensures the success of the repair work, and improves the storage quality of the repaired paper.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to an ancient book paper fiber image classification method and electronic device based on deep learning. Background Art

[0002] As the main carrier of ancient books, paper will inevitably age over time. In addition, natural disasters and improper storage often lead to damage to the paper of ancient books, resulting in blurred handwriting, page detachment and other problems. These damages not only affect the readability and integrity of ancient books, but also cause valuable historical information to face irreparable loss. Therefore, the restoration of ancient books is particularly important. It not only helps to restore the original appearance of ancient books, but also continues their cultural value. In the process of ancient book restoration, the classification of paper types is crucial. By accurately classifying the types of ancient book paper, restoration experts can select appropriate materials and techniques to ensure the success of the restoration work, maximize the retention of the original characteristics and value of ancient books, and improve the preservation quality of the restored ancient book paper.

[0003] The existing classification methods for ancient book paper have the following problems: experts usually classify by observing the color and texture of paper, but the aging process of paper will change its characteristics, resulting in low accuracy of this method; mass spectrometry can provide detailed chemical composition of paper, thereby improving the accuracy of classification, but this method will cause physical damage to ancient book paper; although spectral analysis has high classification accuracy, the equipment cost is high and cannot be widely used. In this context, the use of deep learning technology provides an efficient and low-cost solution for the automatic classification of ancient book paper fibers. The image was collected using a fiber image analyzer, and the features of the collected images were statistically analyzed. The results show that different fiber types have significant distinguishing features, which lays the foundation for the application of deep learning in paper fiber classification.

[0004] However, there are the following problems in applying deep learning technology to the identification of ancient book paper fibers: since deep learning technology is applied to the field of ancient book paper fiber identification for the first time, there is a lack of high-quality paper fiber datasets; the production process of paper fiber specimens is often accompanied by noise, and the fiber morphology is diverse and easy to overlap, which makes it difficult for the model to extract key features; the differences between different categories of fibers are small and usually only reflected in local details, and it is difficult for the model to capture these subtle category differences. Summary of the invention

[0005] In view of this, the present invention provides an ancient book paper fiber image classification method and electronic device based on deep learning, which are used to achieve fast and accurate classification of paper fibers, ensure the success of restoration work, and improve the preservation quality of the restored paper.

[0006] In a first aspect, the present invention provides a method for classifying ancient book paper fiber images based on deep learning, the method comprising: Step 1, collect fiber images, perform data preprocessing on the fiber images, divide them into training sets and test sets, perform online data enhancement on the training sets, and obtain a paper fiber dataset based on the test set and the data-enhanced training set; Step 2: Use the MobileViT model as the base model to extract fiber feature information from the paper fiber dataset and obtain the final output features; Step 3: According to the final output features, the structure of the MobileViT model is improved, and a median-enhanced fusion attention module is designed to obtain the improved model. Step 4: Design an adaptive contrast loss function, train the improved model, and output the final classification result.

[0007] Optionally, step 1 includes: Step 11, observe the fiber slides through a fiber image analyzer to confirm the characteristics of each fiber; after screening, the fiber image data set includes 3350 ancient book paper fiber images with a resolution of 4088×3072 and 23 different types of paper fibers; Step 12: preprocess the collected fiber images and adjust all images to a resolution of 224×224. Step 13: Use the online image enhancement strategy to simulate different microscopic imaging conditions by horizontally flipping, randomly cropping, vertically flipping, randomly rotating within 30 degrees, and adjusting brightness, contrast, saturation, and hue of the image.

[0008] Optionally, the step 2 includes: Step 21: Given an input feature map , where H and W represent the height and width of the feature map respectively. is the number of input channels; a 3×3 convolutional layer is used to extract the initial local information, with a step size of 2 and a halved spatial size. The expression of the initial local information is: ; in, Represents a 3×3 convolution operation; Step 22: Input Lightweight local feature extraction is performed through several MV2 modules. First, the channel is expanded to , apply the ReLU6 activation function for nonlinear mapping; then extract spatial features through 3×3 depth-separable convolution, keeping the number of channels unchanged; apply the ReLU6 activation function again for nonlinear mapping, and its expression is: ; in, represents a 1×1 point-by-point convolution operation, Represents a 3×3 depth-wise separable convolution operation; Step 23, through 1 × 1 convolution, the channel is transformed from Compress back , output through the linear layer Linear, its expression is: ; Step 24: If the input and output channels are the same and the step size is 1, add a residual connection: ; When the step size is 2, the output is ; Output characteristics , ; Step 25: Input features A more fine-grained local feature is extracted through 3×3 convolution, and its expression is: ; Among them, C represents the channel after extracting local features; Step 26: In order to process local features and capture richer spatial relationships, local features are Divide into N non-overlapping image blocks to form an input sequence , whose expression is: ; Among them, d represents the feature dimension of each image block; Unfold represents the unfolding operation, which converts the feature map into a series of small blocks. Each image block is converted into a vector sequence after unfolding, providing input for global feature learning; Step 27: Process the image block sequence through the Transformer encoder , capturing global relationships. After passing through L layers of Transformer, the model can learn the global context information of the image and obtain the encoding features , whose expression is: ; Where L represents the number of layers of the Transformer encoder stack; Step 28: Encoded features Refold it into the same spatial dimension H×W as the input feature map to ensure the integrity of the information, and its expression is: ; in, is the number of global feature channels. The Fold operation encodes the features Restore to the same spatial structure as the input feature map; Step 29: Use the fusion module to combine local features and global features Combined to obtain enhanced feature representation , whose expression is: ; in, Indicates the number of output channels of the fused features; Step 30, pass 3 3. Convolution adjusts the fused feature channels to obtain the final output feature Y, which is expressed as: .

[0009] Optionally, step 3 includes: Aiming at the final output features, a median-enhanced fusion attention module is designed and integrated into the MobileViT model. The fusion attention module removes image noise through median pooling operation and extracts important features of fiber images in combination with the attention mechanism. Step 31: For the channel attention mechanism, given the feature map First, global maximum pooling, global average pooling and global median pooling are performed on it respectively. Global maximum pooling is used to capture the most representative activation area in the feature map and extract significant features; global average pooling is used to capture the overall information of the channel and reflect the global feature distribution; global median pooling is used to remove noise in the image; the size of each pooling result is ; Secondly, the three pooling results are input into the shared multilayer perceptron to generate the corresponding three attention maps, where the shared multilayer perceptron includes two Convolutional layer, a Leaky ReLU activation function, and a sigmoid function; these three attention maps are element-wise added to get the final channel attention map ;Finally, the channel attention map Multiply the original input feature map F element by element to get the output feature map , and its process calculation expression is: ; ; in, represents the shared multilayer perceptron operation, and They are global maximum pooling, global average pooling and global median pooling, Represents element-wise multiplication; Step 32: In the spatial attention mechanism, firstly, the feature map Through a 5×5 convolution layer, spatial information is captured to generate a preliminary spatial feature map ; Then, considering the characteristics of paper fibers being long and of different lengths, a multi-scale convolution layer is designed to refine the feature expression; the multi-scale feature information is integrated by element-by-element addition of the outputs of convolution layers of different depths to generate a fused feature map ; Finally, a 1×1 convolutional layer is used to transform Convert to the final spatial attention map; spatial attention map and channel weighted feature map Multiply element by element to get the final output feature map, that is, the final classification result. The calculation process expression is: ; ; ; in, Represents depth convolution operations of different sizes, n represents the number of depth convolutions, and Represent 5×5 and 1×1 convolution operations respectively.

[0010] Optionally, the adaptive contrast loss function in step 4 is expressed as: ; ; ; ; Where N is the batch size, and Preprocessing is done by L2 regularization. yes and The similarity measure of is the dynamic threshold, is a fixed threshold, is the weight parameter, represents the adaptive threshold; represents the adaptive contrast loss value; z represents the sample; represents the cross entropy loss value; y represents the true label of the sample, Represents the predicted label of the sample; Represents the overall loss value.

[0011] In a second aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to execute the ancient book paper fiber image classification method based on deep learning in the first aspect or any possible implementation of the first aspect.

[0012] In the technical solution provided by the present invention, the method includes constructing a paper fiber image dataset, and preprocessing and data enhancing the paper fiber image dataset to obtain a processed dataset; using the MobileViT model as a basic model to extract the characteristic information of the fibers from the processed dataset; improving the structure of the MobileViT model based on the extracted characteristic information of the fibers, designing a median-enhanced fusion attention module to obtain an improved model; designing an adaptive contrast loss function, and training the improved model to enhance the model's ability to recognize the characteristic differences between fibers of different categories. The method realizes rapid and accurate classification of paper fibers, ensures the success of the restoration work, and improves the preservation quality of the restored paper. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 A flow chart of a method for classifying fiber images of ancient book paper provided by an embodiment of the present invention; Figure 2 A flowchart of obtaining a paper fiber dataset provided by an embodiment of the present invention; Figure 3 A structural diagram of the MobileViT model provided by an embodiment of the present invention; Figure 4 A structural diagram of the improved MobileViT model provided in an embodiment of the present invention; Figure 5 A diagram of a median-enhanced fusion attention module provided by an embodiment of the present invention; Figure 6 A confusion matrix diagram of the test set classification results provided by an embodiment of the present invention; Figure 7 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] It should be clear that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings.

[0018] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0019] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0020] Figure 1 A flowchart of a method for classifying ancient book paper fiber images provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes: Step 1: Collect fiber images, perform data preprocessing on the fiber images, divide them into training sets and test sets, perform online data enhancement on the training sets, and obtain a paper fiber dataset based on the test set and the data-enhanced training set.

[0021] In the embodiment of the present invention, Figure 2 As shown, step 1 includes: Step 11. Observe the fiber slides through the fiber image analyzer to confirm the characteristics of each fiber. After screening, the fiber image data set includes 3350 ancient book paper fiber images with a resolution of 4088×3072 and 23 different types of paper fibers. The number of each type of fiber sample is shown in Table 1.

[0022] Table 1 Fiber category names and category numbers .

[0023] The fiber slides are prepared by experienced experts in strict accordance with standardized processes to ensure the consistency and accuracy of the images and minimize human errors. All images are taken with a fiber image analyzer. To ensure that every feature of the fiber is captured, the resolution under the objective scale used during shooting is 16μm. Considering the possible optical distortion and excessive impurities caused by factors such as the environment, equipment and filming process, each image is strictly screened, labeled, discarded and cleaned after collection to ensure the comprehensiveness and high quality of the image data.

[0024] Step 12: preprocess the collected fiber images and adjust all images to a resolution of 224×224. In order to adapt to the input requirements of the deep learning model and improve the model processing efficiency, the collected fiber images are preprocessed.

[0025] Step 13: Use the online image enhancement strategy to simulate different microscopic imaging conditions by horizontally flipping, randomly cropping, vertically flipping, randomly rotating within 30 degrees, and adjusting brightness, contrast, saturation, and hue of the image.

[0026] In the embodiment of the present invention, the adjustment amplitude is 0.2; different microscopic imaging conditions are simulated, data diversity is increased, the risk of model overfitting is reduced, and the classification accuracy and robustness of the model are improved.

[0027] Step 2: Use the MobileViT model as the base model to extract the fiber feature information from the paper fiber dataset and obtain the final output features.

[0028] In the embodiment of the present invention, Figure 3 As shown, step 2 includes: The MobileViT model is used as the basic model for feature extraction. The whole model consists of three parts: convolutional layer, MV2 module and MobileViT module. The MV2 module is the inverse residual structure in the MobileNet V2 module. The MobileViT module and MV2 module are the core modules of the MobileViT model. Step 21: Given an input feature map , where H and W represent the height and width of the feature map respectively. is the number of input channels; a 3×3 convolutional layer is used to extract the initial local information, with a step size of 2 and a halved spatial size. The expression of the initial local information is: ; in, Represents a 3×3 convolution operation; Step 22: Input Lightweight local feature extraction is performed through several MV2 modules. First, the channel is expanded to , apply the ReLU6 activation function for nonlinear mapping; then extract spatial features through 3×3 depth-separable convolution, keeping the number of channels unchanged; apply the ReLU6 activation function again for nonlinear mapping, and its expression is: ; in, represents a 1×1 point-by-point convolution operation, Represents a 3×3 depth-wise separable convolution operation; Step 23, through 1 × 1 convolution, the channel is transformed from Compress back , output through the linear layer Linear, its expression is: ; Step 24: If the input and output channels are the same and the step size is 1, add a residual connection: ; When the step size is 2, the output is ; Output characteristics , ; Step 25: Input features A more fine-grained local feature is extracted through 3×3 convolution, and its expression is: ; Among them, C represents the channel after extracting local features; Step 26: In order to process local features and capture richer spatial relationships, local features are Divide into N non-overlapping image blocks to form an input sequence , whose expression is: ; Among them, d represents the feature dimension of each image block; Unfold represents the unfolding operation, which converts the feature map into a series of small blocks. Each image block is converted into a vector sequence after unfolding, providing input for global feature learning; Step 27: Process the image block sequence through the Transformer encoder , capturing global relationships. After passing through L layers of Transformer, the model can learn the global context information of the image and obtain the encoding features , whose expression is: ; Where L represents the number of layers of the Transformer encoder stack; Step 28: Encoded features Refold it into the same spatial dimension H×W as the input feature map to ensure the integrity of the information, and its expression is: ; in, is the number of global feature channels. The Fold operation encodes the features Restore to the same spatial structure as the input feature map; Step 29: Use the fusion module to combine local features and global features Combined to obtain enhanced feature representation , whose expression is: ; in, Indicates the number of output channels of the fused features; Step 30, pass 3 3. Convolution adjusts the fused feature channels to obtain the final output feature Y, which is expressed as: .

[0029] Step 3: According to the final output features, the structure of the MobileViT model is improved, and a median-enhanced fusion attention module is designed to obtain the improved model.

[0030] In paper fiber image classification, the paper fiber has diverse morphologies and is prone to overlap. At the same time, the image is often accompanied by irregular noise. In order to improve the model's ability to recognize paper fiber features and reduce the interference of noise on the model performance, the model structure is improved, such as Figure 4As shown in the figure, the improved paper fiber classification model structure contains several key modules: the first is a 3x3 convolution layer, followed by the first layer consisting of MV2 modules with a stride of 2; the second layer consists of MV2 modules with a stride of 2 and a stride of 1; the third layer combines the MV2 module with a stride of 2 and the MobileViT module; the fourth and fifth layers continue to use the third layer module configuration. In addition, a median enhanced fusion attention module (MEFA) is designed to improve the model's ability to extract key features of paper fibers. Finally, the model performs feature compression through a 1x1 convolution layer and generates the final output through a classifier.

[0031] In the embodiment of the present invention, Figure 5 As shown, step 3 includes: Aiming at the final output features, a median-enhanced fusion attention module is designed and integrated into the MobileViT model. The fusion attention module removes image noise through median pooling operation and extracts important features of fiber images by combining the attention mechanism, which significantly improves the model's recognition ability and robustness for key features of fiber images. Step 31: For the channel attention mechanism, given the feature map First, global maximum pooling, global average pooling and global median pooling are performed on it respectively. Global maximum pooling is used to capture the most representative activation area in the feature map and extract significant features; global average pooling is used to capture the overall information of the channel and reflect the global feature distribution; global median pooling is used to remove noise in the image; the size of each pooling result is ; Secondly, the three pooling results are input into the shared multilayer perceptron to generate the corresponding three attention maps, where the shared multilayer perceptron includes two Convolutional layer, a Leaky ReLU activation function, and a sigmoid function; these three attention maps are element-wise added to get the final channel attention map ;Finally, the channel attention map Multiply the original input feature map F element by element to get the output feature map , and its process calculation expression is: ; ; in, represents the shared multilayer perceptron operation, and They are global maximum pooling, global average pooling and global median pooling, Represents element-wise multiplication; Step 32: In the spatial attention mechanism, firstly, the feature map Through a 5×5 convolution layer, spatial information is captured to generate a preliminary spatial feature map ; Then, considering the characteristics of paper fibers being long and of different lengths, a multi-scale convolution layer is designed to refine the feature expression; the multi-scale feature information is integrated by element-by-element addition of the outputs of convolution layers of different depths to generate a fused feature map ; Finally, a 1×1 convolutional layer is used to transform Convert to the final spatial attention map; spatial attention map and channel weighted feature map Multiply element by element to get the final output feature map, that is, the final classification result. The calculation process expression is: ; ; ; in, Represents depth convolution operations of different sizes, n represents the number of depth convolutions, and Represent 5×5 and 1×1 convolution operations respectively.

[0032] In the embodiment of the present invention, the multi-scale convolution layer includes convolution kernels of different sizes, such as 1×7, 7×1, 1×11, 11×1, 1×21, and 21×1. Through multi-scale convolution, the model can more finely capture features at different scales, thereby improving the ability to express paper fiber features.

[0033] The traditional MobileViT model uses a cross entropy loss function, however, this loss function cannot guide the model to identify subtle differences between different categories. Therefore, the present invention proposes an adaptive contrast loss function to enhance the model's sensitivity to subtle feature differences between different categories of fibers.

[0034] Step 4: Design an adaptive contrast loss function, train the improved model, and output the final classification result.

[0035] In the embodiment of the present invention, the adaptive contrast loss function in step 4 is expressed as: ; ; ; ; Where N is the batch size, and Preprocessing is done by L2 regularization. yes and The similarity measure of is the dynamic threshold, is a fixed threshold, is the weight parameter, represents the adaptive threshold; represents the adaptive contrast loss value; z represents the sample; represents the cross entropy loss value; y represents the true label of the sample, Represents the predicted label of the sample; Represents the overall loss value.

[0036] In the embodiment of the present invention, the adaptive contrast loss function is designed to minimize the similarity between samples of different categories and maximize the similarity between samples of the same category. Guide the model to maximize the similarity between them; for samples of different categories, in order to prevent the loss from being dominated by negative samples with less similarity, introduce an adaptive threshold The threshold consists of a dynamic threshold and a fixed threshold. The dynamic threshold is flexibly adjusted according to the similarity of samples in the batch, which helps to capture the subtle differences between different categories; the fixed threshold effectively controls the fluctuation range of the loss, reduces the training instability caused by excessive fluctuations in the dynamic threshold, and thus enhances the robustness of the training process. When Only then will effective gradient updates be generated, prompting the model to learn to distinguish between different categories. By dynamically adjusting the threshold, we can avoid the excessive impact of irrelevant samples on the loss function, ensuring that the model can focus on negative samples with significant discrimination during training, thereby improving the efficiency and stability of training.

[0037] The existing classification methods for ancient book paper have the following problems: experts usually classify by observing the color and texture of paper, but the aging process of paper changes its characteristics, resulting in low accuracy of this method. Mass spectrometry can provide detailed chemical composition of paper, thereby improving the accuracy of classification, but this method will cause physical damage to ancient book paper. Although spectral analysis has high classification accuracy, the equipment cost is high and cannot be widely used. In this context, the use of deep learning technology to automatically classify ancient book paper fibers provides an efficient and low-cost solution. However, the application of deep learning technology to the field of ancient book paper fiber identification faces many challenges, including the lack of public ancient book paper fiber image datasets, the complex structure of paper fibers, and the small differences in the characteristics of different categories of fibers. Therefore, it is necessary to construct a high-quality paper fiber dataset and develop a deep learning algorithm specifically for paper fiber characteristics to achieve fast and accurate classification of paper fibers.

[0038] In the present invention patent, a high-quality paper fiber dataset is constructed. The first 80% of the dataset is used for model training, i.e., the training set, and the remaining 20% ​​is used for testing, i.e., the test set. The class distribution is balanced to reduce classification bias. The present invention compares performance with MobileViT, Resnet 50, MobileNet V3, ConvNeXt, and ViT. Model performance is evaluated by accuracy, precision, and F1 score.

[0039] ; ; ; ; Where c is the number of categories, The number of samples of the i-th category correctly classified as the i-th category, It indicates the number of samples other than the i-th class that are correctly classified as non-i classes. Indicates the number of samples of other categories that are incorrectly classified as the i-th category, It indicates the number of samples of the i-th category that are incorrectly classified as other categories.

[0040] The accuracy, precision and F1 score of the improved model of the present invention are compared with those of other models. The experimental results are shown in Table 2. The model of the present invention is compared with MobileViT, Resnet50, MobileNet v3, ConvNeXt, and ViT-B / 16. It can be seen from the experimental data that the performance of the improved model of the present invention is better than that of other comparison models in all indicators; Figure 6 As shown, the confusion matrix of the improved model of the present invention for paper fiber classification is displayed, indicating that the model can well distinguish the characteristics of different fibers on the test set.

[0041] Table 2 Recognition results of paper fibers by different algorithms .

[0042] Each step of the embodiment of the present invention may be performed by an electronic device, which includes but is not limited to a mobile phone, a tablet computer, a portable PC, a desktop computer, etc.

[0043] In the technical solution provided by the present invention, the method includes constructing a paper fiber image dataset, and preprocessing and data enhancing the paper fiber image dataset to obtain a processed dataset; using the MobileViT model as a basic model to extract the characteristic information of the fibers from the processed dataset; improving the structure of the MobileViT model based on the extracted characteristic information of the fibers, designing a median-enhanced fusion attention module to obtain an improved model; designing an adaptive contrast loss function, and training the improved model to enhance the model's ability to recognize the characteristic differences between fibers of different categories. The method realizes rapid and accurate classification of paper fibers, ensures the success of the restoration work, and improves the preservation quality of the restored paper.

[0044] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the electronic device where the computer-readable storage medium is located is controlled to execute the above-mentioned embodiment of the ancient book paper fiber image classification method based on deep learning.

[0045] Figure 7 A schematic diagram of an electronic device provided by an embodiment of the present invention, such as Figure 7 As shown, the electronic device 21 includes: a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, the ancient book paper fiber image classification method based on deep learning in the embodiment is implemented. To avoid repetition, they are not described one by one here.

[0046] The electronic device 21 includes, but is not limited to, a processor 211 and a memory 212. Those skilled in the art will appreciate that Figure 7 It is only an example of the electronic device 21 and does not constitute a limitation of the electronic device 21. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0047] The processor 211 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0048] The memory 212 may be an internal storage unit of the electronic device 21, such as a hard disk or memory of the electronic device 21. The memory 212 may also be an external storage device of the electronic device 21, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 21. Further, the memory 212 may also include both an internal storage unit of the electronic device 21 and an external storage device. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 may also be used to temporarily store data that has been output or is to be output.

[0049] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for classifying ancient book paper fiber images based on deep learning, characterized in that: The method comprises: Step 1, collect fiber images, perform data preprocessing on the fiber images, divide them into training sets and test sets, perform online data enhancement on the training sets, and obtain a paper fiber dataset based on the test set and the data-enhanced training set; Step 2: Use the MobileViT model as the base model to extract fiber feature information from the paper fiber dataset and obtain the final output features; Step 3: According to the final output features, the structure of the MobileViT model is improved, and a median-enhanced fusion attention module is designed to obtain the improved model. Step 4: Design an adaptive contrast loss function, train the improved model, and output the final classification result.

2. The method according to claim 1, characterized in that The step 1 comprises: Step 11, observe the fiber slides through a fiber image analyzer to confirm the characteristics of each fiber; after screening, the fiber image data set includes 3350 ancient book paper fiber images with a resolution of 4088×3072 and 23 different types of paper fibers; Step 12: preprocess the collected fiber images and adjust all images to a resolution of 224×224. Step 13: Use online image enhancement strategies to simulate different microscopic imaging conditions by horizontally flipping, randomly cropping, vertically flipping, randomly rotating within 30 degrees, and adjusting brightness, contrast, saturation, and hue of the image.

3. The method according to claim 1, characterized in that The step 2 comprises: Step 21: Given an input feature map , where H and W represent the height and width of the feature map respectively. is the number of input channels; a 3×3 convolutional layer is used to extract the initial local information, with a step size of 2 and a halved spatial size. The expression of the initial local information is: ； in, Represents a 3×3 convolution operation; Step 22: Input Lightweight local feature extraction is performed through several MV2 modules. First, the channel is expanded to , apply the ReLU6 activation function for nonlinear mapping; then extract spatial features through 3×3 depth-separable convolution, keeping the number of channels unchanged; apply the ReLU6 activation function again for nonlinear mapping, and its expression is: ； in, represents a 1×1 point-by-point convolution operation, Represents a 3×3 depth-wise separable convolution operation; Step 23, through 1 × 1 convolution, the channel is transformed from Compress back , output through the linear layer Linear, its expression is: ； Step 24: If the input and output channels are the same and the step size is 1, add a residual connection: ； When the step size is 2, the output is ; Output characteristics , ; Step 25: Input features A more fine-grained local feature is extracted through 3×3 convolution, and its expression is: ； Among them, C represents the channel after extracting local features; Step 26: In order to process local features and capture richer spatial relationships, local features are Divide into N non-overlapping image blocks to form an input sequence , whose expression is: ； Where d represents the feature dimension of each image block; Unfold represents the unfolding operation, which converts the feature map into a series of small blocks. Each image block is converted into a vector sequence after unfolding, providing input for global feature learning. Step 27: Process the image block sequence through the Transformer encoder , capturing global relationships. After passing through L layers of Transformer, the model can learn the global context information of the image and obtain the encoding features , whose expression is: ； Where L represents the number of layers of the Transformer encoder stack; Step 28: Encoded features Refold it into the same spatial dimension H×W as the input feature map to ensure the integrity of the information, and its expression is: ； in, is the number of global feature channels. The Fold operation encodes the features Restore to the same spatial structure as the input feature map; Step 29: Use the fusion module to combine local features and global features Combined to obtain enhanced feature representation , whose expression is: ； in, Indicates the number of output channels of the fused features; Step 30, pass 3 3. Convolution adjusts the fused feature channels to obtain the final output feature Y, which is expressed as: 。 4. The method according to claim 1, characterized in that: The step 3 comprises: Aiming at the final output features, a median-enhanced fusion attention module is designed and integrated into the MobileViT model. The fusion attention module removes image noise through median pooling operation and extracts important features of fiber images in combination with the attention mechanism. Step 31: For the channel attention mechanism, given the feature map First, global maximum pooling, global average pooling and global median pooling are performed on it respectively. Global maximum pooling is used to capture the most representative activation area in the feature map and extract significant features; global average pooling is used to capture the overall information of the channel and reflect the global feature distribution; global median pooling is used to remove noise in the image; the size of each pooling result is ; Secondly, the three pooling results are input into the shared multilayer perceptron to generate the corresponding three attention maps, where the shared multilayer perceptron includes two Convolutional layer, a Leaky ReLU activation function, and a sigmoid function; these three attention maps are element-wise added to get the final channel attention map ;Finally, the channel attention map Multiply the original input feature map F element by element to get the output feature map , and its process calculation expression is: ； ； in, represents the shared multilayer perceptron operation, and They are global maximum pooling, global average pooling and global median pooling, Represents element-wise multiplication; Step 32: In the spatial attention mechanism, firstly, the feature map Through a 5×5 convolution layer, spatial information is captured to generate a preliminary spatial feature map ; Then, considering the characteristics of paper fibers being long and of different lengths, a multi-scale convolution layer is designed to refine the feature expression; the multi-scale feature information is integrated by adding the outputs of convolution layers of different depths element by element to generate a fused feature map ; Finally, a 1×1 convolutional layer is used to transform Convert to the final spatial attention map; spatial attention map and channel weighted feature map Multiply element by element to get the final output feature map, that is, the final classification result. The calculation process expression is: ； ； ； in, Represents depth convolution operations of different sizes, n represents the number of depth convolutions, and Represent 5×5 and 1×1 convolution operations respectively.

5. The method according to claim 1, characterized in that The adaptive contrast loss function in step 4 is expressed as: ； ； ； ； Where N is the batch size, and Preprocessing is done by L2 regularization. yes and The similarity measure of is the dynamic threshold, is a fixed threshold, is the weight parameter, represents the adaptive threshold; represents the adaptive contrast loss value; z represents the sample; represents the cross entropy loss value; y represents the true label of the sample, Represents the predicted label of the sample; Represents the overall loss value.

6. An electronic device, characterized in that: include: one or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to execute the ancient book paper fiber image classification method based on deep learning as described in any one of claims 1 to 5.

Images