A method for extracting a colored drawing cultural relic line drawing

Abstract

The application belongs to the technical field of image processing, and specifically discloses a kind of colored drawing cultural relic line drawing extraction method, first, the collected colored drawing cultural relic dataset is classified according to the degree of disease erosion by a classification network, then the gradient map corresponding to the input image is calculated, both are used as network input, through scale enhancement module to fully extract different scale features in original colored drawing cultural relic image and gradient image, provide more detailed information for the network, then introduce attention mechanism in the network, make the network pay more attention to the object contour part, at the same time, add adaptive weight, so that the network automatically adjusts according to the type characteristics of cultural relics. The application effectively utilizes the advantages of multi-scale feature extraction and gradient attention mechanism, realizes the line drawing extraction of colored drawing cultural relics through an end-to-end unsupervised recurrent generative adversarial network, and finally obtains high-quality line drawing with complete semantics, rich details and little noise.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a method for extracting a colored drawing cultural relic line drawing. BACKGROUND

[0002] As important historical relics, colored drawing cultural relics have important cultural value. How to obtain accurate line drawings of colored drawing cultural relics is an important problem in the field of cultural heritage protection and inheritance, and has great significance.

[0003] Currently, the related technologies for extracting line drawings of colored drawing cultural relics mainly fall into three categories: gradient-based algorithms, feature learning-based algorithms, and deep learning-based algorithms. The first two categories are traditional algorithms, and related research has been relatively mature, making it difficult to improve, and the quality of the extracted line drawings is not as good as the emerging deep learning algorithms. The mainstream algorithm for deep learning-based line drawing extraction is divided into convolutional neural networks that need paired images to train, and generative adversarial networks that can be trained based on unpaired images.

[0004] Convolutional neural networks (CNN) have shown strong ability in automatically learning high-level representations of natural images. CNN-based algorithms have been widely used in colored drawing line drawing extraction technology, including BDCN, HED, RCF, etc. They can achieve the purpose of extracting colored drawing cultural relics by automatically learning the features of line drawings. However, on the one hand, CNN-based networks usually need a large amount of paired image data to train the network, and the paired data of colored cultural relics is scarce, which leads to insufficient network training and reduces the accuracy of the extracted line drawing images; on the other hand, with the increase of network layers, the features extracted by convolution become more abstract, and the details of the image are more likely to be ignored, resulting in a lack of details or blurred lines in the extracted line drawing images.

[0005] In recent years, with the rapid development of generative adversarial networks (GAN), GAN-based image translation algorithms have also shown great potential in line drawing extraction. Algorithms based on CycleGAN or its variants no longer rely on paired data and have shown superior performance in image translation, indicating a new direction for line drawing extraction. However, the training of CycleGAN is unstable, which often leads to discontinuous edges or loss of important details in the generated line drawing of colored cultural relics, especially for complex disease background colored cultural relics images. CycleGAN only learns features from the original image domain, and the captured features have color, texture, etc., but are also affected by complex noise features in the image background, resulting in discontinuous and incomplete line drawing images with low accuracy. SUMMARY

[0006] The present application aims to overcome the defects in the prior art and provide a colored drawing cultural relic line drawing extraction method.

[0007] The present application provides a colored drawing cultural relic line drawing extraction method, comprising the following steps:

[0008] Step one: data set pre-classification;

[0009] Collecting colored drawing cultural relic pictures, classifying the colored drawing cultural relic pictures through a classification network model to obtain a clear data set and a disease data set;

[0010] Step two: establishing a generative adversarial model;

[0011] Step three: training the generative adversarial model

[0012] The test set of the clear data set and the disease data set is used to train the generative adversarial model respectively to obtain a first generative adversarial model and a second generative adversarial model;

[0013] Step four: using the trained first generative adversarial model and the second generative adversarial model to extract line drawing pictures.

[0014] Further, the classification process of the colored drawing cultural relic picture is:

[0015] The classification network model is trained using the colored drawing cultural relic pictures classified in advance to obtain a trained classification network model;

[0016] The collected colored drawing cultural relic pictures to be classified are input into the trained classification network model, and the colored drawing cultural relic pictures to be classified are classified.

[0017] Further, the generative adversarial model comprises:

[0018] Scale enhancement module: for multi-scale extraction of features, including one ordinary convolution and three dilated convolutions, the dilated rates of the three dilated convolutions are different;

[0019] Feature extraction module: for extracting features of input images and gradient images respectively; including two 3*3 convolutions and nine residual blocks;

[0020] Adaptive gradient attention fusion module: for obtaining cultural relic feature maps with attention guided by gradient information; including an attention module and an adaptive fusion module, wherein the attention module is composed of two full connection layers, self-learning parameters alpha and beta, a cat() function, a 1*1 convolution and a sigmoid() function; the adaptive fusion module includes a self-learning parameter gamma and a relu() activation function;

[0021] Up-sampling module: including two 3*3 convolutions.

[0022] Further, the training step of the generative adversarial model comprises:

[0023] S3.1: input the clear data set and the disease data set as input pictures I In , respectively

[0024] S3.2: extract the gradient map of the painted cultural relic picture

[0025] S3.3: downsample the cultural relic picture and the gradient map, respectively, and extract features using a feature extraction module to obtain cultural relic feature map F r and gradient feature map E g ;

[0026] S3.4: perform global maximum pooling and global average pooling on the gradient feature map E g to obtain two groups of weight vectors W a , W m ;

[0027] S3.5: multiply and superimpose the cultural relic feature map F r with W a , W m to obtain the global average pooled cultural relic feature map F ra and the global maximum pooled cultural relic feature map F rm , and then put the global average pooled cultural relic feature map F ra and the global maximum pooled cultural relic feature map F rm into a global feature fusion layer with a convolution kernel of 1*1 for adaptive fusion to obtain the gradient information guided attention cultural relic feature map A r ;

[0028] S3.6: multiply and superimpose the two groups of cultural relic feature maps A r with two self-learning parameters α and β, respectively, to obtain the attention cultural relic feature map F′, and the calculation process is as follows:

[0029] F′=f(α×F ra +β×F rm )

[0030] S3.7: multiply the cultural relic feature map E r with the fused attention cultural relic feature map with self-learning parameters γ and (1-γ), respectively, and add them to obtain the fused cultural relic feature map F fuse , and the calculation process is as follows:

[0031] F fuse =(1-γ)×F′+γ×F r

[0032] S3.8: fusing the cultural relic feature maps F fuse upsampling is performed;

[0033] S3.9: outputting the line drawing map, completing one training;

[0034] S3.10: repeating steps S3.1-S3.9 to obtain a first generative adversarial model for outputting a line drawing map of a clear data set and a second generative adversarial model for outputting a line drawing map of a disease data set, respectively.

[0035] Further, in step S3.3, the feature extraction includes:

[0036] The input image I In is first extracted by a convolution kernel with a size of 3*3 and a step of 2 to obtain shallow low-level features F0:

[0037] F0=Conv 3*3 (I In )

[0038] The obtained feature map F0 is put into a scale enhancement module for multi-scale feature extraction, the scale enhancement module includes one ordinary convolution and three dilated convolutions with different dilation rates to extract image features of different scales, and finally the above results are superimposed and fused; the operation of the scale enhancement module includes:

[0039] F1=DConv 3*3 (F0,dilation=1)

[0040] F2=DConv 3*3 (F1,dilation=4*1)

[0041] F3=DConv 3*3 (F1,dilation=4*2)

[0042] F4=DConv 3*3 (F1,dilation=4*3)

[0043] F5=F1+F2+F3+F4

[0044] Wherein, DConv 3*3 · represents a dilated convolution with different dilation rates, dilation represents the dilation rate, F1 represents the output feature of the first ordinary convolution, F2, F3, and F4 represent the output features of the first, second, and third dilated convolutions, respectively, and F5 represents the superimposed multi-scale feature map.

[0045] Further, in S3.5, the cultural relic feature map F r is respectively multiplied by Wa , W m The formula of multiplication superposition is:

[0046]

[0047]

[0048] The gradient information guided artifact feature map A with attention r The calculation formula is:

[0049] A r = Conv 1*1 [Concat(F ra * alpha, F rm * beta)]

[0050] Wherein, n, k represent the number of artifact feature maps, alpha, beta are self-learning parameters, W a , W m It is the weight of the network adaptive adjustment global maximum pooling and global average pooling.

[0051] Further, the alpha, beta, gamma are different variables between 0-1.

[0052] Compared with the prior art, the beneficial effects of the present application are:

[0053] (1) The present application uses the advantages of multi-scale feature extraction and gradient attention, and proposes a new end-to-end unsupervised line drawing extraction network model, which can extract high-quality colored drawing artifact line drawing.

[0054] (2) The present application introduces a scale enhancement module, which uses a group of dilated convolution with different expansion rates to realize the extraction of different scales of image features, increases the richness of image features, and extracts more detailed information in the line drawing.

[0055] (3) The present application proposes a new adaptive gradient attention fusion module, which uses powerful gradient information to guide the attention mechanism, enhances the network learning of image edges, and uses adaptive weights to fuse features, increases the difference between noise information and real line drawing, strengthens the network discrimination ability, and adaptively adjusts the network weight, while ensuring the integrity of the extracted features and suppressing the generation of noise information. BRIEF DESCRIPTION OF DRAWINGS

[0056] The following drawings only schematically illustrate and explain the present application, and do not limit the scope of the present application, wherein:

[0057] Figure 1 The generated adversarial network model structure diagram of the method of the present application;

[0058] Figure 2 is a structural diagram of a scale enhancement module;

[0059] Figure 3 is a structural diagram of an adaptive gradient attention fusion module;

[0060] Figure 4 is a clear colored drawing cultural relic line drawing extraction result; wherein, Figure 4 (a) is a colored drawing cultural relic image, Figure 4 (b) is a reference image, Figure 4 (c) is a Laplacian operator extracted line drawing image, Figure 4 (d) is an RCF method extracted line drawing image, Figure 4 (e) is a BDCN method extracted line drawing image, Figure 4 (f) is a Pix2Pix method extracted line drawing image, Figure 4 (g) is a CycleGAN method extracted line drawing image, Figure 4 (h) is a line drawing image extracted by the method of the present application;

[0061] Figure 5 is a colored drawing cultural relic line drawing extraction result of a disease; wherein, Figure 5 (a) is a colored drawing cultural relic image, Figure 5 (b) is a reference image, Figure 5 (c) is a Laplacian operator extracted line drawing image, Figure 5 (d) is an RCF method extracted line drawing image, Figure 5 (e) is a BDCN method extracted line drawing image, Figure 5 (f) is a Pix2Pix method extracted line drawing image, Figure 5 (g) is a CycleGAN method extracted line drawing image, Figure 5 (h) is a line drawing image extracted by the method of the present application;

[0062] wherein, in Figure 1 : ①->②: the cultural relic image obtains its gradient image;

[0063] ①->③: the cultural relic image is down-sampled and features are extracted;

[0064] ③->⑤: a cultural relic feature map is obtained;

[0065] ②->④: the gradient image is down-sampled and features are extracted;

[0066] ④->⑥: a gradient feature map is obtained;

[0067] ⑥->⑦: the gradient feature map is subjected to global average pooling to obtain a set of weights;

[0068] ⑥->⑧: the gradient feature map is subjected to global maximum pooling to obtain a set of weights;

[0069] ⑤->⑨: multiply the cultural relic feature map with the weight obtained in ⑦ one by one and accumulate to obtain an attention-bearing cultural relic feature map;

[0070] ⑤->⑩: multiply the cultural relic feature map with the weight obtained in ⑧ one by one and accumulate to obtain an attention-bearing cultural relic feature map;

[0071] The two groups of attention-bearing cultural relic feature maps obtained are multiplied by two self-learning parameters α and β respectively and then added to obtain a fused attention cultural relic feature map;

[0072] The original cultural relic feature map and the fused attention cultural relic feature map are multiplied by self-learning parameters γ and (1-γ) respectively and then added to obtain a final fused cultural relic feature map;

[0073] The final cultural relic feature map is up-sampled;

[0074] The model outputs a line drawing map. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical scheme, design method and advantages of the present application more clear and obvious, the present application is further described in detail below by combining with the specific embodiments and the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0076] In combination with Figure 1 As shown in the drawings, the present application provides a colored drawing cultural relic line drawing extraction method, which first classifies the collected colored drawing cultural relic dataset according to the disease erosion degree through a classification network, so that the sub-datasets of the same classification have highly similar styles; then the gradient map corresponding to the input image is calculated, both of which are used as network inputs, through a scale enhancement module to fully extract the features of different scales in the original colored drawing cultural relic image and the gradient image, to provide more detailed information for the network, then an attention mechanism is introduced into the network, the powerful gradient information is used to guide the attention mechanism, so that the network pays more attention to the object contour part, and adaptive weights are added, so that the network automatically adjusts according to the type characteristics of the cultural relics, accurately distinguishes noise and line drawing details, and thus suppresses the generation of noise information.

[0077] Specifically includes the following steps:

[0078] Step one, dataset pre-classification

[0079] The collected colored drawing cultural relic data set is divided into two categories according to the disease erosion degree: clearer and with disease by using a classification network such as AlexNet, so that the sub-data sets of the same category have highly similar styles.

[0080] Step two, multi-scale feature extraction

[0081] The gradient graph of the input image is calculated, two convolutional layers with scale enhancement modules are used to extract features from the input image and its gradient image respectively, and the encoded cultural relic feature graph Fr and gradient feature graph E are obtained g . The scale enhancement module captures multi-scale spatial context information by changing the receptive field size of the network neurons; specifically:

[0082] The input image I In is extracted by a convolution kernel with a size of 3*3 and a step of 2,

[0083] F0=Conv 3*3 (I In )

[0084] The obtained feature graph F0 is put into the scale enhancement module for multi-scale feature extraction. The scale enhancement module includes a normal convolution and three dilated convolutions with different dilation rates to extract image features of different scales, and finally the above results are superimposed and fused. The operation of the scale enhancement module includes:

[0085] F1=DConv 3*3 (F0,dilation=1)

[0086] F2=DConv 3*3 (F1,dilation=4*1)

[0087] F3=DConv 3*3 (F1,dilation=4*2)

[0088] F4=DConv 3*3 (F1,dilation=4*3)

[0089] F5=F1+F2+F3+F4

[0090] Where DConv 3*3 (·) represents a dilated convolution with different dilation rates, dilation represents the dilation rate, F1 represents the output feature of the first normal convolution, F2, F3, and F4 represent the output features of the first, second, and third dilated convolutions, respectively, and F5 represents the superimposed multi-scale feature graph.

[0091] In Figure 2In this module, the scale enhancement module consists of a regular convolution with a 3x3 kernel and a set of parallel dilated convolutions with different dilation rates. The input represents the shallow low-level feature F0. The input is first passed through a regular convolution with a 3x3 kernel to obtain the result. Then, the result is passed through 3x3 convolutions with dilation rates of 4x1, 4x2, and 4x3 respectively. The results are then stacked to obtain the multi-scale extracted feature map of the cultural relics.

[0092] Step 3, Image Translation

[0093] The encoded gradient feature map E obtained in step two g As input to the attention mechanism, two sets of weight vectors W are obtained. a W m The artifact feature map F obtained in step two r Corresponding features are multiplied and stacked, then adaptively fused into a global feature fusion layer with a 1*1 convolutional kernel to obtain an attention-based artifact feature map A guided by gradient information. r Adaptive weighting is used to process the newly obtained artifact feature map A. r Add to original artifact feature map F r Above, the final artifact feature map F was obtained. fuse The translated image I is obtained after upsampling. t ;Specifically:

[0094] Gradient feature map E g After global max pooling and global average pooling, two sets of feature vectors W based on the number of channels are obtained. a W m , as the weight vector. The two sets of weight vectors W are used as weight vectors. a W m With cultural relic feature diagram F r Corresponding features are multiplied and stacked, then adaptively fused into a global feature fusion layer with a 1*1 convolutional kernel to obtain gradient-guided, attention-based artifact feature map A. r ,Right now:

[0095]

[0096]

[0097] A r =Conv 1*1 [Concat(F ra *α,F rm *β)]

[0098] Where n and k represent the number of artifact feature maps, and α and β are self-learning parameters that enable the network to adaptively adjust the weights of global max pooling and global average pooling. The results are then stacked by the number of channels and fed into a 1*1 convolutional kernel for feature learning, reduction, and fusion, resulting in an attention-based artifact feature map A guided by gradient information. r .

[0099] Adaptive weights are used to apply attention to the artifact feature map A. r Add to original artifact feature map F r Above, the final artifact feature map F was obtained. fuse Specifically:

[0100] F fuse =F r *γ+A r *(1-γ)

[0101] Where γ is a self-learning parameter that enables the network to dynamically adjust the weight of the fusion of attention-based features and original artifact features based on the characteristics of the input image, and F fuse This is a diagram showing the characteristics of the artifacts after final integration.

[0102] This embodiment uses two types of painted cultural relic images to verify the effectiveness of the proposed line drawing extraction algorithm: one is a cultural relic image that is relatively well preserved and has less noise, and the other is a cultural relic image that is severely damaged and has a complex background.

[0103] To better evaluate the practicality of the fusion method, a comparative experiment was conducted in this embodiment. All training data were set to a size of 256*256, the network parameters were optimized using Adam, the number of iterations was 200, the batch size was set to 1, the initial learning rate was 0.0002, and the generative adversarial model was implemented in PyTorch using an NVIDIA GeForce RTX 2080 Ti GPU.

[0104] This embodiment mainly compares with five classic and novel line drawing extraction methods: the traditional edge operator Laplacian; deep learning methods based on edge detection, BDCN and RCF; and generative adversarial methods based on image translation, Pix2Pix and CycleGan.

[0105] Experimental Results Analysis:

[0106] Figure 4 To preserve relatively well-preserved painted cultural relics; Figure 4 (a) is an image of a painted cultural relic. Figure 4 (b) is a reference image. Figure 4 (c) to (g) are line art images of the five contrast methods, respectively. Figure 4(h) is the line drawing extraction result of the present method; by visually comparing the line drawing image and the reference image, it can be seen that for the colored drawing cultural relics with clear image and less noise, all methods can extract the approximate line drawing, but Figure 4 (d) (f) (g) The line drawing extracted by RCF, Pix2Pix and CycleGAN method has obvious detail loss, and the facial features of the characters and the texture of the clothes are not completely extracted; Figure 4 (c) (e) The line drawing extracted by Laplacian and BDCN method has serious artifacts and low clarity; and from Figure 4 (h) It can be seen from (h) that the line drawing image generated by the present application has the smallest difference from the reference image, and while effectively extracting the line drawing, more detailed information is retained.

[0107] Figure 5 For the colored drawing cultural relics with serious disease erosion and complex background, the experimental result graph, Figure 5 (a) is the image of the colored drawing cultural relics, Figure 5 (b) is the reference image, Figure 5 (c)-(g) are the line drawing images of five kinds of comparison methods respectively, Figure 5 (h) is the line drawing extraction result of the present method; by visually comparing the line drawing image and the reference image, it can be seen that, Figure 5 (c) is the extraction result of the traditional Laplacian operator, which has serious artifacts and low image clarity; Figure 5 (d) (g) The line drawing extracted by RCF and CycleGAN method has incomplete details, and the facial features of the characters and the outline of the horse's hair are not extracted, and there is noise, which has low accuracy; Figure 5 (e) (f) The line drawing extracted by BDCN and Pix2Pix method has relatively complete details, but the noise problem still exists; and from Figure 5 (h) It can be seen from (h) that the line drawing image generated by the present application has the smallest difference from the reference image, and not only the structure is complete, the detail richness is high, but also the noise is effectively suppressed.

[0108] It is difficult to give the most correct judgment on the fusion result only by subjective evaluation, so it is necessary to combine objective indexes to evaluate the line drawing extraction result; the three objective evaluation indexes of SSIM, AP and RMSE are used to comprehensively evaluate the image; wherein, the structural similarity (SSIM) reflects the structural properties of objects in the scene, and is used to evaluate the structural and style similarity between the generated line drawing and the reference picture; the larger the SSIM value is, the better the similarity is; the average precision (AP) is a pixel-level index, which represents the average value of the precision of different recall points, and reflects the global performance; the higher the AP is, the more effective the model method is; the mean square error (RMSE) reflects the difference degree between pixel-level variables, and is used to measure the error between the generated line drawing and the reference picture; the smaller the RMSE value is, the better the performance is.

[0109] Table 1 objective indexes of line drawing extraction result of clear colored drawing cultural relic image

[0110]

[0111] Table 2 objective indexes of line drawing extraction result of colored drawing cultural relic with serious disease erosion

[0112]

[0113] Table 1 and Table 2 are objective indexes of the line drawing extraction result of different methods in the experiments of clear cultural relic image and cultural relic image with disease respectively; from Table 1 and Table 2, it can be seen that all the objective indexes of the present application are better than those of other methods, that is, the line drawing image generated by the method of the present application has the strongest correlation with the reference image, and can well extract the line drawing image of the colored drawing cultural relic with different styles caused by diseases and the like.

[0114] Comprehensive visual evaluation and objective index evaluation, the algorithm proposed in the present application can well obtain the line drawing image with rich details and low noise.

[0115] The above has described the embodiments of the present application, the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles, practical application or technical improvement in the market of the embodiments, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for extracting a colored drawing historical document line drawing, characterized by, The method comprises the following steps: Step 1: data set pre-classification; Step 2: collecting painted cultural relic pictures, classifying the painted cultural relic data set through a classification network model to obtain a clear data set and a disease data set; Step 3: establishing a generative adversarial model; Step 4: training the generative adversarial model; Step 5: respectively using the test sets of the clear data set and the disease data set to train the generative adversarial model to obtain a first generative adversarial model and a second generative adversarial model; Step 6: using the trained first generative adversarial model and the second generative adversarial model to extract line drawings; The generative adversarial model comprises: A scale enhancement module: used for multi-scale extraction of features, comprising one ordinary convolution and three dilated convolutions with different dilation rates; A feature extraction module: used for extracting features of input images and gradient images respectively; comprising two 3*3 convolutions and nine residual blocks; An adaptive gradient attention fusion module: used for obtaining a cultural relic feature map with attention guided by gradient information; comprising an attention module and an adaptive fusion module, wherein the attention module is composed of two fully connected layers, self-learning parameters α and β, a cat() function and a 1*1 convolution; the adaptive fusion module comprises a self-learning parameter γ; An up-sampling module: comprising two 3*3 convolutions; The training steps of the generative adversarial model comprise: S3.1: input the clear data set and the disease data set as input pictures I In input the generative adversarial model; S3.2: extracting a gradient map of the painted cultural relic picture; S3.3: down-sampling the cultural relic picture and the gradient picture respectively and extracting features by using a feature extraction module to obtain a cultural relic feature map F r and a gradient feature map E g ; S3.4: the gradient feature map E g Global max pooling and global average pooling are performed to obtain two groups of weight vectors W a , W m ; S3.5: the cultural relic feature map F r is multiplied and superimposed with W a , W m respectively to obtain the cultural relic feature map F ra after global average pooling and the cultural relic feature map F rm after global maximum pooling, and the cultural relic feature map F ra after global average pooling and the cultural relic feature map F rm after global maximum pooling are put into a global feature fusion layer with a convolution kernel of 1*1 for adaptive fusion to obtain the cultural relic feature map A r with attention guided by gradient information; S3.6: The obtained two groups of cultural relic feature maps A r The two self-learning parameters α and β are multiplied and superimposed to obtain the cultural relic feature map F' with attention, and the calculation process is as follows: F' = f (a x F + b x F ra + b x F rm ) S3.7: E (Image of cultural relic features) r The fused artifact feature map is obtained by multiplying the fused attention artifact feature map by the self-learning parameters γ and (1-γ) respectively, and then adding them together. fuse The calculation process is as follows: F fuse = (1 - γ) x F' + γ x F r S3.8: to the fused artifact feature map F fuse up-sampling; S3.9: outputting a line drawing, completing one training; S3.10: repeating steps S3.1-S3.9 to obtain a first generative adversarial model for outputting a clear data set line drawing and a second generative adversarial model for outputting a disease data set line drawing.

2. The method of claim 1, wherein the method is characterized by: The classification process of the painted cultural relic picture comprises: Using the previously classified painted cultural relic pictures to train the classification network model to obtain a trained classification network model; Inputting the collected painted cultural relic pictures to be classified into the trained classification network model to classify the painted cultural relic pictures to be classified.

3. The method of claim 2, wherein the method further comprises: In step S3.3, the feature extraction comprises: The input image I In is first extracted by a convolution kernel with a size of 3*3 and a step of 2 to obtain a shallow low-level feature F0 of the input image: F0 = Conv 3*3 (I In ) The obtained feature map F0 is put into the scale enhancement module for multi-scale extraction of features, the scale enhancement module comprises one ordinary convolution and three dilated convolutions with different dilation rates to extract image features of different scales, and finally the output results of the ordinary convolution and the three dilated convolutions in the scale enhancement module are superimposed and fused; the operation of the scale enhancement module comprises: F1 = DConv 3*3 (F0, dilation=1) F2 = DConv 3*3 (F1, dilation=4*1) F3 = DConv 3*3 (F1, dilation=4*2) F4 = DConv 3*3 (F1, dilation=4*3) F5=F1+F2+F3+F4 where DConv 3*3 (·) represents a dilated convolution with different dilation rates, d represents a dilation rate, F1 represents the output feature of the first normal convolution, F2, F3, and F4 represent the output features of the first, second, and third dilated convolutions, respectively, and F5 represents the stacked multi-scale feature map.

4. The method of claim 1, wherein the method is characterized by: S3.5 said artifact feature map F r respectively multiplied by W a , W m and superimposed by the formula: The gradient information guided attention-based artifact feature map A r The calculation formula is: A r = Conv 1*1 [Concat(F ra *α,F rm *β)] wherein n, k represent the number of cultural relic feature maps, and a, β are self-learning parameters set, W a , W m is the weight of network adaptive adjustment of global max pooling and global average pooling.

5. The method of claim 1, wherein the method is characterized by: The α, β and γ are different variables between 0 and 1.

Images