Ancient painting damage detection method

Abstract

An ancient painting damage detection method comprises the steps of collecting ancient painting damage images, constructing an ancient painting damage detection data set, dividing the data set, constructing an ancient painting damage detection network, extracting features of the ancient painting damage images, training the ancient painting damage detection network and testing the ancient painting damage detection network. Due to the fact that the dynamic self-adaptive convolution and global information fusion module is adopted, the damaged area in the damaged ancient painting image is detected, the technical problems that in the prior art, ancient painting damage detection is inaccurate and not attached are solved, position information of the damaged area of the ancient painting can be effectively recognized and positioned, the prediction frame can be adjusted in a self-adaptive mode, and the prediction accuracy is improved. Damage positions of different types of ancient paintings are identified, and the detection precision and the perception identification capability of the network are improved. The method has the advantages of high detection speed, high precision and the like, and can be used for computer detection of ancient painting damage.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and artificial intelligence, and specifically relates to a method for detecting damaged positions of damaged ancient paintings. Background Art

[0002] Damage detection of ancient calligraphy and paintings is to automatically detect and mark damaged areas in ancient painting images. In the original image, rectangular boxes composed of lines are used to mark the locations of different damages in the ancient paintings, where each rectangular box contains information such as location and size. Traditional methods of damage detection of ancient paintings require manual recognition. When identifying ancient paintings, it is difficult to accurately identify and understand the difference between the background and damage in ancient paintings due to the particularity of the detection target. This is mainly because ancient paintings have rich details, lines and colors, as well as unique painting styles, which require computer vision and deep learning technologies for effective detection.

[0003] Existing deep learning methods, such as Faster R-CNN, Single Shot MultiBox Detector (SSD) and other models, have high accuracy, but they have certain limitations in detecting damaged areas of ancient paintings because they cannot fully understand the color and line information of ancient paintings. Specifically, this type of target detection method is usually divided into two stages: candidate region generation and target classification. Single-stage target detectors, such as the Single Shot MultiBox Detector model, directly perform dense predictions on the input image, avoiding the complex candidate region generation process. However, the damage, mildew, and missing of ancient paintings are beyond the design scope of the Single Shot MultiBox Detector model, making it impossible to detect effectively. For the detection of damaged ancient paintings, due to the artistry and complexity of damaged ancient paintings, it becomes very difficult for images such as ancient paintings with rich artistic styles and multiple complex situations.

[0004] In the field of ancient painting damage detection, a technical problem that urgently needs to be solved is to enable computers to automatically detect complex information about ancient painting damage. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method for detecting damage to ancient paintings with fast detection speed and high accuracy.

[0006] The technical solution steps adopted to solve the above technical problems are as follows:

[0007] (1) Collecting damaged images of ancient paintings

[0008] The original electronic images of ancient Chinese paintings with damages were obtained from online digital museums and resized to damaged images of ancient paintings with a height H and a width W of 512×512 pixels.

[0009] (2) Constructing a dataset for ancient painting damage detection

[0010] The selected damaged images of ancient paintings are labeled using the intelligent interactive detection and annotation software labelimg according to the software usage method, and a txt file containing the location information of the damaged area in each damaged image of ancient paintings and class information is obtained.

[0011] (3) Dividing the Dataset

[0012] The annotated damaged images of ancient paintings are divided into training set, validation set and test set according to the ratio of 8:1:1.

[0013] (4) Constructing an ancient painting damage detection network

[0014] The ancient painting damage detection network is composed of a feature extraction network, a feature fusion module 1, and an information distribution and aggregation module connected in series.

[0015] The feature fusion module 1 is composed of a backbone unit, a Head unit, and an attention fusion module connected in series.

[0016] The information distribution and aggregation module is composed of a feature alignment module, a feature fusion module 2 and an injection module connected in series.

[0017] (5) Extracting features of damaged images of ancient paintings

[0018] The damaged image feature φ(I) of ancient paintings is extracted as follows:

[0019] φ(I)=F b (I,θ b )

[0020] Among them, F b is the feature extraction network, I is the damaged image in the training set, θ b are the parameters of the feature extraction network, including weights and biases.

[0021] (6) Training ancient painting damage detection network

[0022] 1) Construct loss function

[0023] The loss function is constructed as follows

[0024]

[0025]

[0026] Among them, IoU is the overlap area ratio between the predicted box and the real box, and the confidence threshold is obtained based on this, w and h are the height and width of the input image, A is the real box, B is the predicted box, are the coordinates of the upper left and lower right corners of the prediction box, respectively. They are the coordinates of the upper left corner and lower right corner of the real box respectively.

[0027] 2) Training ancient painting damage detection network

[0028] The training set is input into the ancient painting damage detection network for training. The server graphics card for training is RTX 3090, the initial learning rate is 0.01, the number of training rounds is 200, the training batch size is 32, and the loss function is optimized through the validation set. The training is performed until the loss function convergence.

[0029] (7) Testing the Ancient Painting Damage Detection Network

[0030] The test set is input into the trained ancient painting damage detection network for testing, the damaged location in the image is detected, and a labeled rectangular box is output to mark the damaged location in the image.

[0031] In step (4) of constructing an ancient painting damage detection network of the present invention, the backbone unit is composed of 1 to 4 dynamic convolution modules and 1 to 4 cross-layer link modules connected in series.

[0032] The dynamic convolution module of the present invention is composed of a dynamic convolution layer 1, a topology constraint layer, and a ReLU activation function layer connected in series in sequence.

[0033] The cross-layer link module of the present invention is composed of a convolutional layer 1, a Block layer, and a splicing layer 1 connected in series in sequence.

[0034] In step (4) of the present invention, in constructing an ancient painting damage detection network, the feature alignment module is composed of a dynamic convolution layer 2, an average pooling layer, and a Transformer module connected in series in sequence.

[0035] In step (4) of the present invention, in constructing an ancient painting damage detection network, the injection module is composed of a dynamic convolution layer 3, a bilinear interpolation layer, and a RepBlock layer connected in series in sequence.

[0036] Since the ancient painting damage detection network of the present invention adopts dynamic adaptive convolution and global information fusion modules, it realizes the detection of damaged areas in damaged ancient painting images, solves the technical problems of inaccurate and non-fitting ancient painting damage detection in the prior art, can effectively identify and locate the position information of damaged areas of ancient paintings, can adaptively adjust the prediction frame, identify different types of damaged ancient painting locations, and improve the detection accuracy and perception recognition ability of the network. The present invention has the advantages of fast detection speed and high accuracy, and can be used for computer detection of ancient painting damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of embodiment 1 of the present invention.

[0038] Figure 2 It is a structural diagram of the ancient painting damage detection network.

[0039] Figure 3 yes Figure 2 Schematic diagram of the structure of feature fusion module 1.

[0040] Figure 4 yes Figure 3 Schematic diagram of the structure of the central backbone unit.

[0041] Figure 5 yes Figure 3 Schematic diagram of the structure of the Head unit

[0042] Figure 6 yes Figure 3 Schematic diagram of the structure of the attention fusion module.

[0043] Figure 7 yes Figure 2 Schematic diagram of the structure of the information distribution and aggregation module.

[0044] Figure 8 This is a graph showing the experimental results of comparing the detection method of the present invention with the original image. DETAILED DESCRIPTION

[0045] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0046] Example 1

[0047] exist Figure 1 In the present embodiment, the steps of the method for detecting damage to ancient paintings are as follows:

[0048] (1) Collecting damaged images of ancient paintings

[0049] The original electronic images of ancient Chinese paintings with damages were obtained from online digital museums and resized to damaged images of ancient paintings with a height H and a width W of 512×512 pixels.

[0050] (2) Constructing a dataset for ancient painting damage detection

[0051] The selected damaged images of ancient paintings are labeled using the intelligent interactive detection and annotation software labelimg according to the software usage method, and a txt file containing the location information of the damaged area in each damaged image of ancient paintings and class information is obtained.

[0052] (3) Dividing the Dataset

[0053] The annotated damaged images of ancient paintings are divided into training set, validation set and test set according to the ratio of 8:1:1.

[0054] (4) Constructing an ancient painting damage detection network

[0055] Figure 2 A schematic diagram of the structure of the damage detection network of the present invention is given. Figure 2 In the present embodiment, the ancient painting damage detection network is composed of a feature extraction network, a feature fusion module 1, and an information distribution and aggregation module connected in series.

[0056] Figure 3 Given Figure 2 Schematic diagram of the structure of feature fusion module 1. Figure 3 In the embodiment, the feature fusion module 1 is composed of a backbone unit, a Head unit, and an attention fusion module connected in series in sequence.

[0057] Figure 4 Given Figure 3 Schematic diagram of the structure of the backbone unit. Figure 4 In the embodiment, the backbone unit is composed of 1 to 4 dynamic convolution modules and 1 to 4 cross-layer link modules connected in series, and the backbone unit is composed of 1 dynamic convolution module and 1 cross-layer link module connected in series.

[0058] The dynamic convolution module of this embodiment is composed of a dynamic convolution layer 1, a topology constraint layer, and a ReLU activation function layer connected in series in sequence.

[0059] The cross-layer link module of this embodiment is composed of a convolutional layer 1, a Block layer, and a splicing layer 1 connected in series in sequence.

[0060] Figure 5 Given Figure 3 Schematic diagram of the structure of the Head unit. Figure 5 In the embodiment, the Head unit is composed of an upsampling layer, a C2f layer, and a splicing layer 2 connected in series in sequence. The C2f layer is a well-known technology and has been disclosed on the website https: / / docs.ultralytics.com / .

[0061] Figure 6 Given Figure 3 Schematic diagram of the structure of the attention fusion module. Figure 6 In the embodiment, the attention fusion module is composed of a deep convolutional layer, a deep scalable convolutional layer, a convolutional layer 2, and a Hadamard convolutional layer connected in series in sequence.

[0062] Figure 7 Given Figure 2 The structural diagram of the information distribution and aggregation module in Fig. Figure 7 In this embodiment, the information aggregation and distribution module is composed of a feature alignment module, a feature fusion module 2, and an injection module connected in series.

[0063] The feature alignment module of this embodiment is composed of a dynamic convolution layer 2, an average pooling layer, and a Transformer module connected in series.

[0064] The feature fusion module 2 of this embodiment is composed of a RepBlock layer and a segmentation layer connected in series.

[0065] The injection module of this embodiment is composed of a dynamic convolution layer 3, a bilinear interpolation layer, and a RepBlock layer connected in series in sequence.

[0066] Since the ancient painting damage detection network of the present invention adopts dynamic adaptive convolution and global information fusion modules, it can detect the damaged areas in the damaged ancient painting images, solve the technical problems of inaccurate and non-fitting ancient painting damage detection in the prior art, and can effectively identify and locate the position information of the damaged areas of the ancient paintings. It can adaptively adjust the prediction frame and identify the damaged locations of different types of ancient paintings, thereby improving the detection accuracy and perception and recognition capabilities of the network.

[0067] (5) Extracting features of damaged images of ancient paintings

[0068] The damaged image feature φ(I) of ancient paintings is extracted as follows:

[0069] φ(I)=F b (I,θ b )

[0070] Among them, F b is the feature extraction network, I is the damaged image in the training set, θ b are the parameters of the feature extraction network, including weights and biases.

[0071] (6) Training ancient painting damage detection network

[0072] 1) Construct loss function

[0073] The loss function is constructed as follows

[0074]

[0075]

[0076] Among them, IoU is the overlap area ratio between the predicted box and the real box, and the confidence threshold is obtained based on this, w and h are the height and width of the input image, A is the real box, B is the predicted box, are the coordinates of the upper left and lower right corners of the prediction box, respectively. They are the coordinates of the upper left corner and lower right corner of the real box respectively.

[0077] 2) Training ancient painting damage detection network

[0078] The training set is input into the ancient painting damage detection network for training. The server graphics card for training is RTX 3090, the initial learning rate is 0.01, the number of training rounds is 200, the training batch size is 32, and the loss function is optimized through the validation set. The training is performed until the loss function convergence.

[0079] (7) Testing the Ancient Painting Damage Detection Network

[0080] The test set is input into the trained ancient painting damage network for testing, the damaged location in the image is detected, and a labeled rectangular box is output to mark the damaged location in the image.

[0081] Example 2

[0082] The steps of the ancient painting damage detection method of this embodiment are as follows:

[0083] (1) Collecting damaged images of ancient paintings

[0084] This step is the same as in Example 1.

[0085] (2) Constructing a dataset for ancient painting damage detection

[0086] This step is the same as in Example 1.

[0087] (3) Dividing the Dataset

[0088] This step is the same as in Example 1.

[0089] (4) Constructing an ancient painting damage detection network

[0090] The ancient painting damage detection network of this embodiment is composed of a feature extraction network, a feature fusion module 1, and an information distribution and aggregation module connected in series. The feature fusion module 1 is composed of a backbone unit, a head unit, and an attention fusion module connected in series. The backbone unit is composed of 1 to 4 dynamic convolution modules and 1 to 4 cross-layer link modules connected in series. The backbone unit of this embodiment is composed of 2 dynamic convolution modules and 2 cross-layer link modules connected in series. The structure of the dynamic convolution module and the cross-layer link module is the same as that of Example 1.

[0091] The other structures of the ancient painting damage detection network are the same as those in Example 1.

[0092] The other steps are the same as those in Example 1. The method for detecting damage of ancient paintings is completed.

[0093] Example 3

[0094] The steps of the ancient painting damage detection method of this embodiment are as follows:

[0095] (1) Collecting damaged images of ancient paintings

[0096] This step is the same as in Example 1.

[0097] (2) Constructing a dataset for ancient painting damage detection

[0098] This step is the same as in Example 1.

[0099] (3) Dividing the Dataset

[0100] This step is the same as in Example 1.

[0101] (4) Constructing an ancient painting damage detection network

[0102] The ancient painting damage detection network of this embodiment is composed of a feature extraction network, a feature fusion module 1, and an information distribution and aggregation module connected in series. The feature fusion module 1 is composed of a backbone unit, a head unit, and an attention fusion module connected in series. The backbone unit is composed of 1 to 4 dynamic convolution modules and 1 to 4 cross-layer link modules connected in series. The backbone unit of this embodiment is composed of 4 dynamic convolution modules and 4 cross-layer link modules connected in series. The structure of the dynamic convolution module and the cross-layer link module is the same as that of Example 1.

[0103] The other structures of the ancient painting damage detection network are the same as those in Example 1.

[0104] The other steps are the same as those in Example 1. The method for detecting damage of ancient paintings is completed.

[0105] In order to verify the beneficial effects of the present invention, a computer simulation experiment was carried out using the ancient painting damage detection method of Example 1 of the present invention. The experimental results are shown in FIG. Figure 8 .exist Figure 8 Among them, Figure a is the damaged picture of Yangniao Youju Picture, Figure b is the damage detection effect picture of Yangniao Youju Picture, Figure c is the damaged picture of Yanlan Xiaosi Picture, Figure d is the damage detection effect picture of Yanlan Xiaosi Picture, Figure e is the damaged picture of Jiangshan Zaochun Picture, Figure f is the damage detection effect picture of Jiangshan Zaochun Picture, Figure g is the damaged picture of Antique Landscape Album, and Figure h is the damage detection effect picture of Antique Landscape Album. In Figures b, d, f, and h, the red rectangular box marks the position and size of the damage in the picture.

[0106] Depend on Figure 8It can be seen that the method of the present invention can not only effectively identify and locate damage information in ancient paintings, but also adaptively adjust the network to adapt to ancient paintings with different damage types, thereby improving the intelligent detection capability of the network.

Claims

1. A method for detecting damage to ancient paintings, characterized in that It consists of the following steps: (1) Collecting damaged images of ancient paintings The original electronic images of ancient Chinese paintings with damages are obtained from online digital museums and adjusted to damaged images of ancient paintings with a height H and a width W of 512×512 pixels; (2) Constructing a dataset for ancient painting damage detection The damaged images of ancient paintings are selected using the intelligent interactive detection and annotation software labelimg. According to the software usage method, the damaged images of ancient paintings are annotated to obtain a txt file containing the location information of the damaged area in each damaged image of ancient paintings, as well as class information; (3) Dividing the Dataset The annotated damaged images of ancient paintings are divided into training set, validation set and test set according to the ratio of 8:1:1; (4) Constructing an ancient painting damage detection network The ancient painting damage detection network is composed of a feature extraction network, a feature fusion module 1, and an information distribution and aggregation module in series; The feature fusion module 1 is composed of a backbone unit, a head unit, and an attention fusion module connected in series in sequence; The information distribution and aggregation module is composed of a feature alignment module, a feature fusion module 2, and an injection module in series; (5) Extracting features of damaged images of ancient paintings The damaged image feature φ(I) of the ancient painting is extracted as follows: φ(I)=F b (1,θ b ) Among them, F b is the feature extraction network, I is the damaged image in the training set, θ b are the parameters of the feature extraction network, including weights and biases; (6) Training ancient painting damage detection network 1) Construct loss function The loss function is constructed as follows Among them, IoU is the overlap area ratio between the predicted box and the real box, and the confidence threshold is obtained based on this, w and h are the height and width of the input image, A is the real box, B is the predicted box, are the coordinates of the upper left and lower right corners of the prediction box, respectively. They are the coordinates of the upper left corner and lower right corner of the real box respectively; 2) Training ancient painting damage detection network The training set is input into the ancient painting damage detection network for training. The server graphics card for training is RTX 3090, the initial learning rate is 0.01, the number of training rounds is 200, the training batch size is 32, and the loss function is optimized through the validation set. The training is performed until the loss function convergence; (7) Testing the Ancient Painting Damage Detection Network The test set is input into the trained ancient painting damage detection network for testing, the damaged location in the image is detected, and a labeled rectangular box is output to mark the damaged location in the image.

2. The method for detecting damage to ancient paintings according to claim 1, characterized in that: In step (4) of constructing the ancient painting damage detection network, the backbone unit is composed of 1 to 4 dynamic convolution modules and 1 to 4 cross-layer link modules connected in series.

3. The method for detecting damage to ancient paintings according to claim 2, characterized in that: The dynamic convolution module is composed of a dynamic convolution layer 1, a topology constraint layer, and a ReLU activation function layer connected in series in sequence.

4. The method for detecting damage to ancient paintings according to claim 2, characterized in that: The cross-layer link module is composed of a convolutional layer 1, a Block layer, and a splicing layer 1 connected in series in sequence.

5. The method for detecting damage to ancient paintings according to claim 1, characterized in that: In step (4) of constructing the ancient painting damage detection network, the feature alignment module is composed of a dynamic convolution layer 2, an average pooling layer, and a Transformer module connected in series.

6. The method for detecting damage to ancient paintings according to claim 1, characterized in that: In step (4) of constructing the ancient painting damage detection network, the injection module is composed of a dynamic convolution layer 3, a bilinear interpolation layer, and a RepBlock layer connected in series in sequence.

Images