A hyperspectral mural image defect detection method based on segmented channel attention

Abstract

The present invention provides a hyperspectral mural image defect detection method based on split channel attention, which is used to solve the technical problem that the existing deep learning-based algorithms have poor performance in multi-scale feature extraction, cannot accurately extract defects at all scales of murals, and are difficult to meet the needs of high-precision, large-scale mural image defect detection. The present invention provides a hyperspectral mural image defect detection method based on split channel attention, which integrates the split channel attention mechanism into a neural network model, constructs a neural network model based on split channel attention, and divides the hyperspectral mural image to be tested into multiple groups of images containing information of different bands according to the band. At the same time, each group of images is convolved with different convolution kernel sizes, which can accurately extract defects at all scales of the mural, effectively improve the detection accuracy and efficiency, and has good practicality.

Description

Technical Field

[0001] The invention relates to mural defect detection, and in particular to a hyperspectral mural image defect detection method based on segmentation channel attention. Background Art

[0002] In the field of cultural heritage protection, murals, as one of the precious heritages of ancient art, carry important information about history, culture and art. However, due to the passage of time and the influence of the external environment, murals often have various defects, such as cracks, fading, paint peeling, etc., which seriously affect their preservation and ornamental value. Therefore, it is of great significance to detect and repair mural defects in a timely manner.

[0003] Mural defect detection is an important cultural relic protection work, which aims to detect defects on the surface of murals in a timely manner, so as to take measures to repair them and protect the integrity of cultural heritage. In the past, cultural relic protection work mainly relied on manual inspections and traditional restoration methods, which had problems such as low efficiency, strong subjectivity and unsustainability. With the continuous development of digital technology, especially the application of image processing, infrared photography, laser scanning, computer vision and deep learning, mural defect detection has entered a new stage of development. These new technologies provide new possibilities for the rapid and accurate detection of mural defects, and provide cultural relic protection workers with more powerful tools and methods.

[0004] At present, the industry usually uses computer vision-based algorithms and deep learning-based algorithms for mural defect detection. However, it is difficult for computer vision-based algorithms to fully model and migrate defect features, and they are not very reusable. They also require different working conditions, which will waste a lot of manpower costs. Although deep learning-based algorithms have better effects in feature extraction and positioning, the causes of mural defects are complex and diverse, resulting in complex and diverse defect sizes and forms. It is necessary to comprehensively consider multiple factors for analysis and judgment. Existing deep learning-based algorithms perform poorly in multi-scale feature extraction and cannot accurately extract defects at all scales of murals, making it difficult to meet the needs of high-precision, large-scale mural image defect detection. Summary of the invention

[0005] The purpose of the present invention is to solve the technical problems that the existing deep learning-based algorithms perform poorly in multi-scale feature extraction, cannot accurately extract defects at all scales of murals, and are difficult to meet the needs of high-precision, large-scale mural image defect detection, and provide a hyperspectral mural image defect detection method based on segmented channel attention.

[0006] In order to achieve the above purpose, the technical solution provided by the present invention is as follows:

[0007] A hyperspectral mural image defect detection method based on segmented channel attention is special in that it includes the following steps:

[0008] Step 1, construct a neural network model based on segmentation channel attention;

[0009] The neural network model includes a first segmentation channel attention module, a normalization layer, a second segmentation channel attention module and a linear classification layer which are arranged in sequence according to the input and output order;

[0010] The first segmentation channel attention module is used to receive the input hyperspectral mural image to be tested, and segment the hyperspectral mural image to be tested into multiple groups of images containing information of different bands according to the band, and convolve each group of images with different convolution kernel sizes to extract different scale features of the hyperspectral mural image to be tested, so as to improve the comprehensive performance of the neural network model on different scale features; the standardization layer is used to eliminate the adverse effects caused by singular sample data, and accelerate convergence and alleviate gradient diffusion problems during training; the second segmentation channel attention module is used to further extract different scale features; the linear classification layer is used to output the abnormal probability of each pixel in the hyperspectral mural image to be tested;

[0011] Step 2: Model training;

[0012] Cut out a part of the hyperspectral mural image to be tested as a training sample, input it into the neural network model constructed in step 1 for model training, and obtain a trained neural network model;

[0013] Step 3, image cutting;

[0014] Cut the hyperspectral mural image to be tested to obtain multiple sub-images of the same size, and record and store the position information of each sub-image;

[0015] Step 4, anomaly detection;

[0016] The multiple sub-images that have been cut are sequentially input into the neural network model trained in step 2, and the neural network model assigns an abnormal probability to each pixel point on each sub-image; according to the abnormal probability of each pixel point on each sub-image, it is determined and marked whether each pixel point is abnormal, and multiple sub-images with marked results are obtained;

[0017] Step 5, output of defect detection results;

[0018] Multiple sub-images with marked results are spliced ​​according to the corresponding position information to obtain the defect detection result of the hyperspectral mural image to be tested.

[0019] Furthermore, in step 1, the first segmentation channel attention module includes a segmentation module, n convolutional layers with different convolution kernel sizes, n spatial compression modules and a weighting module, and the value of n is equal to the number of bands contained in the hyperspectral mural image;

[0020] The input end of the segmentation module is used to receive the input hyperspectral mural image to be tested, and n channels are arranged inside the segmentation module, which is used to segment the hyperspectral mural image to be tested into n groups of images through the n channels according to the waveband;

[0021] The input ends of the n convolutional layers are respectively connected to the output ends of the n channels of the segmentation module, so as to perform convolutions of different convolution kernel sizes on the n groups of images respectively, and obtain n groups of different attention vectors;

[0022] The input ends of the n spatial compression modules are respectively connected to the output ends of the n convolutional layers, so as to respectively obtain the average values ​​of the n groups of attention vectors in the spatial domain, calculate the corresponding adaptive weight for each group of attention vectors, and obtain the features of n different scales of the hyperspectral mural image to be tested;

[0023] The input end of the weighting module is connected to the output ends of the n spatial compression modules and the output ends of the n convolutional layers, and is used to weight the n channels according to the adaptive weights obtained from each group of attention vectors to achieve attention allocation of features of different scales, and its output end is connected to the input end of the normalization layer;

[0024] The second segmentation channel attention module has the same structure as the first segmentation channel attention module, the input end of its segmentation module is connected to the output end of the normalization layer, and the output end of its weighting module is connected to the input end of the linear classification layer.

[0025] Furthermore, in step 1, the activation functions of the first segmentation channel attention module, the normalization layer, the second segmentation channel attention module and the linear classification layer all use the leaky_relu function.

[0026] Furthermore, in step 2, the training loss function during model training is L for:

[0027]

[0028] In the above formula, N represents the total number of pixels in the training sample. Indicates The label value of each pixel, Indicates that the detection The probability that a pixel is an abnormality.

[0029] Furthermore, in step 2, the optimizer during model training uses the Adam function.

[0030] Furthermore, in step 4, determining whether each pixel point is abnormal according to the abnormal probability of each pixel point on each sub-image is specifically as follows:

[0031] Compare the abnormal probability of each pixel on each sub-image With setting threshold The size of , it means that the pixel is abnormal, otherwise it means that the pixel is normal.

[0032] Furthermore, in step 4, the threshold is set =0.8.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention provides a hyperspectral mural image defect detection method based on split channel attention. The split channel attention mechanism is integrated into a neural network model to construct a neural network model based on split channel attention. The hyperspectral mural image to be tested is divided into multiple groups of images containing information of different bands according to the bands. At the same time, convolution of different convolution kernel sizes is performed on each group of images. The defects of the murals at all scales can be accurately extracted, and the detection accuracy and efficiency are effectively improved. The method has good practicality and promotes the widespread application of digital technology in the field of cultural relics protection. It not only helps to strengthen the protection of cultural heritage such as murals, but also provides new ideas and methods for the inheritance and development of traditional culture. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a hyperspectral mural image to be tested in an embodiment of a hyperspectral mural image defect detection method based on segmented channel attention of the present invention;

[0036] Figure 2 for Figure 1 A partial enlarged view of

[0037] Figure 3 It is a flow chart of an embodiment of a hyperspectral mural image defect detection method based on segmented channel attention of the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of the neural network model in step 1 of the embodiment of the present invention;

[0039] Figure 5 Schematic diagram of the principle of the first segmentation channel attention module in step 1 of the embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of image cutting in step 3 of an embodiment of the present invention;

[0041] Figure 7This is a schematic diagram of the defect detection results obtained in step 5 of an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the advantages and features of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] After being left unused for hundreds or thousands of years, murals will be damaged for various reasons. For example, sulfur dioxide produced by burning fossil fuels will cause the painting layer to fall off. Another example is that the calcium carbonate component in the mud layer will be lost as the water evaporates slowly, causing the pores to gradually increase and unable to support the painting layer, causing the painting layer to fall off. In addition, excessive humidity, earthquakes, human damage and other reasons will damage the murals to varying degrees, resulting in various defects that affect the beauty of the murals and archaeological research. Figure 1 As shown, it is a hyperspectral mural image that needs to be tested for mural defects in this embodiment, that is, a hyperspectral mural image to be tested; Figure 2 for Figure 1 A partial enlarged view, in which the black box shows some defects in the mural caused by the peeling of the painting layer.

[0044] In order to better detect these defects in murals, this paper proposes a hyperspectral mural image defect detection method based on segmented channel attention, such as Figure 3 As shown, the specific steps include:

[0045] Step 1: Build a neural network model based on segmentation channel attention.

[0046] In order to solve the problem of being unable to accurately extract defects at all scales of murals, the present invention constructs a neural network model based on segmentation channel attention, such as Figure 4 As shown, it includes a first segmentation channel attention module, a normalization layer, a second segmentation channel attention module and a linear classification layer which are arranged in the order of input and output.

[0047] The first segmentation channel attention module includes a segmentation module, n convolutional layers with different convolution kernel sizes, n spatial compression modules and a weighting module, wherein the value of n is equal to the number of bands of the hyperspectral mural image to be tested.

[0048] like Figure 5As shown, the input end of the segmentation module is used to receive the input hyperspectral mural image to be tested, and n channels are set inside it, which are respectively recorded as channel 0, channel 1, ..., channel n, and are used to segment the hyperspectral mural image to be tested into n groups of images through n channels according to the band. The n groups of images carry the information of each band of the hyperspectral mural image to be tested. The n convolutional layers use n convolution kernels of different scales, which are correspondingly recorded as convolution kernel 0, convolution kernel 1, ..., convolution kernel n; the input ends of the n convolutional layers are respectively connected to the output ends of the n channels of the segmentation module, and are used to use convolution kernels of different scales to convolve the n groups of images output by the n channels, so as to extract different scale features of the hyperspectral mural image to be tested and obtain n groups of different attention vectors. The input ends of the n spatial compression modules are respectively connected to the output ends of the n convolutional layers, and are used to compress the spatial dimension, that is, to obtain the average values ​​of the n groups of attention vectors in the spatial domain, and calculate the corresponding adaptive weights for each group of attention vectors. The n adaptive weights calculated at this time are the features of each group of attention vectors, which are correspondingly recorded as feature 0, feature 1, ..., feature n. The input end of the weighting module is connected to the output ends of the n spatial compression modules and the output ends of the n convolutional layers, and is used to weight the attention vectors of the n channels according to the features of each group of attention vectors, so as to improve the comprehensive performance of the neural network model on features of different scales. Therefore, the neural network model based on the split channel attention of the present invention can adaptively choose to assign different weights to convolution kernels of different scales, and then adaptively detect defects of different scales, avoiding the influence of convolution kernels of inappropriate scales.

[0049] The input end of the normalization layer is connected to the output end of the weighting module, which is used to eliminate the adverse effects caused by singular sample data, accelerate convergence during training, and alleviate the gradient diffusion problem. The specific operation is: adjust the mean of the output data to 0, the variance to 1, and obey the standard normal distribution.

[0050] The second segmentation channel attention module has the same structure as the first segmentation channel attention module. The input end of the segmentation module is connected to the output end of the normalization layer to further extract features of different scales and improve the comprehensive performance of the neural network model on all scale features.

[0051] There are two nodes at the end of the linear classification layer, and the output result represents the abnormal probability of each pixel in the input hyperspectral mural image to be tested.

[0052] In this embodiment, the activation functions of the first segmentation channel attention module, the normalization layer, the second segmentation channel attention module and the linear classification layer of the neural network model all use the leaky_relu function. In other embodiments of the present invention, each layer of the neural network model can also use other existing activation functions. At the same time, before model training, the network parameters of the neural network model are randomly initialized.

[0053] Step 2: Model training.

[0054] Cut out a part of the hyperspectral mural image to be tested, manually label this part, and then input this part of the data as a training sample into the neural network model constructed in step 1 for model training to obtain a trained neural network model.

[0055] The training loss function during model training in this embodiment L for:

[0056]

[0057] In the above formula, N represents the total number of pixels in the training sample. Indicates The label value of each pixel, Indicates that the detection The probability that a pixel is abnormal. The optimizer used in model training is the Adam function.

[0058] Step 3: Image cutting.

[0059] Considering that most of the hyperspectral mural images are high-definition images, they have too many pixels to be input into the neural network model at one time, such as Figure 6 As shown, the hyperspectral mural image to be tested needs to be cut into multiple sub-images of the same size. At the same time, the position information of each sub-image needs to be recorded and stored so as to facilitate image stitching after anomaly detection.

[0060] Specifically, when cutting images, the size of each sub-image can be determined by the computer's memory resources. If the memory resources allow, the sub-images can be cut as large as possible to better detect larger defects.

[0061] Step 4: Anomaly detection.

[0062] The multiple sub-images that have been cut are sequentially input into the neural network model trained in step 2. The neural network model will assign an abnormal probability to each pixel point on each sub-image. , abnormal probability Used to indicate the probability that the corresponding pixel is abnormal.

[0063] According to the abnormal probability of each pixel on each sub-image Determine whether each pixel is abnormal, specifically: compare the abnormal probability of each pixel on each sub-image With setting threshold In this embodiment, the threshold is set Take 0.8. , it means that the pixel is abnormal and is marked as 1. < , it means that the pixel is normal, and it is marked as 0, and finally multiple sub-images with marked results are obtained. At this time, the multiple sub-images with marked results are saved corresponding to their position information.

[0064] Step 5: Output the defect detection results.

[0065] Multiple sub-images with marked results are spliced ​​according to the corresponding position information to obtain the defect detection result of the hyperspectral mural image to be tested.

[0066] like Figure 7 As shown, this is a schematic diagram of the defect detection results obtained by using the method for the hyperspectral mural image to be tested. It can be seen that the hyperspectral mural image defect detection method based on segmented channel attention of the present invention can accurately extract defects at all scales of the mural, has good practicality, and can be widely used in the field of mural defect detection.

[0067] The above description is only used to illustrate the technical solution of the present invention rather than to limit it. For ordinary professional and technical personnel in the field, the specific technical solution recorded in the above embodiment can be modified, or some of the technical features therein can be replaced by equivalents, and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution protected by the present invention.

Claims

1. A hyperspectral mural image defect detection method based on segmented channel attention, characterized in that: The following steps are involved: Step 1, construct a neural network model based on segmentation channel attention; The neural network model includes a first segmentation channel attention module, a normalization layer, a second segmentation channel attention module and a linear classification layer which are arranged in sequence according to the input and output order; The first segmentation channel attention module is used to receive the input hyperspectral mural image to be tested, and segment the hyperspectral mural image to be tested into multiple groups of images containing different band information according to the band, and convolve each group of images with different convolution kernel sizes to extract different scale features of the hyperspectral mural image to be tested; The normalization layer is used to eliminate the adverse effects caused by singular sample data, accelerate convergence and alleviate the gradient diffusion problem during training; the second segmentation channel attention module is used to further extract features of different scales; The linear classification layer is used to output the abnormal probability of each pixel in the hyperspectral mural image to be tested; Step 2: Model training; Cut out a part of the hyperspectral mural image to be tested as a training sample, input it into the neural network model constructed in step 1 for model training, and obtain a trained neural network model; Step 3, image cutting; Cut the hyperspectral mural image to be tested to obtain multiple sub-images of the same size, and record and store the position information of each sub-image; Step 4, anomaly detection; The multiple sub-images that have been cut are sequentially input into the neural network model trained in step 2, and the neural network model assigns an abnormal probability to each pixel point on each sub-image; according to the abnormal probability of each pixel point on each sub-image, it is determined and marked whether each pixel point is abnormal, and multiple sub-images with marked results are obtained; Step 5, output of defect detection results; Multiple sub-images with marked results are spliced ​​according to the corresponding position information to obtain the defect detection result of the hyperspectral mural image to be tested.

2. According to claim 1, a hyperspectral mural image defect detection method based on segmented channel attention is characterized in that: In step 1, the first segmentation channel attention module includes a segmentation module, n convolutional layers with different convolution kernel sizes, n spatial compression modules and a weighting module, where the value of n is equal to the number of bands contained in the hyperspectral mural image; The input end of the segmentation module is used to receive the input hyperspectral mural image to be tested, and n channels are arranged inside the segmentation module, which is used to segment the hyperspectral mural image to be tested into n groups of images through the n channels according to the waveband; The input ends of the n convolutional layers are respectively connected to the output ends of the n channels of the segmentation module, so as to perform convolutions of different convolution kernel sizes on the n groups of images respectively, and obtain n groups of different attention vectors; The input ends of the n spatial compression modules are respectively connected to the output ends of the n convolutional layers, so as to respectively obtain the average values ​​of the n groups of attention vectors in the spatial domain, calculate the corresponding adaptive weight for each group of attention vectors, and obtain the features of n different scales of the hyperspectral mural image to be tested; The input end of the weighting module is connected to the output ends of the n spatial compression modules and the output ends of the n convolutional layers, and is used to weight the n channels according to the adaptive weights obtained from each group of attention vectors to achieve attention allocation of features of different scales, and its output end is connected to the input end of the normalization layer; The second segmentation channel attention module has the same structure as the first segmentation channel attention module, the input end of its segmentation module is connected to the output end of the normalization layer, and the output end of its weighting module is connected to the input end of the linear classification layer.

3. The hyperspectral mural image defect detection method based on segmented channel attention according to claim 2 is characterized by: In step 1, the activation functions of the first segmentation channel attention module, the normalization layer, the second segmentation channel attention module, and the linear classification layer all use the leaky_relu function.

4. A hyperspectral mural image defect detection method based on segmented channel attention according to any one of claims 1 to 3, characterized in that: In step 2, the training loss function during model training is L for: ； In the above formula, N represents the total number of pixels in the training sample. Indicates The label value of each pixel, Indicates that the detection The probability that a pixel is an abnormality.

5. The hyperspectral mural image defect detection method based on segmented channel attention according to claim 4 is characterized by: In step 2, the optimizer during model training uses the Adam function.

6. The hyperspectral mural image defect detection method based on segmented channel attention according to claim 5 is characterized by: In step 4, whether each pixel point is abnormal is determined according to the abnormal probability of each pixel point on each sub-image as follows: Compare the abnormal probability of each pixel on each sub-image With setting threshold The size of , it means that the pixel is abnormal, otherwise it means that the pixel is normal.

7. The hyperspectral mural image defect detection method based on segmented channel attention according to claim 6 is characterized by: In step 4, the threshold is set =0.8.

Images