Crack identification method and device based on graph convolution network and hyperspectral imaging

Abstract

The application provides a crack identification method and device based on a graph convolution network and hyperspectral imaging. The method comprises: obtaining crack data based on a hyperspectral imaging method; preprocessing the crack data to obtain preprocessed crack data; inputting the preprocessed crack data into a pre-constructed double-branch network, the double-branch network comprising a PConv branch and a miniGCN branch, wherein the PConv branch is used to extract local spatial features of the crack data, and the miniGCN branch is used to extract global features containing long-distance correlation information between pixels; splicing and fusing the local spatial features and the global features output by the double-branch network to obtain fused features; and determining a crack identification result according to the fused features. Through the above method, the crack data is obtained by using the hyperspectral imaging method, and the double-branch network is combined to extract features, thereby realizing accurate identification of a crack region.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a crack identification method and device based on graph convolution network and hyperspectral imaging. BACKGROUND

[0002] Building surface cracks are a key problem in engineering structure health monitoring. Due to natural environmental erosion, unstable construction quality, material aging, and long-term load action, cracks are prone to appear on the surface of buildings. These cracks may not be obvious at first, but over time, under the repeated action of natural factors such as rain erosion, freezing, high temperature exposure, and strong wind impact, the cracks will gradually expand and deepen, and may even cause local damage or damage to the overall stability of the structure, thereby causing safety accidents. Therefore, timely and accurate detection of building surface cracks is of great significance for preventing structural failure and ensuring building safety. SUMMARY

[0003] The present application aims to provide a crack identification method and device based on graph convolution network and hyperspectral imaging, to improve the efficiency and accuracy of crack detection.

[0004] In a first aspect, a crack identification method based on graph convolution network and hyperspectral imaging is provided, the crack identification method comprising:

[0005] obtaining crack data based on a hyperspectral imaging method;

[0006] preprocessing the crack data to obtain preprocessed crack data;

[0007] inputting the preprocessed crack data into a pre-constructed double-branch network, the double-branch network comprising a PConv branch and a miniGCN branch, wherein the PConv branch is used to extract local spatial features of the crack data, and the miniGCN branch is used to extract global features containing long-distance correlation information between pixels;

[0008] concatenating and fusing the local spatial features and the global features output by the double-branch network to obtain fusion features;

[0009] determining a crack identification result according to the fusion features.

[0010] Optionally, the preprocessing at least includes denoising and dimensionality reduction processing, normalization processing, constructing an adjacency matrix structure of a graph using a KNN algorithm, and constructing a normalized Laplacian matrix.

[0011] Optionally, before inputting the preprocessed crack data into the pre-constructed double-branch network, the crack identification method further comprises:

[0012] The Adam optimizer is used to train end-to-end with the cross-entropy loss and L2 regularization to optimize the parameters of the double-branch network.

[0013] The accuracy of the double-branch network is determined based on the overall accuracy, average accuracy, Kappa value, crack class recognition accuracy, and confusion matrix of the double-branch network.

[0014] Optionally, before the pre-processed crack data is input into the pre-constructed double-branch network, the crack recognition method further comprises:

[0015] The lightweight of the double-branch network is determined according to the number of floating point operations of the double-branch network, wherein the number of floating point operations of the double-branch network includes the number of floating point operations of the miniGCN branch and the number of floating point operations of the PConv branch.

[0016] The number of floating point operations of the miniGCN branch is represented as:

[0017]

[0018] wherein, represents the number of floating point operations of the miniGCN branch, N represents the number of graph nodes, represents the input feature dimension, represents the output feature dimension.

[0019] The number of floating point operations of the PConv branch is represented as:

[0020]

[0021] wherein, represents the number of floating point operations of the PConv branch, H represents the height of the input feature map, and W represents the width of the input feature map, represents the number of channels participating in convolution, and K represents the size of the convolution kernel.

[0022] In a second aspect, a crack recognition device based on a graph convolution network and hyperspectral imaging is provided, and the crack recognition device comprises:

[0023] An acquisition module is configured to acquire crack data based on a hyperspectral imaging method.

[0024] A preprocessing module is configured to preprocess the crack data to obtain pre-processed crack data.

[0025] inputting the preprocessed crack data into a pre-constructed double-branch network, the double-branch network comprising a PConv branch and a miniGCN branch, wherein the PConv branch is configured to extract local spatial features of the crack data, and the miniGCN branch is configured to extract global features containing long-distance correlation information between pixels;

[0026] fusing the local spatial features and the global features output by the double-branch network to obtain fused features;

[0027] determining a crack recognition result according to the fused features.

[0028] In a third aspect, an electronic device is provided, comprising:

[0029] a memory configured to store instructions; and

[0030] a processor configured to invoke the instructions from the memory and implement the crack recognition method based on a graph convolution network and hyperspectral imaging according to the first aspect of the embodiments of the present application when the instructions are executed.

[0031] In a fourth aspect, a machine-readable storage medium is provided, the machine-readable storage medium storing instructions for causing a machine to perform the crack recognition method based on a graph convolution network and hyperspectral imaging according to the above.

[0032] In a fifth aspect, a computer program product is provided, the instructions in the computer program product being executed by a processor of an electronic device to cause the electronic device to perform the crack recognition method based on a graph convolution network and hyperspectral imaging as described above.

[0033] The crack recognition method and device based on a graph convolution network and hyperspectral imaging are provided. The method comprises: acquiring crack data based on a hyperspectral imaging method; preprocessing the crack data to obtain preprocessed crack data; inputting the preprocessed crack data into a pre-constructed double-branch network, the double-branch network comprising a PConv branch and a miniGCN branch, wherein the PConv branch is configured to extract local spatial features of the crack data, and the miniGCN branch is configured to extract global features containing long-distance correlation information between pixels; fusing the local spatial features and the global features output by the double-branch network to obtain fused features; and determining a crack recognition result according to the fused features. The present application has the beneficial effect of improving the accuracy of crack region recognition by acquiring crack data based on a hyperspectral imaging method and extracting features by using a double-branch network. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1is a flowchart of a crack identification method based on a graph convolution network and hyperspectral imaging provided by an embodiment of the present application.

[0035] Figure 2 is a structural diagram of a crack identification device based on a graph convolution network and hyperspectral imaging provided by an embodiment of the present application.

[0036] Figure 3 is a hyperspectral building surface crack dataset and an image graph displayed by absorbing light under different wavebands provided by an embodiment of the present application. (a) is a 264-waveband integrated hyperspectral image without processing; (b) is a 14-waveband hyperspectral image; (c) is a 17-waveband hyperspectral image; (d) is a 33-waveband hyperspectral image; (e) is a 167-waveband hyperspectral image.

[0037] Figure 4 is an accuracy curve graph in the model training and testing process provided by an embodiment of the present application.

[0038] Figure 5 is a loss curve graph in the model training and testing process provided by an embodiment of the present application.

[0039] Figure 6 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0041] The terms “first”, “second”, and the like in the specification and claims of the present application are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by “first”, “second”, and the like are generally a category, not limited to the number of objects, for example, the first object can be one or more. In addition, “and / or” in the specification and claims means at least one of the connected objects, and the character “ / ” generally represents an “or” relationship between the objects before and after it.

[0042] The crack identification method and device based on a graph convolution network and hyperspectral imaging provided by the embodiments of the present application will be described in detail below with reference to the drawings, through specific embodiments and their application scenarios.

[0043] Please refer to Figure 1 FIG. 1 is a flowchart of a crack identification method based on a graph convolution network and hyperspectral imaging according to an embodiment of the present application. The crack identification method is applied to an electronic device. As shown in FIG. 1, the crack identification method includes the following steps S100-S500. Figure 1

[0044] Step S100: Obtain crack data based on a hyperspectral imaging method.

[0045] In an embodiment of the present application, the core principle of hyperspectral imaging is to use a dispersive element (such as a prism or grating) to decompose incident light into light of different wavelengths, and record the intensity distribution of each wavelength of light by a detector, thereby generating a three-dimensional data cube of space-spectrum. The physical properties and material composition of the measured object can be analyzed based on the position and intensity of the absorption peak. This technology divides the spectrum into narrow bands, so that each pixel not only contains spatial position information, but also contains complete spectral characteristics. In an example, a hyperspectral camera can be used to capture a building surface crack scene, thereby collecting hyperspectral building surface crack data.

[0046] Step S200: Preprocess the crack data to obtain preprocessed crack data.

[0047] In an embodiment of the present application, the distribution of environmental noise in the spectral dimension is non-uniform. Some wavebands are disturbed by the absorption or reflection of cracks and background, resulting in overlapping or crossing of the reflectance curves of the two, reducing the difference in gray value, and reducing the distinguishability of the edge features of the cracks and the texture of the background. At the same time, atmospheric scattering and surface diffuse reflection may enhance in some wavebands, forming stray light to smooth the gray gradient of the cracks and the background, making the edge profile blurred. In addition, the response sensitivity of each pixel of the array detector to different waveband light signals is non-uniform, resulting in systematic attenuation of the signal intensity of some wavebands. Therefore, the crack data needs to be preprocessed.

[0048] In an embodiment of the present application, the preprocessing first performs denoising and dimensionality reduction processing on the crack data. The three-dimensional hyperspectral crack image data is converted into a two-dimensional matrix for normalization processing, the numerical value is scaled to the range of [0, 1], the original spectral curve data is reduced in dimension, and a KNN algorithm is used to construct an adjacency matrix and a Laplacian matrix.

[0049] Step S300: Input the preprocessed crack data into a pre-constructed double-branch network, the double-branch network including a PConv branch and a miniGCN branch, wherein the PConv branch is used to extract local spatial features of the crack data, and the miniGCN branch is used to extract global features containing long-distance correlation information between pixels.

[0050] ​In the embodiments of the present application, the dual-branch network includes a PConv branch and a miniGCN branch. The PConv branch and the miniGCN branch are two different neural network structure branches. The core of the PConv branch is partial convolution (PConv), which aims to solve the problems of low computational efficiency and frequent memory access in operations such as depthwise separable convolution. It improves the floating point operation speed (FLOPS) by reducing redundant calculations and memory access, thereby improving the calculation speed and efficiency of the neural network. The miniGCN branch is an improvement of the traditional graph convolution network (GCN), which aims to solve the problems of high computational cost and large memory consumption of the traditional GCN when processing large-scale data. It reduces the computational cost through batch processing while retaining part of the capabilities of GCN.

[0051] In the embodiments of the present application, in order to effectively extract local spatial features in hyperspectral crack images, a PConv branch (partial convolution branch) is designed in this paper. This branch is aimed at local spatial feature extraction of hyperspectral images, and captures local spatial features of hyperspectral crack images by performing convolution operations on part of the input feature channels while retaining the original features of the remaining channels, which is complementary to the global topological features of the GCN branch. Considering the continuity of the crack features of hyperspectral data, and in order to effectively capture the global topological correlation and long-range dependence relationship between pixels in the hyperspectral crack image, the model introduces a miniGCN branch. The focus of the graph convolution network is the information transmission of the intermediate hidden layer, which depends on the construction of the adjacency matrix and the Laplacian matrix.

[0052] Step S400: The local spatial features and the global features output by the dual-branch network are spliced and fused to obtain a fused feature.

[0053] In the embodiments of the present application, the fusion module splices and fuses the feature vectors output by the two branches to obtain a fused feature vector. The output features of PConv and miniGCN differ in physical meaning or numerical distribution, and splicing and fusion avoids feature information compression or loss that may be caused by addition fusion and other strategies.

[0054] Step S500: Determine a crack recognition result according to the fused feature.

[0055] In the embodiments of the present application, the fused feature is mapped to the category space through two fully connected layers, and the prediction probability of two categories is obtained through the use of a Softmax activation function, and a prediction label is output to obtain a crack recognition result.

[0056] Through the steps S100-S500, the crack data is acquired based on the hyperspectral imaging method; the crack data is preprocessed to obtain preprocessed crack data; the preprocessed crack data is input into a pre-constructed double-branch network, the double-branch network including a PConv branch and a miniGCN branch, wherein the PConv branch is used to extract local spatial features of the crack data, and the miniGCN branch is used to extract global features containing long-distance correlation information between pixels; the local spatial features and the global features output by the double-branch network are spliced and fused to obtain fused features; and a crack recognition result is determined according to the fused features. In this way, the crack data is acquired by using the hyperspectral imaging method, and feature extraction is performed by using the double-branch network, so that accurate recognition of a crack region is realized.

[0057] In some embodiments, the preprocessing at least includes denoising and dimensionality reduction processing, normalization processing, construction of an adjacency matrix structure of a graph by using a KNN algorithm, and construction of a normalized Laplacian matrix.

[0058] Specifically, in the embodiment of the present application, the distribution of environmental noise in the spectral dimension has non-uniformity, and part of the wavebands are disturbed in the absorption or reflection of cracks and backgrounds, which causes the reflectivity curves of the cracks and the backgrounds to possibly overlap or cross, the gray value difference to be reduced, and the distinguishability of the edge features of the cracks and the textures of the backgrounds to be reduced. At the same time, atmospheric scattering and surface diffuse reflection may be enhanced in some wavebands, forming stray light to smooth the gray gradient of the cracks and the backgrounds, and making the edge contour blurred. In addition, the response sensitivity of each pixel of the array detector to different waveband light signals is non-uniform, causing the signal intensity of part of the wavebands to present systematic attenuation. Therefore, the crack data needs to be preprocessed.

[0059] In the embodiment of the present application, the preprocessing first performs denoising and dimensionality reduction processing on the crack data. The three-dimensional hyperspectral crack image data is converted into a two-dimensional matrix for normalization processing, the numerical value is scaled to the range of [0, 1], and then an adjacency matrix and a normalized Laplacian matrix are constructed by using a KNN algorithm.

[0060] In some embodiments, before the preprocessed crack data is input into the pre-constructed double-branch network, the crack recognition method further includes:

[0061] The Adam optimizer is used to perform end-to-end training combined with cross-entropy loss and L2 regularization to optimize the parameters of the double-branch network.

[0062] The accuracy of the double-branch network is determined based on the overall accuracy, the average accuracy, the Kappa value, the recognition accuracy of the crack category, and the confusion matrix.

[0063] Specifically, the dual-branch network is first trained before the pre-processed crack data is input into the pre-constructed dual-branch network.

[0064] The Adam optimizer is used for parameter optimization, the learning rate is set to an exponential decay strategy, the initial learning rate and the decay coefficient are determined, and the decay step is dynamically calculated according to the total amount of training samples and the batch size. This setting makes the learning rate decay exponentially with the training process, using a larger learning rate in the early stage of training to quickly converge, and using a smaller learning rate in the later stage to fine-tune the parameters. To improve the generalization performance of the model, a multi-regularization mechanism is used, and a batch normalization operation is introduced after the graph convolution layer. The momentum parameter is set to standardize the output of the layer, suppress the overfitting phenomenon in the training process, and significantly speed up the model convergence. L2 regularization constraint is applied to the fully connected layer, and the regularization coefficient can be set to 0.001. By limiting the norm size of the weight parameter, the risk of overfitting is further reduced.

[0065] To comprehensively evaluate the classification performance of the model, we use overall accuracy (OA), average accuracy (AA), Kappa value (k), and crack class recognition accuracy. OA represents the proportion of correctly classified samples to the total number of samples, AA is the average of the accuracy of each class, and Kappa coefficient measures the consistency difference between classification results and random classification. These indicators can reflect the accuracy and comprehensiveness of the model in predicting positive and negative samples. The closer the overall accuracy and average accuracy to 100% indicates that the model's ability to distinguish between cracks and background is better, the closer the Kappa value to 1.00 indicates that the model's ability to distinguish between cracks and background is better, and the closer the crack class recognition accuracy to 100% indicates that the model's crack recognition effect is better. By comparing multiple values of different models, it is found that the detection effect of the dual-branch network is excellent.

[0066] wherein,

[0067]

[0068]

[0069]

[0070] wherein TP, TF, FP and FN are true positive samples, true negative samples, false positive samples and false negative samples, respectively, for evaluating images with positive and negative labels that are correctly predicted or incorrectly predicted. Pe represents the consistency of random classification, is the sum of the ith row in the confusion matrix, and represents the number of true samples in the ith row. is the sum of the ith column in the confusion matrix, and represents the number of predicted samples in the ith column, represents the total number of samples.

[0071] In some embodiments, before the pre-processed crack data is input into the pre-constructed dual-branch network, the crack identification method further comprises:

[0072] The lightness of the dual-branch network is determined according to the number of floating point operations of the dual-branch network, and the number of floating point operations of the dual-branch network includes the number of floating point operations of the miniGCN branch and the number of floating point operations of the PConv branch.

[0073] The number of floating point operations of the miniGCN branch is represented as:

[0074]

[0075] Wherein, represents the number of floating point operations of the miniGCN branch, N represents the number of graph nodes, represents the input feature dimension, represents the output feature dimension.

[0076] The number of floating point operations of the PConv branch is represented as:

[0077]

[0078] Wherein, represents the number of floating point operations of the PConv branch, H represents the height of the input feature map, and W represents the width of the input feature map, represents the number of channels participating in convolution, and K represents the size of the convolution kernel.

[0079] Specifically, in order to comprehensively evaluate the actual deployment feasibility of the model, in addition to the classification performance indicators, the application can also use the number of floating point operations FLOPs to determine the accuracy of the dual-branch network. FLOPs directly reflects the computational overhead and hardware adaptation ability of the model, and is the core reference index for lightweight model design and efficiency optimization. FLOPs reduces the number of network calculations to improve the running speed, so that the model has the characteristics of lightness and can reduce the hardware requirements of GPU.

[0080] According to the graph convolution calculation formula, the FLOPs calculation formula of the miniGCN module is:

[0081]

[0082] Wherein, represents the number of floating point operations of the miniGCN branch, N represents the number of graph nodes, represents the input feature dimension, represents the output feature dimension.

[0083] The partial convolution FLOPs calculation formula is:

[0084]

[0085] wherein, represents the number of floating point operations of the PConv branch, H represents the height of the input feature map, W represents the width of the input feature map, represents the number of channels participating in convolution, and K represents the convolution kernel size.

[0086] See Figure 2 is a structural schematic diagram of a crack identification device based on a graph convolution network and hyperspectral imaging provided by an embodiment of the present application, and the second aspect of the present application provides a crack identification device based on a graph convolution network and hyperspectral imaging. The crack identification device comprises:

[0087] An acquisition module is configured to acquire crack data based on a hyperspectral imaging method.

[0088] A preprocessing module is configured to preprocess the crack data to obtain preprocessed crack data.

[0089] An input module is configured to input the preprocessed crack data into a pre-constructed double-branch network, wherein the double-branch network comprises a PConv branch and a miniGCN branch, wherein the PConv branch is configured to extract local spatial features of the crack data, and the miniGCN branch is configured to extract global features containing long-distance correlation information between pixels.

[0090] A fusion module is configured to splice and fuse the local spatial features and the global features output by the double-branch network to obtain fusion features.

[0091] A determination module is configured to determine a crack identification result according to the fusion features.

[0092] The crack identification device based on the graph convolution network and the hyperspectral imaging provided by the second aspect of the present application can realize each process realized by the method embodiments and achieve the same beneficial effects. To avoid repetition, details are not described here.

[0093] An embodiment of the present application provides a specific process of a crack identification method based on a graph convolution network and hyperspectral imaging.

[0094] (I) Obtain a hyperspectral crack data set of a building surface.

[0095] The hyperspectral image acquisition system selected in this embodiment is FigSpec hyperspectral camera FS-15 (line array scanning camera), which uses a high diffraction efficiency transmission grating spectrometer module combined with a high sensitivity area array camera. The spectral line range of the hyperspectral camera sensor is 900-1700 nm, the spectral resolution is 8 nm, and there are 256 wavebands. The image width is 264 pixels, the image length is 320 pixels, and the camera is automatically controlled by a microcomputer to form a three-dimensional hyperspectral image. The specific parameters are shown in Table 1.

[0096]

[0097] First, clean up the site, control stable light, then fix the camera on the guide rail and aim at the crack area, set the spectral range, exposure time and line frequency matched with the scanning speed, confirm that the parameters are correct after pre-scanning, start the guide rail movement and synchronize data collection, finally save the data and do preliminary quality check, avoid vibration and light change throughout the process to ensure clear image and stable spectrum. The hyperspectral camera with the above parameters is used to shoot the building surface crack scene, so as to collect the hyperspectral building surface crack data set used in this study, which has a resolution of 264x320 and contains information of 256 wavebands.

[0098] (II) Label division

[0099] According to the inherent spectral difference of the reflection, absorption and scattering characteristics of different wavelengths of electromagnetic radiation of wall cracks and background area, two types of pixels are divided. Since the wall area in the data set is significantly larger than the crack area, all the crack pixel points in the hyperspectral image except the training set are labeled for testing. The crack pixel points and wall pixel points in the training sample are set in a 1:1 ratio, which aims to alleviate the training bias caused by the imbalance of the number of classes, so that the performance index can more accurately reflect the effectiveness of the non-destructive detection method. Different training sample quantities are set for experiments, and it is found that only 10 pixel points of background and crack are enough to make the model have excellent classification performance and robustness, while the two-dimensional crack data set needs a large number of collected pictures and tedious manual labeling.

[0100] (III) Data set processing

[0101] The distribution of environmental noise in the spectral dimension is non-uniform, and some wavebands are disturbed by the absorption or reflection of cracks and background, resulting in overlapping or crossing of the reflectivity curves of the two, reducing the difference in gray value, and reducing the distinguishability of the edge features of the cracks and the texture of the background. At the same time, atmospheric scattering and surface diffuse reflection may enhance in some wavebands, forming stray light to smooth the gray gradient of the cracks and the background, making the edge profile blurred. Moreover, the response sensitivity of each pixel of the array detector to different waveband light signals is non-uniform, resulting in systematic attenuation of the signal intensity in some wavebands. The hyperspectral building surface crack data set collected in this embodiment originally contains 256 wavebands, after removing 56 interference wavebands, the remaining 200 wavebands are used for research.

[0102] Figure 3 The hyperspectral building surface crack data set and the images displayed by absorbing light at different wavebands are included. The data set displayed in the figure is an unprocessed hyperspectral image integrated from 264 wavebands, and images of four randomly selected wavebands are displayed. It can be seen that the reflected light content presented by different wavebands is inconsistent, and the crack features in the images of some wavebands are completely blurred, such as band 14 and band 167. If these wavebands are not removed but added to the model training and testing, the accuracy of the model will be reduced, and the noise information will increase the computational load of the model, resulting in slower output results.

[0103] By reading the hyperspectral crack data file, the training label file and the test label file, the three-dimensional hyperspectral crack image data is converted into a two-dimensional matrix for normalization processing, and the numerical value is scaled to the range of [0, 1]. The training and test labels are also converted into two-dimensional, and then the training and test samples and their corresponding labels are extracted from the two-dimensional matrix, which are merged into matrices X and Y. Then, using the K nearest neighbor algorithm, setting the number of neighbor nodes k = 10 and the bandwidth parameter of the heat kernel weight mode to = 1, the adjacency matrix and the normalized Laplacian matrix are constructed for the training sample and the test sample respectively, and the unit matrix is added to enhance the stability, which is used for the training and testing of the GCN branch network. The three-dimensional hyperspectral data is converted into a two-dimensional matrix, and normalized, and according to the hyperspectral image, the training label and the test label, a view field with a patch size of 3 is generated to generate training samples and test samples for the training and testing of the PConv branch network.

[0104] (4) Model training

[0105] The Adam optimizer is used for parameter optimization in the model training stage, with an exponential decay strategy for the learning rate. The initial learning rate is set to 0.001, and the decay step is dynamically calculated based on the total number of training samples and the batch size, specifically 50 x training sample size / batch size, with a decay factor of 0.5. This setting allows the learning rate to decay exponentially with the training process, using a larger learning rate in the early stages of training for fast convergence, and a smaller learning rate in the later stages for fine-tuning of parameters. To improve the generalization performance of the model, a multi-regularization mechanism is used, with batch normalization operations introduced after the graph convolution layer. The momentum parameter is set to 0.9, and the layer output is standardized to suppress overfitting during training and significantly speed up model convergence. L2 regularization is applied to the fully connected layer with a regularization coefficient of 0.001, further reducing the risk of overfitting by limiting the norm of the weight parameters.

[0106] (Five) Curve analysis

[0107] In model training, to balance the frequency of learning rate adjustment and the stability of model convergence, the learning rate is set to be dynamically updated every 50 epochs, and 200 epochs can make the model fully learn the data features on the hyperspectral dataset, so the total training epoch number is set to 200. Combined with batch normalization and L2 regularization strategies, the performance and efficiency can be balanced within the batch range. From Figure 5 It can be seen from the figure that as the number of iterations of model training increases, the loss function of the model is constantly decreasing, which means that the model is constantly optimizing parameters to improve performance and does not appear overfitting phenomenon. In addition, the loss function of the model training is lower than that of the test, which means that the model has learned the details of the training data, and the classification effect on the test set shows that the model has excellent performance. Figure 4 The Accuracy curve during model training and testing is schematically shown; Figure 5 The Loss curve during model training and testing is schematically shown. The blue curve is the change of the parameter with the increase of epoch in the training process, and the orange curve is the change of the parameter with the increase of epoch in the testing process.

[0108] (Six) Evaluation index

[0109] To comprehensively evaluate the actual deployment feasibility of the model, in addition to the classification performance index, the application can also use the number of floating point operations FLOPs to determine the accuracy of the double branch network. FLOPs directly reflects the computational overhead and hardware adaptation ability of the model, and is the core reference index for lightweight model design and efficiency optimization.

[0110] According to the graph convolution calculation formula, the FLOPs calculation formula of the miniGCN module is:

[0111]

[0112] wherein, represents the number of floating-point operations of the miniGCN branch, N represents the number of graph nodes, represents the input feature dimension, represents the output feature dimension.

[0113] The partial convolution FLOPs calculation formula is as follows:

[0114]

[0115] wherein, represents the number of floating-point operations of the PConv branch, H represents the height of the input feature map, and W represents the width of the input feature map, represents the number of channels participating in convolution, and K represents the convolution kernel size.

[0116] See Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application, and the third aspect of the present application provides an electronic device 6000, which comprises a processor 6100 and a memory 6200. The memory 6200 stores machine executable instructions capable of being executed by the processor 6100. The processor 6100 can execute the machine executable instructions to implement the crack identification method based on the graph convolution network and hyperspectral imaging described above.

[0117] In some embodiments, the present application further provides a machine readable storage medium, and the machine readable storage medium stores instructions. When the instructions are executed by a processor, the processor implements the crack identification method based on the graph convolution network and hyperspectral imaging described above.

[0118] In some embodiments, the present application further provides a computer program product, which comprises a computer program. When the computer program is executed by a processor, the computer program implements the crack identification method based on the graph convolution network and hyperspectral imaging according to the above embodiments.

[0119] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0120] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0121] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0122] The memory can include non-persistent memory, Random Access Memory (RAM), and / or non-volatile memory, etc. in the form of a computer-readable medium, such as read only memory (ROM) or flash memory. The memory is an example of computer-readable media.

[0123] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0124] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed, or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0125] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

[0126] In addition, any combination of various embodiments of the present application can also be made, as long as it does not deviate from the idea of the present application, it should also be considered as disclosed by the present application.

Claims

1. A crack identification method based on graph convolution network and hyperspectral imaging, characterized in that, The crack identification method comprises: obtaining crack data based on a hyperspectral imaging method; preprocessing the crack data to obtain preprocessed crack data; inputting the preprocessed crack data into a pre-constructed double-branch network, the double-branch network comprising a PConv branch and a miniGCN branch, wherein the PConv branch is used to extract local spatial features of the crack data, and the miniGCN branch is used to extract global features containing long-distance correlation information between pixels; splicing and fusing the local spatial features and the global features output by the double-branch network to obtain fused features; determining a crack identification result according to the fused features.

2. The crack identification method according to claim 1, characterized by, The preprocessing at least comprises denoising and dimensionality reduction processing, normalization processing, constructing an adjacency matrix structure of a graph by using a KNN algorithm, and constructing a normalized Laplacian matrix.

3. The crack identification method of claim 1, wherein, Before the preprocessed crack data is input into the pre-constructed double-branch network, the crack identification method further comprises: using an Adam optimizer to perform end-to-end training in combination with cross-entropy loss and L2 regularization to optimize parameters of the double-branch network; determining accuracy of the double-branch network based on overall accuracy, average accuracy, Kappa value, crack category recognition accuracy, and a confusion matrix of the double-branch network.

4. The crack identification method of claim 1, wherein, Before the preprocessed crack data is input into the pre-constructed double-branch network, the crack identification method further comprises: determining lightness of the double-branch network according to a number of floating-point operations of the double-branch network, the number of floating-point operations of the double-branch network comprising a number of floating-point operations of the miniGCN branch and a number of floating-point operations of the PConv branch; the number of floating-point operations of the miniGCN branch is represented as: wherein, represents the number of floating point operations of the miniGCN branch, N represents the number of graph nodes, represents the input feature dimension, represents the output feature dimension; the number of floating-point operations of the PConv branch is represented as: wherein, represents the number of floating-point operations of the PConv branch, H represents the height of the input feature map, and W represents the width of the input feature map, represents the number of channels participating in convolution, and K represents the size of the convolution kernel. 5.A crack identification device based on a graph convolution network and hyperspectral imaging, characterized in that, The crack identification device comprises: an obtaining module configured to obtain crack data based on a hyperspectral imaging method; a preprocessing module configured to preprocess the crack data to obtain preprocessed crack data; an inputting module configured to input the preprocessed crack data into a pre-constructed double-branch network, the double-branch network comprising a PConv branch and a miniGCN branch, wherein the PConv branch is used to extract local spatial features of the crack data, and the miniGCN branch is used to extract global features containing long-distance correlation information between pixels; a fusing module configured to splice and fuse the local spatial features and the global features output by the double-branch network to obtain fused features; a determining module configured to determine a crack identification result according to the fused features.

6. An electronic device, comprising: comprise: a memory configured to store instructions; and a processor configured to call the instructions from the memory and enable implementation of the crack identification method based on a graph convolution network and hyperspectral imaging according to any one of claims 1 to 4 when the instructions are executed.

7. A machine-readable storage medium, characterized in that, The machine-readable storage medium has instructions stored thereon, the instructions being used to cause a machine to perform the crack identification method based on a graph convolution network and hyperspectral imaging according to any one of claims 1 to 4.

8. A computer program product, characterised in that, The instructions in the computer program product are executed by a processor of an electronic device, so that the electronic device performs the crack identification method based on the graph convolution network and the hyperspectral imaging according to any one of claims 1 to 4.

Images