Rock slope joint crack detection method, device, equipment and medium

Abstract

The invention discloses a rock slope joint crack detection method, device and equipment and a medium, and the method comprises the steps: obtaining a rock image, and carrying out the preprocessing of each channel of the rock image; stacking channels of the preprocessed rock image, inputting the stacked channels into a preset feature extractor for feature extraction, and outputting to obtain a feature vector; the feature extractor comprises an encoder, and the encoder is obtained based on cascaded decoder training; clustering the feature vectors of different rock quality images through self-organizing mapping to obtain a background category of each rock quality image; performing pseudo marking on the corresponding rock quality image based on the background category; performing classification training on a pre-configured deep learning network based on the rock quality image marked with the background category or the crack label; and performing crack detection on a to-be-detected target rock image by using the trained deep learning network to obtain a detection result. The method can accurately detect the joint cracks of the rock slope, and can be widely applied to the technical field of image processing.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, equipment and medium for detecting cracks in rock slope joints. Background Art

[0002] Rock slopes often develop cracks during the use or construction phase due to structural degradation or environmental changes. The continued expansion of cracks will accelerate the degradation of the structure, causing failure or unreliability of the structural system. Cracks are one of the earliest indicators for estimating structural degradation. Therefore, accurate and timely detection of cracks has high practical application value for preventing rock slopes from suffering greater damage or potential accidents. However, existing research works focus on supervised convolutional neural networks (CNNs) without non-crack features or unsupervised crack analysis with limited accuracy, which is less effective for detecting cracks in rock slope joints. Summary of the invention

[0003] The main purpose of the embodiments of the present invention is to provide a method, device, equipment and medium for detecting joint cracks in rock slopes, in order to solve at least one problem of the prior art. The present invention can accurately detect joint cracks in rock slopes.

[0004] To achieve the above object, one aspect of an embodiment of the present invention provides a method for detecting cracks in rock slope joints, the method comprising:

[0005] Acquire a rock image and pre-process each channel of the rock image;

[0006] The channels of the preprocessed rock images are stacked and input into a preset feature extractor for feature extraction, and a feature vector is output; the feature extractor includes an encoder, and the encoder is trained based on a cascaded decoder;

[0007] The feature vectors of different rock images are clustered by self-organizing mapping to obtain the background category of each rock image; the corresponding rock images are pseudo-labeled based on the background category;

[0008] A pre-configured deep learning network is trained for classification based on rock images labeled with background categories or crack labels;

[0009] The trained deep learning network is used to perform crack detection on the target rock image to obtain the detection results.

[0010] In some embodiments, preprocessing is performed on each channel of the rock image, including the following steps:

[0011] Each channel of the rock image is filtered by a Wiener filter;

[0012] Normalization is applied to each channel of the filtered rock image to adjust the maximum intensity and minimum intensity of each channel.

[0013] In some embodiments, the method further comprises the following steps:

[0014] Construct CAE architecture through encoder and decoder;

[0015] The encoder includes a first encoder layer and a second encoder layer, and the first encoder layer and the second encoder layer are both configured with filters; the number of filters in the first encoder layer is less than the number of filters in the second encoder layer; the decoder includes a hierarchical structure symmetrical to the encoder and a Sigmoid function layer;

[0016] The CAE architecture is trained to minimize the difference between input and output images using the preset image data combined with the preset loss function, and then the encoder parameters are optimized and adjusted based on the training results to obtain a feature extractor;

[0017] Among them, the loss functions include mean absolute error and mean square error.

[0018] In some embodiments, before the step of building the CAE architecture, the method further includes the following steps:

[0019] Insert a convolutional layer after the second encoder layer and insert a convolutional layer before the first layer of the decoder;

[0020] And / or, configure rectified linear unit activation function and batch normalization in each layer in the encoder and decoder.

[0021] In some embodiments, the method further comprises the following steps:

[0022] Determine the classification category based on the total number of background categories; wherein the classification category is one more than the total number of background categories, and the extra one category represents the crack category;

[0023] Configure deep learning networks based on classification categories combined with deep CNN architecture.

[0024] In some embodiments, classification training of a preconfigured deep learning network based on a rock image labeled with a background category or a crack label comprises the following steps:

[0025] The rock image is input into the deep learning network and the classification result is output;

[0026] The parameters of the deep learning network are adjusted based on the category results and the loss results of the background category or crack label of the rock image.

[0027] In some embodiments, using a trained deep learning network to perform crack detection on a target rock image to be detected to obtain a detection result includes the following steps:

[0028] The trained deep learning network is used to perform crack detection on the target rock image to be detected, and the classification category probability is output; the classification category probability includes the probability of each background category and the probability of the crack category;

[0029] The classification category probability is thresholded to obtain the detection result of the target rock image.

[0030] To achieve the above object, another aspect of the embodiment of the present invention provides a rock slope joint crack detection device, the device comprising:

[0031] The first module is used to obtain rock images and preprocess each channel of the rock images;

[0032] The second module is used to stack the channels of the preprocessed rock image and input them into a preset feature extractor for feature extraction, and output a feature vector; the feature extractor includes an encoder, and the encoder is trained based on a cascaded decoder;

[0033] The third module is used to cluster the feature vectors of different rock images through self-organizing mapping to obtain the background category of each rock image; and pseudo-label the corresponding rock image based on the background category;

[0034] The fourth module is used to perform classification training on a preconfigured deep learning network based on rock images labeled with background categories or crack labels;

[0035] The fifth module is used to use the trained deep learning network to perform crack detection on the target rock image to be detected and obtain the detection results.

[0036] In some embodiments, the time warning threshold includes a sleep time warning threshold and a reaction time warning threshold; the device further includes:

[0037] The sixth module is used to build the CAE architecture through encoder and decoder;

[0038] The encoder includes a first encoder layer and a second encoder layer, and the first encoder layer and the second encoder layer are both configured with filters; the number of filters in the first encoder layer is less than the number of filters in the second encoder layer; the decoder includes a hierarchical structure symmetrical to the encoder and a Sigmoid function layer;

[0039] The seventh module is used to train the CAE architecture to minimize the difference between input and output images using preset image data combined with a preset loss function, and then optimize and adjust the parameters of the encoder based on the training results to obtain a feature extractor;

[0040] Among them, the loss functions include mean absolute error and mean square error.

[0041] In some embodiments, the apparatus further comprises:

[0042] an eighth module for inserting a convolutional layer after the second encoder layer and inserting a convolutional layer before the first layer of the decoder;

[0043] and / or, a ninth module for configuring a rectified linear unit activation function and batch normalization in each layer in the encoder and the decoder;

[0044] The eighth and ninth modules are executed before the sixth module builds the CAE architecture.

[0045] To achieve the above objective, another aspect of an embodiment of the present invention provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0046] To achieve the above objective, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0047] The embodiments of the present invention include at least the following beneficial effects: the present invention provides a method, device, equipment and medium for detecting cracks in rock slope joints. The scheme obtains rock images and preprocesses each channel of the rock images; the channels of the preprocessed rock images are stacked and input into a preset feature extractor for feature extraction, and a feature vector is output; the feature extractor includes an encoder, and the encoder is trained based on a cascaded decoder; the feature vectors of different rock images are clustered by self-organizing mapping to obtain the background category of each rock image; the corresponding rock images are pseudo-labeled based on the background category; the preconfigured deep learning network is classified and trained based on the rock images marked with background categories or crack labels; the trained deep learning network is used to perform crack detection on the target rock image to be detected to obtain the detection result. The embodiments of the present invention increase the background classification, and by increasing the number of background classifications about non-crack categories, the convolutional neural network is driven to learn non-crack features and improve the crack detection performance. The embodiments of the present invention can accurately detect cracks in rock slope joints. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1is a flow chart of a rock slope joint crack detection method provided by an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of an example of a CAE architecture provided by an embodiment of the present invention;

[0050] Figure 3 It is a schematic diagram of the process architecture of the rock slope joint crack detection method provided by an embodiment of the present invention;

[0051] Figure 4 A schematic structural diagram of a rock slope joint crack detection device provided by an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0054] It is understood that the terms "first", "second", etc. used in the present invention may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0055] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0057] The rock slope joint crack detection method provided by the embodiment of the present invention relates to the field of image processing technology. The rock slope joint crack detection method provided by the embodiment of the present invention can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the rock slope joint crack detection method, etc., but is not limited to the above forms.

[0058] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0059] Figure 1 is an optional flow chart of a rock slope joint crack detection method provided by an embodiment of the present invention. Figure 1 The method may include but is not limited to steps S100 to S500.

[0060] S100, acquiring a rock image, and preprocessing each channel of the rock image;

[0061] It should be noted that, in some embodiments, preprocessing each channel of the rock image may include the following steps: filtering each channel of the rock image through a Wiener filter; applying normalization to each channel of the filtered rock image, and adjusting the maximum intensity and minimum intensity of each channel.

[0062] Exemplarily, in some embodiments, preprocessing is applied to the entire image. Preprocessing is mandatory for effective unsupervised feature extraction. Studies have found that pixel-level speckle-like noise dominates the rock slope joint image. Preprocessing is applied to each RGB channel, and Wiener filter and normalization are applied. The filter size is 16 by 16 pixels. The Wiener filter is an adaptive filter. The noise area is strongly smoothed, while the crack and feature areas are weakly smoothed. The noise is reduced, while the cracks remain clear. The larger the filter size, the smoother the image. Too large a filter size may blur the features of the cracks. The result is insensitive to size. A total of 16 pixels are sufficient. Normalization is applied to each RGB channel to adjust the maximum intensity to 1 and the minimum intensity to 0. This process is intended to eliminate the influence of lighting conditions and color of rock slope joints. In the absence of Wiener filter and normalization, the feature extractor can learn the differences in noise level and color as important features. After preprocessing, the channel images are stacked to input into the feature extractor.

[0063] S200, stacking the channels of the preprocessed rock image and inputting them into a preset feature extractor for feature extraction, and outputting a feature vector;

[0064] The feature extractor includes an encoder, and the encoder is trained based on a cascaded decoder;

[0065] It should be noted that, in some embodiments, the method may further include the following steps: constructing a CAE architecture through an encoder and a decoder; wherein the encoder includes a first encoder layer and a second encoder layer, and the first encoder layer and the second encoder layer are both configured with filters; the number of filters in the first encoder layer is less than the number of filters in the second encoder layer; the decoder includes a hierarchical structure and a Sigmoid function layer symmetrical to the encoder; using preset image data combined with a preset loss function to train the CAE architecture to minimize the difference between input and output images, and then optimizing and adjusting the encoder parameters based on the training results to obtain a feature extractor; wherein the loss function includes mean absolute error and mean square error.

[0066] For example, in some specific embodiments, Figure 2 As shown in Figure 1, the proposed five-layer CAE architecture is shown. The input and output of CAE are RGB color three-channel images of 128×128 pixels. The goal of CAE is to extract spatial features to compress images. At the same time, the reconstructed image from the compressed features should be as similar as possible to the input image to confirm that the extracted features are valid. CAE consists of an encoder and a decoder part. The optimized encoder corresponds to the feature extractor. The following symmetrical structure decoder layer is also trained to decompress the feature map to reconstruct the image.

[0067] The CAE architecture consists of an encoder and a decoder. The encoder compresses an input image with a size of 128 by 128 pixels by three channels to output 15 feature maps with a size of 32 by 32. The decoder decompresses the feature maps to output an image, thereby reconstructing the input image. The optimized encoder is a feature extractor.

[0068] Two important parameters should be considered: the number of layers and the choice of loss function. This is a trade-off; the deeper the CAE, the smaller the compressed data, but the greater the information loss may be. The CAE is trained to minimize the difference between the input and output images. An appropriate loss function should be specified to develop an effective autoencoder.

[0069] In terms of the number of layers, the dimension of the input image is about 49000. It was observed that it took several days to analyze these features; the computation time increases dramatically around this number. The first encoder layer computes spatial features from the input image to transform it into 10 feature maps with a size of 64 by 64. Max pooling is used to suppress the map size. However, the size is still around 41000. The second encoder layer further compresses the features into 15 feature maps with a size of 32 by 32. The size is 15000, which is still large. However, it is one-third of the original image and is relatively light.

[0070] The autoencoder is trained using mean absolute error (MAE). MAE is the sum of the absolute differences in the intensity of the corresponding pixels (x,y) between the original image and the reconstructed image divided by the number of pixels N. The expression of MAE is as follows:

[0071]

[0072] In the formula, I o -I r Represents the absolute difference in intensity at the pixel (x,y).

[0073] A convolutional layer is inserted after the second encoder layer with the same architecture, and a convolutional layer is inserted before the first decoder layer with the same structure (seven-layer CAE). By adding these layers, the loss of information is obvious. Another typical loss function is the mean squared error (MSE).

[0074] Among them, in some embodiments, before the step of constructing the CAE architecture, the method may also include the following steps: inserting a convolutional layer after the second encoder layer and inserting a convolutional layer before the first layer of the decoder; and / or, configuring a rectified linear unit activation function and batch normalization in each layer in the encoder and decoder.

[0075] Exemplarily, in some specific implementations, the number of filters and the size of the filters are also important parameters. These parameters should be large enough to extract image features. The parameters are selected to confirm that the loss value converges. The number of filters in the first layer of the encoder and the second layer of the decoder is less than the number of filters in the second level of the encoder and the first level of the decoder. This is to avoid a sudden reduction in mapping information. With respect to other typical DL techniques, a rectified linear unit (ReLU) activation function is used, and batch normalization is used in the encoder part to output stable results and avoid saturation.

[0076] S300, clustering the feature vectors of different rock images through self-organizing mapping to obtain the background category of each rock image; and pseudo-labeling the corresponding rock image based on the background category;

[0077] For example, in some embodiments, a SOM (Self-organizing map) algorithm automatically optimizes the map size and clusters the features of the developed CAE output. The non-crack images are pseudo-labeled with reference to the obtained clusters. The labels are represented as background 1, background 2, ..., background n.

[0078] Among them, SOM (Self-organizing map) is a typical map that is a hexagonal honeycomb structure. A weight vector is assigned to each cell of the structure. The input feature vector is classified into one of the cells that matches the input vector and the weight vector. After matching, the corresponding weight vector and the weight vector close to the corresponding cell are updated to increase the matching coefficient. This is a mutual optimization process. After repeating the classification of the input vector and the update of the weight vector, similar input vectors are stored in a compact area. The distribution of grouped input vectors in the map visualizes the structure of the data. The distance between adjacent weight vectors means the similarity of adjacent cells.

[0079] S400, performing classification training on a preconfigured deep learning network based on a rock image marked with a background category or a crack label;

[0080] In some embodiments, the method may further include the following steps: determining the classification category based on the total number of background categories; wherein the classification category is one more than the total number of background categories, and the extra category represents a crack category; configuring a deep learning network based on the classification category combined with a deep CNN architecture.

[0081] It should be noted that, in some embodiments, step S400 may include: inputting the rock image into the deep learning network and outputting a classification result; and adjusting the parameters of the deep learning network based on the classification result and the loss result of the background category or crack label marked in the rock image.

[0082] For example, in some embodiments, ordinary deep learning (DL) classification can be used, and considering the addition of n background categories and crack images, N = n + 1 classification networks are trained to learn detailed non-crack image features. This method also improves the ability to distinguish between crack and non-crack images. The architecture can be a typical deep CNN architecture, such as VGG16 of TL using ImageNet parameters.

[0083] S500: Use the trained deep learning network to perform crack detection on the target rock image to be detected to obtain a detection result.

[0084] It should be noted that, in some embodiments, step S500 may include: using a trained deep learning network to perform crack detection on the target rock image to be detected, and outputting classification category probabilities; the classification category probabilities include the probabilities of each background category and the probabilities of the crack category; and performing threshold processing on the classification category probabilities to obtain the detection results of the target rock image.

[0085] For example, in some embodiments, the detection outputs the probability of N categories. The probability of the crack category is thresholded to determine the presence of a crack. The probability indirectly takes into account all remaining non-crack categories.

[0086] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0087] First of all, it should be noted that the presence of joint cracks in rock slopes is an important indicator of damage. Deep learning is the mainstream method for detecting cracks in images due to its detection capabilities. Previous research works have focused on supervised convolutional neural networks (CNNs) without non-crack features or unsupervised crack analysis with limited accuracy. The novelty of the present study lies in the addition of background classification. By increasing the number of non-crack categories, the convolutional neural network is driven to learn non-crack features and improve crack detection performance. The crack-free images are preprocessed and their features are extracted in an unsupervised manner by a deep convolutional autoencoder. Self-organizing maps cluster the features to obtain non-crack categories and unravel how pixel-level fine crack detection results are affected by different loss functions. The focus of this study is classification, although the method can be used in parallel with state-of-the-art segmentation algorithms. Compared with previous models, the improved deep CNN model significantly improves the accuracy by 1%-4% and the f-measure by 3%-8% using common crack datasets. The improved visual geometry group (VGG)16 shows top performance, 96% accuracy and 84%-85% f-measure. These models significantly reduce false detections while maintaining their crack detection capabilities.

[0088] The present invention adopts unsupervised feature extraction and pseudo-labeling methods to integrate the information of the background image into the classification task. This method is different from the ordinary "crack" and "non-crack" classification, which does not consider the difference in spatial features in the background image.

[0089] By leveraging the classification power of pseudo-labeled supervised deep CNNs, the highest classification accuracy is achieved compared to previous CNNs and unsupervised methods. State-of-the-art CNN architectures can be used with minor modifications to the CNNs. The training time is almost the same as previous CNNs.

[0090] The method of the present invention is as follows Figure 3 As shown. The method includes two steps. In the first step, the non-crack image is interpreted. In order to perform unsupervised analysis effectively, preprocessing is very important. The present invention adopts Wiener filter and normalization. Using an optimized convolutional autoencoder (CAE) architecture, the features of patterns and objects, such as shadow areas and speckled paint and dust, are analyzed in an unsupervised manner. The SOM algorithm automatically optimizes the mapping size and clusters the features output by the developed CAE. The non-crack image is pseudo-labeled with reference to the obtained clusters. The labels are denoted as background 1, background 2,..., background n.

[0091] Specifically, refer to Figure 3The method of the present invention includes two steps: upper: background analysis through preprocessing, convolutional autoencoder (CAE) and self-organizing map (SOM), lower: crack detection through convolutional neural network (CNN).

[0092] The second step (below) is a normal deep learning (DL) classification. Considering the addition of n background categories and crack images, N=n+1 classification networks are trained to learn detailed non-crack image features. This approach also improves the discrimination between crack and non-crack images. The architecture is a typical deep CNN architecture, such as VGG16 with TL using ImageNet parameters. The probabilities of N categories are output. The probability of the crack category is thresholded to determine the presence of cracks. This probability indirectly takes into account all remaining non-crack categories.

[0093] All CNNs and CAEs used in this invention are considered in depth for their performance. The selection of preprocessing filters, CAE architecture, SOM size, CNN architecture and hyperparameters are important parameters. Each process should be optimized to achieve the highest performance. The method consists of two parts: feature extraction and classification criterion construction. The method uses supervised learning of labeled data. In any case, one category is pseudo-labeled by unsupervised feature extraction and clustering. Therefore, a fully supervised two-class classification and an unsupervised feature extraction and classification algorithm should be compared to highlight the effectiveness of the proposed method.

[0094] based on Figure 3 Based on the improved U-Net architecture in this paper, the effects of six different loss functions on pixel-level thin crack detection are first explored. The experimental loss functions include L bce or binary cross entropy loss, L dice or dice loss, L dice_bce or a mixture of Dice and Binary Cross Entropy Loss, L iou Or intersection over union (iou) loss, L log_iou Or the negative log loss with respect to IoU, and L bf Or binary focal loss. L bce is the baseline loss function for obtaining comparison results with the modified U-Net. bce , other experimental loss functions are well known in segmentation tasks or class imbalance tasks. For example, the Dice coefficient (also known as –Dice coefficient) is a common metric for binary classification tasks such as pixel segmentation, and IoU (also known as Jaccard index) is another popular segmentation metric. Therefore, L is chosen here dice and L iou This is to directly optimize the F1 performance during training. dice_bce In order to obtain more diversity, we try to use a composite loss function, while L log_iou is another attempt to smooth the loss surface of iou values. bf It is a binary classification version of the well-known focal loss function, which directly addresses the extreme class imbalance problem.

[0095] Assume N px is the number of pixels in the image (same for all input images, ground truth images, and output images), and y i and denote the ground truth value and the related predicted value of the i-th pixel in the ground truth image and the predicted image respectively. Then, the experimental loss function is calculated according to equations (1)-(6). bce and L iou , using the original formula without modification. For L dice , instead of the original equation, a simplified version is used so that the denominator is a simple sum, not a sum of squares. dice_bce , the sum of the two losses is equal. For L bf , using a constant value of α = 3 and three different values ​​of γ = {1, 2, 4}; the three binary focal losses will be referred to as L bf1 , L bf2 and L bf4 . In addition, since there are many alternative implementations of binary focusing loss, preliminary experiments were performed to select an appropriate implementation as written in Equation (6). Note that in Equation (6), L bf_pos is the loss calculated only from positive (crack) pixels, where y i =1(y pos ), L bf_neg is the loss calculated only from negative (non-crack) pixels, where y i =0(y neg ).

[0098]

[0099]

[0100] L dice_bce =L dice +L bce (3)

[0101]

[0102] L bf =L bf_pos +L bf_neg (6)

[0103]

[0104] Specifically, for CAE feature extraction, this section compares feature extraction methods for rock slope joint crack detection. A Wiener filter is applied and the image is normalized. Regarding feature extraction, in addition to CAE, typical image processing methods are also compared. For comparison, the extracted features are connected to a SVM with a nonlinear kernel to develop a classifier. The specific implementation is as follows:

[0105] 1. Preprocessing:

[0106] Preprocessing is applied to the entire image. Preprocessing is mandatory for effective unsupervised feature extraction. It was found that pixel-level speckle-like noise dominates the rock slope joint images. Preprocessing is applied to each RGB channel and Wiener filter and normalization are applied. The filter size is 16 by 16 pixels. Wiener filter is an adaptive filter. Noise areas are strongly smoothed, while crack and feature areas are weakly smoothed. Noise is reduced while cracks remain clear. The larger the filter size, the smoother the image. Too large a filter size may blur the features of cracks. The result is not sensitive to size. A total of 16 pixels are sufficient. Normalization is applied to each RGB channel to adjust the maximum intensity to 1 and the minimum intensity to 0. This process is designed to eliminate the influence of lighting conditions and color of rock slope joints. In the absence of Wiener filter and normalization, the feature extractor can learn the differences in noise level and color as important features. After preprocessing, the channel images are stacked to input into the feature extractor.

[0107] Figure 2 The proposed five-layer CAE architecture is shown. The input and output of CAE are RGB color three-channel images of 128×128 pixels. The goal of CAE is to extract spatial features to compress the image. At the same time, the reconstructed image from the compressed features should be as similar as possible to the input image to confirm that the extracted features are valid. CAE consists of an encoder and a decoder part. The optimized encoder corresponds to the feature extractor. The following symmetrical structure decoder layer is also trained to decompress the feature map to reconstruct the image.

[0108] The architecture consists of an encoder and a decoder. The encoder compresses the input image with a size of 128 by 128 pixels by three channels to output 15 feature maps with a size of 32 by 32. The decoder decompresses the feature maps to output images, thereby reconstructing the input image. The optimized encoder is a feature extractor.

[0109] Two important parameters should be considered: the number of layers and the choice of loss function. This is a trade-off; the deeper the CAE, the smaller the compressed data, but the greater the information loss may be. The CAE is trained to minimize the difference between the input and output images. An appropriate loss function should be specified to develop an effective autoencoder.

[0110] In terms of the number of layers, the dimension of the input image is about 49000. It was observed that it took several days to analyze these features; the computation time increases dramatically around this number. The first encoder layer computes spatial features from the input image to transform it into 10 feature maps with a size of 64 by 64. Max pooling is used to suppress the map size. However, the size is still around 41000. The second encoder layer further compresses the features into 15 feature maps with a size of 32 by 32. The size is 15000, which is still large. However, it is one-third of the original image and is relatively light.

[0111] The autoencoder is trained using the mean absolute error (MAE). MAE is the sum of the absolute differences in the intensity of the corresponding pixels (x,y) between the original image and the reconstructed image, divided by the number of pixels N.

[0112]

[0113] A convolutional layer is inserted after the second encoder layer with the same architecture, and a convolutional layer is inserted before the first decoder layer with the same structure (seven-layer CAE). By adding these layers, the loss of information is obvious. Another typical loss function is the mean squared error (MSE).

[0114] The number of filters and the size of the filters are also important parameters. These parameters should be large enough for extracting image features. The parameters are selected to confirm that the loss value converges. The number of filters in the first layer of the encoder and the second layer of the decoder is smaller than the number of filters in the second level of the encoder and the first level of the decoder. This is to avoid a sudden decrease in mapping information. Regarding other typical DL techniques, the rectified linear unit (ReLU) activation function is adopted, and batch normalization is used in the encoder part to output stable results and avoid saturation.

[0115] 2. Comparison of unsupervised methods:

[0116] In some embodiments, an attempt was made to develop a completely unsupervised crack detection algorithm that combines an unsupervised feature extractor with an unsupervised clustering algorithm. The clusters are then interpreted based on the clustered images. This strategy naturally incorporates features from crack images and non-crack images. The unsupervised feature extractor discussed in this section is combined with a SOM discussed in the next section. However, the classification accuracy of the classifier is less than 60%.

[0117] The next strategy is to feed the extracted features into a supervised SVM with a nonlinear kernel function. The SVM is trained with features of crack and non-crack images and labels to compare the performance of the developed feature extractor. Figure 2 The output of the feature extractor including the encoder shown is flattened to construct the feature vector. SVM requires less data than DL methods because the small support vector defines the classification criterion. The kernel function adopted is the radial basis function, which is used to construct the nonlinear partitioning plane. For reproducibility, there are some other parameters: 2% of the outlier vectors are removed. The regularization factor is 0.5. A logistic function is fitted to output the probability of cracks.

[0118] For comparison, HOG features as typical image eigenvalues, Gabor filters as typical frequency filters, and Hough transform after edge detection by Canny method as typical image processing methods and simple average pooling were considered. In addition to the above features, image processing techniques such as Fourier transform and speeded up robust features (SURF) were also extensively compared and similar results were obtained. Gabor filter is a set of 2D filters that emphasize certain frequency components. It is related to Fourier transform. HOG feature is a fast version of SURF feature. Therefore, Gabor filter and HOG feature are compared to represent frequency analysis and image eigenvalue methods.

[0119] For reproducibility, the parameters of the feature extractor are: the cell size of the HOG feature is 32 by 32; the wavelength and direction of the Gabor filter are 10, 20 and 30 pixels and 0, 30, 60, 90, 120 and 150 degrees, and the filtered image is downsampled to one quarter; the threshold of the Canny method is 0.7, and the resolution of rho and theta is 2 pixels and 2 degrees respectively. Average age pooling is to divide the image into four images of 32 by 32 pixels and calculate the average of each image in each RGB channel. This is the most primitive downsampling method. The size of the original image is too large and unacceptable for SVM. The size of these features is comparable to the size of the encoder.

[0120] The proposed encoder achieved the highest accuracy of 92% and f-measure of 69%. The accuracy of HOG features, Gabor filter, and Hough+Canny were 20%, 10%, and 26% lower than that of CAE, respectively. The f-measures decreased by 20%, 12%, and 26%, respectively. The extent of the reduction may depend on the dataset. In terms of the dataset considered, these features are not suitable for extracting thin cracks from noisy asphalt pavement patterns. These features are inferior to simple average pooling. Average pooling considers the dark areas of cracks and their combinations. On the other hand, CAE effectively extracts the spatial features of cracks and pavement patterns. The accuracy and f-measure of CAE are 6% higher than that of average pooling. Conclusion CAE is the most effective unsupervised crack feature extraction method.

[0121] However, the performance of the best CAE is different from that of the state-of-the-art deep CNNs. This is because the supervised DL architecture with TL accurately learns the crack features and builds precise classification criteria inside the network. However, during training, supervised CNNs may ignore the details of non-crack images. Therefore, the proposed method adopts SOM to build finer non-crack categories to encourage CNN to learn the features of non-crack images by leveraging the advantages of unsupervised and supervised methods.

[0122] Since the true structure of the non-crack image data is unknown, it is impossible to quantitatively discuss the impact of misclassified pseudo-labels on the CNN through unsupervised methods. In the worst case, if all non-crack images are randomly classified, the training of the CNN will fail. The improvement in accuracy shown by using multiple categories means that the pseudo-labeling has some effectiveness. However, in the field of DL research, the contamination of a dataset with 8% imprecise labels (92% accuracy) is usually not negligible. Better feature extractors have the potential to lead to better CNN performance because appropriate features help the CNN to accurately classify features. The best CAE was used in this study, although the CAE can be further improved by adopting a well-designed architecture.

[0123] In some specific implementations, the embodiments of the present invention also provide a class-specific image augmentation (CSIA). As an important data-centric method, image augmentation is widely used to increase training data sets. The Wasserstein generative adversarial network (wGAN) is modified in this work and used to generate realistic top-down crack images.

[0124] The main difference between wGAN and traditional GAN ​​is that wGAN uses the Wasserstein distance metric to measure the distance between the generated images and the real images, which makes the training of wGAN more stable. In wGAN, the generator and discriminator are trained to optimize the Wasserstein distance between the generated images and the real images instead of minimizing the cross entropy loss. This leads to better training stability, clearer and more realistic images, and better control over the diversity of generated images.

[0125] This work involves training a wGAN for the dataset. The wGAN is trained to generate images of a specific class. This approach works well when there are significant differences between the classes. By training a separate wGAN for each class, the generated images can be optimized for that specific class, resulting in high-quality, realistic images. The wGAN is trained for 20,000 epochs to generate = images.

[0126] In deep learning, it is common to add images to each class equally. However, in real life, data is not always evenly distributed. For example, linear cracks are more common than alligator and block cracks. In addition, different damages are distinguished differently, as seals and repairs are more distinguished than cracks. In order to determine how many images to add to each class, a class-specific image augmentation (CSIA) method is proposed as an optimistic augmentation method.

[0127] The following steps outline the implementation process of the CSIA approach.

[0128] First, the loss value for each category can be calculated using Equation (7). Based on the benchmark study, the YOLOv5s model was trained and tested on the original training and testing datasets with 300 epochs, a batch size of 64, and an initial learning rate of 0.01.

[0129] Loss k =1-mAP50 k (7)

[0130] Among them, Loss k is the loss for class k; mAP50 k is the mAP50 of class k in the baseline. Smaller loss values ​​mean less need for image augmentation in this case.

[0131] The distribution of raw data is also an important factor in image enhancement strategy. The weight score of each class is calculated based on equation (8).

[0132]

[0133] Where Ok is the original number of class k, W k is the weight score of class k.

[0134] Then, the number of enhanced images of category k can be determined by equation (9).

[0135]

[0136] Among them, A k is the number of enhanced images of class k, and N is the total number of images planned to be added in the deep learning task (1400 in this work). According to equation (5), in this CSIA method, both the distribution of the original data and the distinction between different classes are considered to optimize the number of generated images.

[0137] Among them, in some preferred implementations, the embodiments of the present invention also provide an orthogonal test for parameter fine-tuning, which is specifically implemented as follows:

[0138] In order to find out the impact of parameters including epochs, batch size, learning rate on the performance of YOLOv5s model, orthogonal experiments were considered to study the impact of each factor. Orthogonal experiments are a statistical method that can help reduce the number of experiments required to identify the best combination of parameters by systematically varying the parameters to maximize the amount of information gained from each experiment. It allows determining the best parameter values ​​with fewer experiments, thus significantly saving time and resources.

[0139] Epochs (factor A), batch size (factor B), and initial learning rate (factor C) were selected as the three factors considered to have an impact on the accuracy of the YOLOv5s model. The levels of epochs were 200, 300, and 400. The batch sizes were 16, 32, and 64. The initial learning rate had three levels of 1e-1, 1e-2, and 1e-3. The orthogonal experiment in this study was designed with the help of a 3-factor 3-level orthogonal table.

[0140] Analysis of variance (ANOVA) was used to calculate the impact of each factor on the predictive performance of the model. This is a statistical method that illustrates the significance of different factors based on the results of orthogonal experiments. This method can be described by the following equation.

[0141] SS T =SS f +SS e (10)

[0142] Among them SS T is the sum of squares, SS f is the sum of squares of the regression. SS e is the sum of squares of errors;

[0143] df T =df f +df e (11)

[0144] where df T is the total degree of freedom (DF), df f is a factor DF, df e is the DF of the error;

[0145] The mean square of each factor (MSf), the mean square of each error (MSe), and the F statistic of each factor (F) can be obtained using equations (12) to (14).

[0146]

[0147] In ANOVA, the critical F value (Fc) is used to compare with the calculated F statistic to determine whether the observed differences between groups are statistically significant. A significance level (α) of 0.05 was used in this work, which represents a 5% probability of making a Type I error (rejecting the null hypothesis when it is true). The critical F value is chosen based on the chosen α level and the degrees of freedom associated with the test.

[0148] Specifically, the non-crack image analysis provided by the embodiment of the present invention is specifically implemented as follows:

[0149] 1. Theory:

[0150] SOM was proposed by Kohonen. The typical mapping is a hexagonal honeycomb structure. A weight vector is assigned to each cell of the structure. The input feature vector is classified into one of the cells that matches the input vector and the weight vector. After matching, the corresponding weight vector and the weight vector close to the corresponding cell are updated to increase the matching coefficient. This is a mutual optimization process. After repeating the classification of the input vector and the update of the weight vector, similar input vectors are stored in a close area. The distribution of grouped input vectors in the mapping visualizes the structure of the data. The distance between adjacent weight vectors means the similarity of adjacent cells.

[0151] The two main points of SOM are how to choose the appropriate eigenvector and map size. The discussion in the previous section is about the appropriate eigenvector. Regarding the map size, the number of vectors in the cell becomes too small or too large due to the size being too large or too small. In this case, it is difficult to find meaningful structures in the map. The map size is automatically determined, and the features are automatically clustered. It selects important eigenvalues ​​and sets thresholds. The MATLAB library provided by the research team was used.

[0152] 2. Mapping results:

[0153] The feature vectors of the non-crack image extracted by the developed CAE were input. The estimated map size was (28,11). These numbers are the number of classified vectors in each cell. To reduce the computation time, the number of feature vectors was reduced to one tenth. The computation time after reduction was 2–3 hours. Without the reduction, the computation time would have taken several days and was not feasible. It was confirmed that the size and color of the SOM map (distribution of inter-cell distances, U-mat) converged under a certain number of vectors. The color inside the cell corresponds to the average of the distances of the neighboring cells. The size, distribution and color are automatically estimated in the SOM development.

[0154] Some features consist of 40-50 vectors, while others consist of 5-10 vectors. Similar vectors are grouped into the same cell. As a qualitative discussion, two significant features are observed, feature 1 with 42 vectors and feature 2 with 51 vectors, as these two features have a large number of graphs. For discussion purposes, these features are manually extracted along with other features.

[0155] For labeling, according to previous research works, further automatic clustering was performed considering the distance between weight vectors. Common features were taken into account. In order to address various characteristics, the number of clusters should be large. However, a large amount of training data will reduce it. The number of non-crack images is six to seven times that of crack images. The number of the largest cluster is twice that of other clusters. Considering this situation, five groups were adopted.

[0156] The mapping shows that non-crack images have rich features. If more non-crack images are collected, the number of clusters can be increased.

[0157] As mentioned above, the discrimination ability of SOM is not as high as that of DL model. A classification model of crack and non-crack is established using SOM. However, its accuracy is low. Two reasons why SOM is not very accurate may be that CNNs are composed of deep convolutional layers with high representation ability, and CNNs can effectively learn crack features with crack labels. For future work, advanced autoencoders and unsupervised clustering methods can be developed in the field of DL research. They have the potential to solve the problem of unsupervised learning.

[0158] 3. Explanation:

[0159] SOM is a black-box algorithm; it is unclear what features are considered during the learning process. In this section, the characteristics of the background image of the rock slope joints are qualitatively discussed.

[0160] R channel image after preprocessing. No difference is observed between RGB channels. The reason is that SOM takes into account the contrast of the image; SOM can focus on the fuzziness of the edges in the shadow image.

[0161] In summary, compared with the prior art, the present invention has at least the following beneficial effects:

[0162] For the purpose of joint crack detection in rock slopes, an effective preprocessing and feature extraction scheme is proposed. In terms of preprocessing, Wiener filter and normalization are used. Regarding feature extraction, in addition to CAE, the original image detected by the Canny method with average pooling, oriented histogram (HOG) features, Gabor filter and edge Hough transform are compared.

[0163] The extracted features are quantitatively evaluated and extensively compared using a nonlinear SVM classifier optimized for the crack detection task.

[0164] For the first time, unsupervised rock slope joint crack analysis was performed based on SOM and a common dataset. The mapping results and example images were discussed to gain insight into the image characteristics of joint cracks in crack-free rock slopes.

[0165] For the first time, multi-class deep CNNs are proposed by combining unsupervised and supervised learning using CAE, SOM, and CNNs to achieve the highest crack detection performance incorporating non-crack image features. The performance of the developed deep CNN is significantly improved compared to recent CNNs such as VGG16 and MobileNetV3. A confusion matrix is ​​given to discuss the properties of the developed multi-class CNN.

[0166] See also Figure 4 The embodiment of the present invention further provides a rock slope joint crack detection device 900, which can implement the above rock slope joint crack detection method. The device may include:

[0167] The first module 910 is used to obtain a rock image and pre-process each channel of the rock image;

[0168] The second module 920 is used to stack the channels of the preprocessed rock image and input them into a preset feature extractor for feature extraction, and output a feature vector; the feature extractor includes an encoder, and the encoder is obtained by training based on a cascaded decoder;

[0169] The third module 930 is used to cluster the feature vectors of different rock images through self-organizing mapping to obtain the background category of each rock image; and pseudo-label the corresponding rock image based on the background category;

[0170] A fourth module 940 is used to perform classification training on a preconfigured deep learning network based on a rock image marked with a background category or a crack label;

[0171] The fifth module 950 is used to use the trained deep learning network to perform crack detection on the target rock image to be detected and obtain the detection result.

[0172] In some embodiments, the time warning threshold includes a sleep time warning threshold and a reaction time warning threshold; the device may further include:

[0173] The sixth module is used to build the CAE architecture through encoder and decoder;

[0174] The encoder includes a first encoder layer and a second encoder layer, and the first encoder layer and the second encoder layer are both configured with filters; the number of filters in the first encoder layer is less than the number of filters in the second encoder layer; the decoder includes a hierarchical structure symmetrical to the encoder and a Sigmoid function layer;

[0175] The seventh module is used to train the CAE architecture to minimize the difference between input and output images using preset image data combined with a preset loss function, and then optimize and adjust the parameters of the encoder based on the training results to obtain a feature extractor;

[0176] Among them, the loss functions include mean absolute error and mean square error.

[0177] In some embodiments, the apparatus may further include:

[0178] an eighth module for inserting a convolutional layer after the second encoder layer and inserting a convolutional layer before the first layer of the decoder;

[0179] and / or, a ninth module for configuring a rectified linear unit activation function and batch normalization in each layer in the encoder and the decoder;

[0180] The eighth and ninth modules are executed before the sixth module builds the CAE architecture.

[0181] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0182] The embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the rock slope joint crack detection method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0183] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0184] See also Figure 5 , Figure 5 The hardware structure of an electronic device 1000 of another embodiment is illustrated. The electronic device includes:

[0185] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0186] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the rock slope joint crack detection method of the embodiment of the present invention;

[0187] Input / output interface 1003, used to implement information input and output;

[0188] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0189] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0190] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0191] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned rock slope joint crack detection method is implemented.

[0192] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0193] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0194] The rock slope joint crack detection method, rock slope joint crack detection device, equipment and medium provided by the embodiment of the present invention obtain rock images and preprocess each channel of the rock images; stack the channels of the preprocessed rock images and input them into a preset feature extractor for feature extraction, and output a feature vector; the feature extractor includes an encoder, and the encoder is trained based on a cascaded decoder; cluster the feature vectors of different rock images through self-organizing mapping to obtain the background category of each rock image; pseudo-label the corresponding rock images based on the background category; classify and train the preconfigured deep learning network based on the rock images marked with background categories or crack labels; and use the trained deep learning network to perform crack detection on the target rock image to be detected to obtain the detection result. The embodiment of the present invention adds background classification, and by increasing the number of background classifications about non-crack categories, the convolutional neural network is driven to learn non-crack features and improve crack detection performance. The embodiment of the present invention can accurately detect rock slope joint cracks.

[0195] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art can appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0196] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0197] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0198] Those skilled in the art will appreciate that all or some of the steps, devices, and functional modules / units in the above disclosed methods may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0199] The terms "first", "second", "third", "fourth", etc. (if any) in the description of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0200] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0201] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0202] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0203] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0204] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store programs.

[0205] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the embodiments of the present invention is not limited thereby. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present invention shall be within the scope of the rights of the embodiments of the present invention.

Claims

1. A method for detecting cracks in rock slope joints, characterized in that: The method comprises the following steps: Acquire a rock image, and preprocess each channel of the rock image; The channels of the preprocessed rock images are stacked and input into a preset feature extractor for feature extraction, and a feature vector is obtained as output; the feature extractor includes an encoder, and the encoder is obtained by training based on a cascaded decoder; Clustering the feature vectors of different rock images through self-organizing mapping to obtain a background category of each rock image; pseudo-labeling the corresponding rock image based on the background category; Performing classification training on a preconfigured deep learning network based on the rock image marked with the background category or crack label; The trained deep learning network is used to perform crack detection on the target rock image to obtain the detection result.

2. The method according to claim 1, characterized in that The preprocessing of each channel of the rock image comprises the following steps: Performing filtering processing on each channel of the rock image by using a Wiener filter; Normalization is applied to each of the channels of the rock image after the filtering process, and the maximum intensity and the minimum intensity of each channel are adjusted.

3. The method according to claim 1, characterized in that The method further comprises the following steps: Constructing a CAE architecture through the encoder and the decoder; The encoder includes a first encoder layer and a second encoder layer, and the first encoder layer and the second encoder layer are both configured with filters; the number of filters in the first encoder layer is less than the number of filters in the second encoder layer; the decoder includes a hierarchical structure symmetrical to the encoder and a Sigmoid function layer; Using preset image data combined with a preset loss function to train the CAE architecture to minimize the difference between input and output images, and then optimizing and adjusting the parameters of the encoder based on the training results to obtain the feature extractor; The loss function includes mean absolute error and mean square error.

4. The method according to claim 3, characterized in that Before the step of building the CAE architecture, the method further includes the following steps: Inserting a convolutional layer after the second encoder layer and inserting the convolutional layer before the first layer of the decoder; And / or, configuring a rectified linear unit activation function and batch normalization in each layer in the encoder and the decoder.

5. The method according to claim 1, characterized in that The method further comprises the following steps: Determine a classification category based on the total number of the background categories; wherein the classification category is one more than the total number of the background categories, and the extra category represents a crack category; The deep learning network is configured based on the classification categories in combination with a deep CNN architecture.

6. The method according to claim 5, characterized in that The classification training of a preconfigured deep learning network based on the rock image marked with the background category or crack label comprises the following steps: Inputting the rock image into the deep learning network and outputting a classification result; The parameters of the deep learning network are adjusted based on the category result and the loss result of the background category or the crack label marked in the rock image.

7. The method according to claim 1, characterized in that The method of using the trained deep learning network to perform crack detection on the target rock image to be detected to obtain a detection result includes the following steps: Using the trained deep learning network to perform crack detection on the target rock image to be detected, and outputting classification category probabilities; the classification category probabilities include the probabilities of each background category and the probabilities of crack categories; Threshold processing is performed on the classification category probability to obtain the detection result of the target rock image.

8. A rock slope joint crack detection device, characterized in that: The device comprises: The first module is used to obtain a rock image and preprocess each channel of the rock image; The second module is used to stack the channels of the preprocessed rock image and input them into a preset feature extractor for feature extraction, and output a feature vector; the feature extractor includes an encoder, and the encoder is obtained based on cascaded decoder training; The third module is used to cluster the feature vectors of different rock images through self-organizing mapping to obtain the background category of each rock image; and pseudo-label the corresponding rock image based on the background category; A fourth module is used to perform classification training on a preconfigured deep learning network based on the rock image marked with the background category or crack label; The fifth module is used to use the trained deep learning network to perform crack detection on the target rock image to be detected to obtain the detection result.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images