A rock slope joint crack detection method, device, equipment and medium

Abstract

The present invention discloses a method, device, equipment, and medium for detecting joint cracks in rock slopes. The method comprises: acquiring a rock image and preprocessing each channel of the rock image; stacking the channels of the preprocessed rock image and inputting them into a preset feature extractor for feature extraction, outputting a feature vector; the feature extractor includes an encoder, which is trained based on a cascaded decoder; clustering the feature vectors of different rock images through a self-organizing map to obtain a background category for each rock image; pseudo-labeling the corresponding rock images based on the background category; training a preconfigured deep learning network for classification based on rock images labeled with background categories or crack labels; and performing crack detection on the target rock image to be detected using the trained deep learning network to obtain a detection result. The present invention can accurately detect joint cracks in rock slopes and can be widely applied in the field of image processing technology.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and medium for detecting joint cracks in a rock slope. Background Art

[0002] Rock slopes often develop cracks during use or construction due to structural degradation or environmental changes. Continued crack propagation accelerates structural degradation, leading to failure or unreliability of the system. Cracks are one of the earliest indicators of structural degradation. Therefore, accurate and timely crack detection is of great practical value for preventing rock slopes and other structures from suffering further damage or potential accidents. However, existing research focuses on supervised convolutional neural networks (CNNs) that lack non-crack features or unsupervised crack analysis with limited accuracy, resulting in poor results for detecting cracks in rock slope joints. Summary of the Invention

[0003] The main purpose of the embodiments of the present invention is to provide a method, device, equipment and medium for detecting joint cracks in rock slopes, in order to solve at least one problem of the prior art. The present invention can accurately detect joint cracks in rock slopes.

[0004] To achieve the above-mentioned purpose, one aspect of an embodiment of the present invention provides a method for detecting cracks in rock slope joints, the method comprising:

[0005] Acquire rock images and preprocess each channel of the rock images;

[0006] The channels of the preprocessed rock image are stacked and input into a preset feature extractor for feature extraction, and a feature vector is output; the feature extractor includes an encoder, which is trained based on a cascaded decoder;

[0007] The feature vectors of different rock images are clustered by self-organizing mapping to obtain the background category of each rock image; the corresponding rock images are pseudo-labeled based on the background category;

[0008] A pre-configured deep learning network is trained for classification based on rock images labeled with background categories or crack labels.

[0009] The trained deep learning network is used to perform crack detection on the target rock image to obtain the detection results.

[0010] In some embodiments, preprocessing is performed on each channel of the rock image, including the following steps:

[0011] Each channel of the rock image is filtered using a Wiener filter;

[0012] Normalization is applied to each channel of the filtered rock image to adjust the maximum intensity and minimum intensity of each channel.

[0013] In some embodiments, the method further comprises the following steps:

[0014] Build CAE architecture through encoder and decoder;

[0015] The encoder includes a first encoder layer and a second encoder layer, both of which are configured with filters; the number of filters in the first encoder layer is smaller than the number of filters in the second encoder layer; the decoder includes a hierarchical structure symmetrical to the encoder and a Sigmoid function layer;

[0016] The CAE architecture is trained using preset image data combined with a preset loss function to minimize the difference between input and output images. The encoder parameters are then optimized based on the training results to obtain a feature extractor.

[0017] Among them, the loss functions include mean absolute error and mean square error.

[0018] In some embodiments, before the step of building the CAE architecture, the method further includes the following steps:

[0019] Insert a convolutional layer after the second encoder layer and a convolutional layer before the first decoder layer;

[0020] And / or, configure rectified linear unit activation functions and batch normalization in each layer in the encoder and decoder.

[0021] In some embodiments, the method further comprises the following steps:

[0022] Determine the classification category based on the total number of background categories; wherein the classification category is one more than the total number of background categories, and the extra category represents the crack category;

[0023] Configure deep learning networks based on classification categories combined with deep CNN architecture.

[0024] In some embodiments, classification training of a preconfigured deep learning network based on rock images labeled with background categories or crack labels includes the following steps:

[0025] Input the rock image into the deep learning network and output the classification result;

[0026] The parameters of the deep learning network are adjusted based on the category results and the loss results of the background category or crack label of the rock image.

[0027] In some embodiments, crack detection is performed on a target rock image using a trained deep learning network to obtain a detection result, including the following steps:

[0028] The trained deep learning network is used to perform crack detection on the target rock image to be inspected, and the classification probability is output; the classification probability includes the probability of each background category and the probability of the crack category;

[0029] Threshold processing is performed on the classification category probability to obtain the detection result of the target rock image.

[0030] To achieve the above-mentioned purpose, another aspect of the present invention provides a rock slope joint crack detection device, comprising:

[0031] The first module is used to obtain rock images and preprocess each channel of the rock images;

[0032] The second module is used to stack the channels of the preprocessed rock image and input them into a preset feature extractor for feature extraction, and output a feature vector; the feature extractor includes an encoder, which is trained based on a cascaded decoder;

[0033] The third module is used to cluster the feature vectors of different rock images through self-organizing mapping to obtain the background category of each rock image; and pseudo-label the corresponding rock images based on the background category;

[0034] The fourth module is used to train a pre-configured deep learning network for classification based on rock images labeled with background categories or crack labels;

[0035] The fifth module is used to use the trained deep learning network to perform crack detection on the target rock image to be detected and obtain the detection results.

[0036] In some embodiments, the time warning threshold includes a sleep time warning threshold and a reaction time warning threshold; the device further comprises:

[0037] The sixth module is used to build a CAE architecture through encoders and decoders;

[0038] The encoder includes a first encoder layer and a second encoder layer, both of which are configured with filters; the number of filters in the first encoder layer is smaller than the number of filters in the second encoder layer; the decoder includes a hierarchical structure symmetrical to the encoder and a Sigmoid function layer;

[0039] The seventh module is used to train the CAE architecture using preset image data combined with a preset loss function to minimize the difference between input and output images, and then optimize the encoder parameters based on the training results to obtain a feature extractor;

[0040] Among them, the loss functions include mean absolute error and mean square error.

[0041] In some embodiments, the apparatus further comprises:

[0042] an eighth module for inserting a convolutional layer after the second encoder layer and a convolutional layer before the first decoder layer;

[0043] and / or, a ninth module for configuring a rectified linear unit activation function and batch normalization in each layer in the encoder and decoder;

[0044] The eighth and ninth modules are executed before the sixth module builds the CAE architecture.

[0045] To achieve the above object, another aspect of an embodiment of the present invention provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0046] To achieve the above object, another aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above method is implemented.

[0047] Embodiments of the present invention include at least the following beneficial effects: The present invention provides a method, apparatus, device, and medium for detecting cracks in rock slope joints. This solution obtains rock images and preprocesses each channel of the rock images. The preprocessed rock image channels are stacked and input into a preset feature extractor for feature extraction, outputting a feature vector. The feature extractor includes an encoder trained based on a cascade of decoders. Feature vectors of different rock images are clustered using a self-organizing map to obtain a background category for each rock image. The corresponding rock images are pseudo-labeled based on the background category. A preconfigured deep learning network is trained for classification based on rock images labeled with background categories or crack labels. The trained deep learning network is used to perform crack detection on the target rock image to be detected, obtaining a detection result. This embodiment of the present invention adds background classification. By increasing the number of background classifications related to non-crack categories, the convolutional neural network is driven to learn non-crack features and improve crack detection performance. This embodiment of the present invention enables accurate detection of cracks in rock slope joints. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1This is a flow chart of a rock slope joint crack detection method provided by an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of an example of a CAE architecture provided by an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the process architecture of a rock slope joint crack detection method provided by an embodiment of the present invention;

[0051] Figure 4 A schematic structural diagram of a rock slope joint crack detection device provided by an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the appended claims.

[0054] It will be understood that the terms "first," "second," and the like used in the present invention may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are merely used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of," "when," or "in response to a determination."

[0055] The terms "at least one", "plurality", "each", "any", etc. used in the present invention include at least one, two or more, multiple, two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0057] The rock slope joint crack detection method provided by the embodiment of the present invention relates to the field of image processing technology. The rock slope joint crack detection method provided by the embodiment of the present invention can be applied to a terminal, can be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the rock slope joint crack detection method, etc., but is not limited to the above forms.

[0058] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0059] Figure 1 This is an optional flow chart of a rock slope joint crack detection method provided by an embodiment of the present invention. Figure 1 The method may include but is not limited to steps S100 to S500.

[0060] S100, acquiring a rock image and preprocessing each channel of the rock image;

[0061] It should be noted that, in some embodiments, preprocessing each channel of the rock image may include the following steps: filtering each channel of the rock image through a Wiener filter; applying normalization to each channel of the filtered rock image, and adjusting the maximum intensity and minimum intensity of each channel.

[0062] For example, in some embodiments, preprocessing is applied to the entire image. Preprocessing is mandatory for effective unsupervised feature extraction. Research has found that pixel-level speckle-like noise dominates images of rock slope joints. Preprocessing is applied to each RGB channel, with a Wiener filter and normalization. The filter size is 16 by 16 pixels. The Wiener filter is an adaptive filter. It strongly smoothes noisy areas, while weakly smoothes cracks and feature areas. Noise is reduced while cracks remain clear. Larger filter sizes produce smoother images. Excessively large filter sizes may obscure crack features. The results are size-insensitive. A total of 16 pixels is sufficient. Normalization is applied to each RGB channel, adjusting the maximum intensity to 1 and the minimum intensity to 0. This process aims to eliminate the influence of lighting conditions and color on rock slope joints. Without the Wiener filter and normalization, the feature extractor can learn differences in noise levels and color as important features. After preprocessing, the channel images are stacked for input into the feature extractor.

[0063] S200, stacking the channels of the preprocessed rock image and inputting them into a preset feature extractor for feature extraction, and outputting a feature vector;

[0064] The feature extractor includes an encoder, which is trained based on a cascaded decoder;

[0065] It should be noted that, in some embodiments, the method may further include the following steps: constructing a CAE architecture through an encoder and a decoder; wherein, the encoder includes a first encoder layer and a second encoder layer, and the first encoder layer and the second encoder layer are both configured with filters; the number of filters in the first encoder layer is less than the number of filters in the second encoder layer; the decoder includes a hierarchical structure and a Sigmoid function layer symmetrical to the encoder; using preset image data combined with a preset loss function to train the CAE architecture to minimize the difference between input and output images, and then optimizing and adjusting the encoder parameters based on the training results to obtain a feature extractor; wherein the loss function includes mean absolute error and mean square error.

[0066] For example, in some specific embodiments, Figure 2 Figure 2 shows the proposed five-layer CAE architecture. The input and output of the CAE are 128×128 pixel RGB color three-channel images. The goal of the CAE is to extract spatial features to compress the image. At the same time, the reconstructed image from the compressed features should be as similar as possible to the input image to confirm that the extracted features are valid. The CAE consists of an encoder and a decoder. The optimized encoder corresponds to the feature extractor. The following symmetrically structured decoder layer is also trained to decompress the feature map to reconstruct the image.

[0067] The CAE architecture consists of an encoder and a decoder. The encoder compresses an input image with dimensions of 128 by 128 pixels and three channels to output 15 feature maps with dimensions of 32 by 32. The decoder decompresses the feature maps to output an image, thereby reconstructing the input image. The optimized encoder acts as a feature extractor.

[0068] Two important parameters should be considered: the number of layers and the choice of loss function. This is a trade-off; the deeper the CAE, the smaller the compressed data, but the greater the potential loss of information. The CAE is trained to minimize the difference between the input and output images. An appropriate loss function should be specified to develop an effective autoencoder.

[0069] In terms of the number of layers, the input image has a size of approximately 49,000. It was observed that analyzing these features took several days; computation time increases dramatically around this number. The first encoder layer computes spatial features from the input image to convert it into 10 feature maps of size 64 by 64. Max pooling is used to suppress the map size, but this size is still around 41,000. The second encoder layer further compresses the features into 15 feature maps of size 32 by 32. The size is 15,000, which is still large. However, it is one-third the size of the original image and is relatively lightweight.

[0070] The autoencoder is trained using the mean absolute error (MAE). MAE is the sum of the absolute differences in the intensity of the corresponding pixels (x,y) between the original image and the reconstructed image, divided by the number of pixels N. The expression for MAE is as follows:

[0071]

[0072] Where, I o -I r Represents the absolute difference in intensity at pixel (x,y).

[0073] A convolutional layer is inserted after the second encoder layer with the same architecture, and a convolutional layer is inserted before the first decoder layer with the same structure (seven-layer CAE). By adding these layers, the loss of information is obvious. Another typical loss function is the mean squared error (MSE).

[0074] Among them, in some embodiments, before the step of constructing the CAE architecture, the method may also include the following steps: inserting a convolutional layer after the second encoder layer and inserting a convolutional layer before the first layer of the decoder; and / or, configuring a rectified linear unit activation function and batch normalization in each layer in the encoder and decoder.

[0075] For example, in some specific embodiments, the number of filters and the size of the filters are also important parameters. These parameters should be large enough to extract image features. The parameters are selected to confirm that the loss value converges. The number of filters in the first layer of the encoder and the second layer of the decoder is smaller than the number of filters in the second level of the encoder and the first level of the decoder. This is to avoid a sudden reduction in mapping information. With respect to other typical DL techniques, a rectified linear unit (ReLU) activation function is adopted, and batch normalization is used in the encoder part to output stable results and avoid saturation.

[0076] S300, clustering the feature vectors of different rock images through self-organizing mapping to obtain a background category of each rock image; and pseudo-labeling the corresponding rock image based on the background category;

[0077] For example, in some embodiments, a self-organizing map (SOM) algorithm automatically optimizes the map size and clusters the features output by the developed CAE. Non-crack images are pseudo-labeled based on the resulting clusters. The labels are denoted as background 1, background 2, ..., background n.

[0078] Among them, the SOM (Self-Organizing Map) is a typical map with a hexagonal honeycomb structure. A weight vector is assigned to each cell of the structure. The input feature vector is classified into one of the cells that matches the input vector and weight vector. After matching, the corresponding weight vector and the weight vectors of the corresponding cells are updated to increase the matching coefficient. This is a mutually optimizing process. After repeating the classification of input vectors and the updating of weight vectors, similar input vectors are stored in a close proximity. The distribution of grouped input vectors in the map allows visualization of the structure of the data. The distance between adjacent weight vectors indicates the similarity of adjacent cells.

[0079] S400, performing classification training on a preconfigured deep learning network based on a rock image marked with a background category or a crack label;

[0080] In some embodiments, the method may further include the following steps: determining a classification category based on the total number of background categories; wherein the classification category is one more than the total number of background categories, and the extra category represents a crack category; configuring a deep learning network based on the classification category in combination with a deep CNN architecture.

[0081] It should be noted that, in some embodiments, step S400 may include: inputting the rock image into the deep learning network and outputting a classification result; and adjusting the parameters of the deep learning network based on the classification result and the loss result of the background category or crack label marked in the rock image.

[0082] For example, in some specific implementations, conventional deep learning (DL) classification can be employed. Given the addition of n background categories and crack images, an N = n + 1 classification network is trained to learn detailed features of non-crack images. This approach also improves the ability to distinguish between cracked and non-cracked images. The architecture can be a typical deep CNN architecture, such as VGG16 from DL using ImageNet parameters.

[0083] S500: Use the trained deep learning network to perform crack detection on the target rock image to be detected to obtain a detection result.

[0084] It should be noted that, in some embodiments, step S500 may include: using a trained deep learning network to perform crack detection on the target rock image to be detected, and outputting classification category probabilities; the classification category probabilities include the probabilities of each background category and the probabilities of the crack category; and performing threshold processing on the classification category probabilities to obtain the detection results of the target rock image.

[0085] For example, in some embodiments, the detection outputs the probabilities of N categories. The probability of the crack category is thresholded to determine the presence of a crack. This probability indirectly takes into account all remaining non-crack categories.

[0086] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0087] First, it should be noted that the presence of joint cracks in rock slopes is a key indicator of damage. Deep learning is a mainstream approach for crack detection in images due to its detection capabilities. Previous research has focused on supervised convolutional neural networks (CNNs) without non-crack features or on unsupervised crack analysis with limited accuracy. The novelty of this study lies in the addition of background classification. By increasing the number of non-crack categories, the convolutional neural network is driven to learn non-crack features and improve crack detection performance. Crack-free images are preprocessed, and their features are extracted in an unsupervised manner using a deep convolutional autoencoder. Self-organizing maps cluster features to obtain non-crack categories and disentangle how pixel-level fine crack detection results are affected by different loss functions. This study focuses on classification, although the method can be used in parallel with state-of-the-art segmentation algorithms. Compared to previous models, using a common crack dataset, the improved deep CNN model significantly improves accuracy by 1%-4% and f-measure by 3%-8%. A modified Visual Geometry Group (VGG)16 dataset demonstrates top-tier performance, with 96% accuracy and 84%-85% f-measure. These models significantly reduce false detections while maintaining their crack detection capabilities.

[0088] This paper uses unsupervised feature extraction and pseudo-labeling to incorporate background image information into the classification task. This method differs from the conventional "crack" and "non-crack" classification, which does not consider the differences in spatial features in the background image.

[0089] By leveraging the classification capabilities of pseudo-labeled supervised deep CNNs, we achieve state-of-the-art classification accuracy compared to previous CNNs and unsupervised methods. State-of-the-art CNN architectures can be used with minor modifications to the original CNNs. Training time is nearly identical to previous CNNs.

[0090] The method of the present invention is as follows Figure 3 As shown. The method includes two steps. In the first step, the non-crack image is interpreted. In order to perform unsupervised analysis effectively, preprocessing is very important. The present invention adopts Wiener filter and normalization. Using an optimized convolutional autoencoder (CAE) architecture, the features of patterns and objects, such as shadow areas and speckled paint and dust, are analyzed in an unsupervised manner. The SOM algorithm automatically optimizes the mapping size and clusters the features output by the developed CAE. The non-crack image is pseudo-labeled with reference to the obtained clusters. The labels are denoted as background 1, background 2, ..., background n.

[0091] Specifically, refer to Figure 3The method of the present invention includes two steps: top: background analysis through preprocessing, convolutional autoencoder (CAE) and self-organizing map (SOM), bottom: crack detection through convolutional neural network (CNN).

[0092] The second step (below) is conventional deep learning (DL) classification. Given the addition of n background categories and crack images, an N = n + 1 classification network is trained to learn detailed features of non-crack images. This approach also improves the ability to distinguish between cracked and non-cracked images. The architecture is a typical deep CNN architecture, such as the VGG16 from DL with ImageNet parameters. It outputs probabilities for N classes. The crack class probability is thresholded to determine the presence of a crack. This probability indirectly accounts for all remaining non-crack classes.

[0093] All CNNs and CAEs used in this invention were carefully considered for their performance. The selection of preprocessing filters, CAE architecture, SOM size, CNN architecture, and hyperparameters are important parameters. Each process should be optimized to achieve the highest performance. The method consists of two parts: feature extraction and classification criterion construction. The method uses supervised learning of labeled data. In any case, one category is pseudo-labeled through unsupervised feature extraction and clustering. Therefore, a fully supervised two-class classification and an unsupervised feature extraction and classification algorithm should be compared to highlight the effectiveness of the proposed method.

[0094] based on Figure 3 Based on the improved U-Net architecture in this paper, the effects of six different loss functions on pixel-level thin crack detection are first explored. The experimental loss functions include L bce or binary cross entropy loss, L dice or dice loss, L dice_bce or a mixture of Dice and binary cross entropy losses, L iou Or intersection-of-union (iou) loss, L log_iou Or the negative logarithmic loss about IoU, and L bf Or binary focal loss. L bce is the baseline loss function for obtaining comparison results with the modified U-Net. bce , other experimental loss functions are well known in segmentation tasks or class imbalance tasks. For example, the Dice coefficient (also known as –Dice coefficient) is a common metric for binary classification tasks such as pixel segmentation, and IoU (also known as Jaccard index) is another popular segmentation metric. Therefore, L is chosen here. dice and L iou The purpose is to directly optimize the F1 performance during training. dice_bce In order to obtain more diversity, we try to use a composite loss function, and L log_iou is another attempt to smooth the loss surface of iou values. bf It is a binary classification version of the well-known focal loss function, which directly addresses the extreme class imbalance problem.

[0095] Assume N px is the number of pixels in the image (same for all input images, ground truth images, and output images),

[0096] ∧

[0097] And y i and y i Denote the ground truth value and the related predicted value of the i-th pixel in the ground truth image and the predicted image respectively. Then, the experimental loss function is calculated according to equations (1)-(6). For L bce and L iou , use the original formula without modification. For L dice , instead of the original equation, a simplified version is used so that the denominator is a simple sum, not a sum of squares. dice_bce , the sum of the two losses is equal. For L bf , using a constant value of α = 3 and three different values of γ = {1, 2, 4}; the three binary focal losses will be referred to as L bf1 、L bf2 and L bf4 . In addition, since there are many alternative implementations of binary focusing loss, preliminary experiments were performed to select an appropriate implementation as written in Equation (6). Note that in Equation (6), L bf_pos is the loss calculated from only positive (crack) pixels, where y i =1(y pos ), L bf_neg is the loss calculated from only negative (non-crack) pixels, where y i =0(y neg ).

[0098]

[0099]

[0100] L dice_bce =L dice +Lbce (3)

[0101]

[0102] L bf =L bf_pos +L bf_neg (6)

[0103]

[0104] Specifically, this section compares feature extraction methods for detecting cracks in rock slope joints using CAE. A Wiener filter is applied, and the image is normalized. In addition to CAE, typical image processing methods are also compared for feature extraction. For comparison purposes, the extracted features are connected to a support vector machine (SVM) with a nonlinear kernel to develop a classifier. The specific implementation is as follows:

[0105] 1. Preprocessing:

[0106] Preprocessing is applied to the entire image. Preprocessing is mandatory for effective unsupervised feature extraction. Research has found that pixel-level speckle-like noise dominates images of rock slope joints. Preprocessing is applied to each RGB channel, with a Wiener filter and normalization. The filter size is 16 by 16 pixels. The Wiener filter is an adaptive filter. It strongly smoothes noisy areas, while weakly smoothes cracks and feature areas. Noise is reduced while cracks remain clear. Larger filter sizes produce smoother images. Excessively large filter sizes may obscure crack features. The results are size-insensitive. A total of 16 pixels is sufficient. Normalization is applied to each RGB channel, adjusting the maximum intensity to 1 and the minimum intensity to 0. This process aims to eliminate the influence of lighting conditions and color on rock slope joints. Without the Wiener filter and normalization, the feature extractor can learn differences in noise levels and color as important features. After preprocessing, the channel images are stacked for input into the feature extractor.

[0107] Figure 2 The proposed five-layer CAE architecture is shown. The input and output of the CAE are 128×128 pixel RGB three-channel images. The goal of the CAE is to extract spatial features to compress the image. At the same time, the reconstructed image from the compressed features should be as similar as possible to the input image to confirm that the extracted features are valid. The CAE consists of an encoder and a decoder. The optimized encoder corresponds to the feature extractor. The following symmetrical structure decoder layer is also trained to decompress the feature map to reconstruct the image.

[0108] The architecture consists of an encoder and a decoder. The encoder compresses an input image with a size of 128 by 128 pixels and three channels to output 15 feature maps with a size of 32 by 32. The decoder decompresses the feature maps to output an image, thereby reconstructing the input image. The optimized encoder is a feature extractor.

[0109] Two important parameters should be considered: the number of layers and the choice of loss function. This is a trade-off; the deeper the CAE, the smaller the compressed data, but the greater the potential loss of information. The CAE is trained to minimize the difference between the input and output images. An appropriate loss function should be specified to develop an effective autoencoder.

[0110] In terms of the number of layers, the input image has a size of approximately 49,000. It was observed that analyzing these features took several days; computation time increases dramatically around this number. The first encoder layer computes spatial features from the input image to convert it into 10 feature maps of size 64 by 64. Max pooling is used to suppress the map size, but this size is still around 41,000. The second encoder layer further compresses the features into 15 feature maps of size 32 by 32. The size is 15,000, which is still large. However, it is one-third the size of the original image and is relatively lightweight.

[0111] The autoencoder is trained using the mean absolute error (MAE). MAE is the sum of the absolute differences in the intensity of the corresponding pixels (x,y) between the original image and the reconstructed image, divided by the number of pixels N.

[0112]

[0113] A convolutional layer is inserted after the second encoder layer with the same architecture, and a convolutional layer is inserted before the first decoder layer with the same structure (seven-layer CAE). By adding these layers, the loss of information is obvious. Another typical loss function is the mean squared error (MSE).

[0114] The number of filters and filter size are also important parameters. These parameters should be large enough to extract image features. The parameters are selected to confirm the convergence of the loss value. The number of filters in the first layer of the encoder and the second layer of the decoder is smaller than the number of filters in the second level of the encoder and the first level of the decoder. This is to avoid a sudden decrease in mapping information. Regarding other typical DL techniques, the rectified linear unit (ReLU) activation function is adopted, and batch normalization is used in the encoder part to output stable results and avoid saturation.

[0115] 2. Comparison of unsupervised methods:

[0116] In some implementations, attempts were made to develop a fully unsupervised crack detection algorithm by combining an unsupervised feature extractor with an unsupervised clustering algorithm. Clusters were then interpreted based on the clustered images. This strategy naturally incorporates features from both cracked and non-cracked images. The unsupervised feature extractor discussed in this section was combined with the SOM discussed in the next section. However, the classifier achieved classification accuracy below 60%.

[0117] The next strategy is to feed the extracted features into a supervised SVM with a nonlinear kernel function. The SVM is trained with features of crack and non-crack images and labels to compare the performance of the developed feature extractor. Figure 2 The output of the feature extractor, including the encoder shown, is flattened to construct a feature vector. Support Vector Machines (SVMs) require less data than DL methods because small support vectors define the classification criterion. The kernel function used is a radial basis function, which constructs a nonlinear partitioning plane. For reproducibility, additional parameters are added: 2% of outlier vectors are removed. A regularization factor of 0.5 is used. A logistic function is fitted to output the probability of a crack.

[0118] For comparison, we considered HOG features as typical image eigenvalues, Gabor filters as typical frequency filters, Hough transforms after edge detection using the Canny method, and simple average pooling as typical image processing methods. In addition to the aforementioned features, extensive comparisons were conducted with image processing techniques such as the Fourier transform and Speeded Up Robust Features (SURF), yielding similar results. The Gabor filter is a set of 2D filters that emphasize certain frequency components. It is related to the Fourier transform. The HOG feature is a fast version of the SURF feature. Therefore, the Gabor filter and HOG features were compared to represent frequency analysis and image eigenvalue methods.

[0119] For reproducibility, the feature extractor parameters are: the cell size of the HOG feature is 32 by 32; the wavelength and orientation of the Gabor filter are 10, 20, and 30 pixels and 0, 30, 60, 90, 120, and 150 degrees, and the filtered image is downsampled to one-quarter; the Canny method has a threshold of 0.7, and the resolution of rho and theta is 2 pixels and 2 degrees, respectively. Mean age pooling divides the image into four 32 by 32 pixel images and calculates the average of each image in each RGB channel. This is the most primitive downsampling method. The original image size is too large to be acceptable for the SVM. The size of these features is comparable to the size of the encoder.

[0120] The proposed encoder achieved the highest accuracy of 92% and f-measure of 69%. The accuracy of HOG features, Gabor filter, and Hough+Canny were 20%, 10%, and 26% lower than that of CAE, respectively. The f-measures decreased by 20%, 12%, and 26%, respectively. The extent of the reduction may depend on the dataset. In the case of the dataset considered, these features are not suitable for extracting thin cracks from noisy asphalt pavement patterns. These features are inferior to simple average pooling. Average pooling takes into account the dark areas of cracks and their combinations. On the other hand, CAE effectively extracts the spatial features of cracks and pavement patterns. The accuracy and f-measure of CAE are 6% higher than those of average pooling. Conclusion CAE is the most effective unsupervised crack feature extraction method.

[0121] However, the performance of the best CAE differs from that of state-of-the-art deep CNNs. This is because the supervised DL architecture with TL accurately learns crack features and builds precise classification criteria within the network. However, supervised CNNs may overlook details of non-crack images during training. Therefore, the proposed method adopts SOM to construct finer non-crack categories to encourage CNN to learn features of non-crack images by leveraging the strengths of both unsupervised and supervised methods.

[0122] Since the true structure of the non-crack image data is unknown, it is impossible to quantitatively discuss the impact of misclassified pseudo-labels on CNNs using unsupervised methods. In the worst case, if all non-crack images are randomly classified, the training of the CNN will fail. The improvement in accuracy shown by using multiple categories means that pseudo-labeling has some effectiveness. However, in the field of DL research, the contamination of a dataset with 8% imprecise labels (92% accuracy) is generally not negligible. Better feature extractors have the potential to lead to better CNN performance because appropriate features help CNNs accurately classify features. The best CAE was used in this study, although the CAE can be further improved by adopting a carefully designed architecture.

[0123] In some specific implementations, embodiments of the present invention also provide a class-specific image augmentation (CSIA). As an important data-centric method, image augmentation is widely used to increase training datasets. In this work, the Wasserstein generative adversarial network (wGAN) is modified and used to generate realistic top-down crack images.

[0124] The main difference between wGAN and traditional GAN is that wGAN uses the Wasserstein distance metric to measure the distance between generated images and real images, which makes wGAN training more stable. In wGAN, the generator and discriminator are trained to optimize the Wasserstein distance between generated images and real images instead of minimizing the cross-entropy loss. This leads to better training stability, clearer and more realistic images, and better control over the diversity of generated images.

[0125] This work involves training a wGAN for a dataset. The wGAN is trained to generate images of a specific class. This approach works well when there are significant differences between classes. By training a separate wGAN for each class, the generated images can be optimized for that specific class, resulting in high-quality, realistic images. The wGAN was trained for 20,000 epochs to generate images.

[0126] In deep learning, it's common to add images equally to each class. However, in real life, data isn't always evenly distributed. For example, linear cracks are more common than alligator and block cracks. Furthermore, different damage types are more discriminative, as seals and repairs are more discriminative than cracks. To determine how many images to add to each class, we propose a class-specific image augmentation (CSIA) method as an optimistic augmentation approach.

[0127] The following steps outline the implementation process of the CSIA approach.

[0128] First, the loss value for each class can be calculated using Equation (7). Based on the benchmark study, the YOLOv5s model was trained and tested on the original training and testing datasets with 300 epochs, a batch size of 64, and an initial learning rate of 0.01.

[0129] Loss k =1-mAP50 k (7)

[0130] Among them, Loss k is the loss for class k; mAP50 k is the mAP50 of class k in the benchmark. Smaller loss values mean less need for image augmentation in this case.

[0131] The distribution of the original data is also an important factor in the image enhancement strategy. The weight score of each class is calculated based on Equation (8).

[0132]

[0133] Where Ok is the original number of class k, W k is the weight score of class k.

[0134] Then, the number of enhanced images of category k can be determined by Equation (9).

[0135]

[0136] Among them, A k is the number of augmented images of class k, and N is the total number of images planned to be augmented in the deep learning task (1400 in this work). According to Equation (5), in this CSIA method, both the distribution of the original data and the distinction between different classes are considered to optimize the number of generated images.

[0137] Among them, in some preferred implementations, the embodiments of the present invention also provide an orthogonal test for parameter fine-tuning, which is specifically implemented as follows:

[0138] To find out the impact of parameters including epochs, batch size, and learning rate on the performance of the YOLOv5s model, orthogonal experiments were considered to study the impact of each factor. Orthogonal experiments are a statistical method that can help reduce the number of experiments required to identify the best combination of parameters by systematically varying the parameters to maximize the amount of information gained from each experiment. It allows for the optimal parameter values to be determined with fewer experiments, significantly saving time and resources.

[0139] Epochs (Factor A), batch size (Factor B), and initial learning rate (Factor C) were selected as the three factors believed to have an impact on the accuracy of the YOLOv5 model. The epoch levels were 200, 300, and 400, respectively. The batch sizes were 16, 32, and 64. The initial learning rate levels were 1e-1, 1e-2, and 1e-3. The orthogonal experiment in this study was designed with the help of a 3-factor, 3-level orthogonal table.

[0140] Analysis of variance (ANOVA) is used to calculate the impact of each factor on the model's predictive performance. This statistical method is used to demonstrate the significance of different factors based on the results of an orthogonal experiment. This method can be described by the following equation.

[0141] SS T =SS f +SS e (10)

[0142] Among them SS T is the sum of squares of the sum, SS f is the sum of squares of the regression. SS e is the sum of squares of errors;

[0143] df T =df f +df e (11)

[0144] where df T is the total degree of freedom (DF), df f is a factor DF, df e is the DF of the error;

[0145] The mean square of each factor (MSf), the mean square of each error (MSe), and the F statistic of each factor (F) can be obtained using equations (12) to (14).

[0146]

[0147] In ANOVA, the critical F value (Fc) is used to compare with the calculated F statistic to determine whether the observed differences between groups are statistically significant. A significance level (α) of 0.05 was used in this work, which represents a 5% probability of making a Type I error (rejecting the null hypothesis when it is true). The critical F value is chosen based on the chosen α level and the degrees of freedom associated with the test.

[0148] Specifically, the non-crack image analysis provided by the embodiment of the present invention is specifically implemented as follows:

[0149] 1. Theory:

[0150] The SOM was proposed by Kohonen. A typical mapping is a hexagonal honeycomb structure. A weight vector is assigned to each cell of the structure. An input feature vector is classified into a cell that matches the input vector and weight vector. After matching, the corresponding weight vector and the weight vectors of cells near the corresponding cell are updated to increase the matching coefficient. This is a mutually optimizing process. After repeated classification of input vectors and updating of weight vectors, similar input vectors are stored in close proximity. The distribution of grouped input vectors in the mapping allows visualization of the data structure. The distance between adjacent weight vectors indicates the similarity of adjacent cells.

[0151] Two key points of the SOM are how to select appropriate eigenvectors and map size. The previous section discussed appropriate eigenvectors. Regarding map size, the number of vectors in a cell can be too small or too large due to being too large or too small. In this case, it is difficult to find meaningful structure in the map. The map size is automatically determined and features are automatically clustered. Important eigenvalues are selected and thresholds are set. The MATLAB library provided by the research team was utilized.

[0152] 2. Mapping results:

[0153] The input was the feature vectors of a non-cracked image extracted by the developed CAE. The estimated map size was (28, 11). These numbers represent the number of classified vectors in each cell. To reduce computational time, the number of feature vectors was reduced by one-tenth. The computational time after reduction was 2–3 hours. Without this reduction, the computational time would have taken several days and would have been infeasible. It was confirmed that the size and color of the SOM map (the distribution of intercellular distances, U-mat) converged at a certain number of vectors. The color inside a cell corresponds to the average distance of neighboring cells. The size, distribution, and color were automatically estimated in the SOM development.

[0154] Some features consist of 40-50 vectors, while others consist of 5-10 vectors. Similar vectors are grouped into the same cell. For qualitative discussion, two notable features were observed: Feature 1, which has 42 vectors, and Feature 2, which has 51 vectors, due to their large number of graphs. For discussion purposes, these features were manually extracted along with the other features.

[0155] For labeling, based on previous research, automatic clustering was further performed, taking into account the distance between weight vectors. Common features were considered. To account for a wide range of characteristics, the number of clusters should be large. However, this is reduced with a large amount of training data. The number of non-crack images is six to seven times that of crack images. The largest cluster has twice as many members as the other clusters. Given this, five clusters were used.

[0156] The mapping shows that non-crack images have rich features. If more non-crack images are collected, the number of clusters can be increased.

[0157] As shown above, the SOM's discriminative ability is not as high as that of the DL model. A crack / non-crack classification model was established using the SOM. However, its accuracy was very low. Two possible reasons for the SOM's poor accuracy are that the cellular neural network consists of deep convolutional layers with high representation power, and that the cellular neural network can effectively learn crack features with crack labels. Future work could develop advanced autoencoders and unsupervised clustering methods in the DL research field. These methods have the potential to address the challenges of unsupervised learning.

[0158] 3. Explanation:

[0159] SOM is a black-box algorithm; it is unclear what features are considered during the learning process. In this section, the characteristics of the background image of the rock slope joints are qualitatively discussed.

[0160] The R channel image after preprocessing. No difference is observed between the RGB channels. This is because SOM takes into account the contrast of the image; SOM can focus on the fuzziness of edges in shadow images.

[0161] In summary, compared with the prior art, the present invention has at least the following beneficial effects:

[0162] To detect cracks in rock slope joints, an effective preprocessing and feature extraction scheme was proposed. For preprocessing, a Wiener filter and normalization were used. For feature extraction, in addition to CAE, the original image detected using the Canny method was compared with average pooling, histogram-oriented graph (HOG) features, Gabor filters, and Hough transforms of edges.

[0163] The extracted features are quantitatively evaluated and extensively compared using a nonlinear SVM classifier optimized for the crack detection task.

[0164] For the first time, unsupervised analysis of joint cracks in rock slopes was performed based on SOM and a common dataset. The mapping results and example images were discussed to gain a deeper understanding of the image characteristics of joint cracks in crack-free rock slopes.

[0165] For the first time, we propose multi-class deep CNNs that combine unsupervised and supervised learning using CAE, SOM, and CNNs to achieve state-of-the-art crack detection performance by incorporating non-crack image features. The developed deep CNN significantly improves performance compared to recent CNNs such as VGG16 and MobileNetV3. A confusion matrix is presented to discuss the properties of the developed multi-class CNN.

[0166] See also Figure 4 The embodiment of the present invention further provides a rock slope joint crack detection device 900, which can implement the above rock slope joint crack detection method. The device may include:

[0167] The first module 910 is used to obtain a rock image and pre-process each channel of the rock image;

[0168] The second module 920 is used to stack the channels of the preprocessed rock image and input them into a preset feature extractor for feature extraction, and output a feature vector; the feature extractor includes an encoder, and the encoder is trained based on a cascaded decoder;

[0169] The third module 930 is configured to cluster the feature vectors of different rock images through self-organizing mapping to obtain a background category of each rock image; and pseudo-label the corresponding rock image based on the background category;

[0170] The fourth module 940 is used to perform classification training on a preconfigured deep learning network based on the rock image marked with background categories or crack labels;

[0171] The fifth module 950 is used to use the trained deep learning network to perform crack detection on the target rock image to be detected and obtain the detection results.

[0172] In some embodiments, the time warning threshold includes a sleep time warning threshold and a reaction time warning threshold; the apparatus may further include:

[0173] The sixth module is used to build a CAE architecture through encoders and decoders;

[0174] The encoder includes a first encoder layer and a second encoder layer, both of which are configured with filters; the number of filters in the first encoder layer is smaller than the number of filters in the second encoder layer; the decoder includes a hierarchical structure symmetrical to the encoder and a Sigmoid function layer;

[0175] The seventh module is used to train the CAE architecture using preset image data combined with a preset loss function to minimize the difference between input and output images, and then optimize the encoder parameters based on the training results to obtain a feature extractor;

[0176] Among them, the loss functions include mean absolute error and mean square error.

[0177] In some embodiments, the apparatus may further include:

[0178] an eighth module for inserting a convolutional layer after the second encoder layer and a convolutional layer before the first decoder layer;

[0179] and / or, a ninth module for configuring a rectified linear unit activation function and batch normalization in each layer in the encoder and decoder;

[0180] The eighth and ninth modules are executed before the sixth module builds the CAE architecture.

[0181] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0182] An embodiment of the present invention further provides an electronic device comprising a memory and a processor. The memory stores a computer program, and the processor implements the rock slope joint crack detection method when executing the computer program. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0183] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0184] See also Figure 5 , Figure 5 The hardware structure of an electronic device 1000 according to another embodiment is shown. The electronic device includes:

[0185] The processor 1001 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0186] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program codes are stored in the memory 1002 and are called by the processor 1001 to execute the rock slope joint crack detection method of the embodiment of the present invention.

[0187] Input / output interface 1003, used to implement information input and output;

[0188] Communication interface 1004, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0189] Bus 1005 , which transmits information between various components of the device (e.g., processor 1001 , memory 1002 , input / output interface 1003 , and communication interface 1004 );

[0190] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via a bus 1005 .

[0191] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the above-mentioned rock slope joint crack detection method.

[0192] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0193] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0194] Embodiments of the present invention provide a rock slope joint crack detection method, rock slope joint crack detection device, equipment, and medium. These methods acquire rock images and preprocess each channel of the rock images. The preprocessed rock image channels are stacked and input into a preset feature extractor for feature extraction, outputting a feature vector. The feature extractor includes an encoder trained based on a cascade of decoders. Feature vectors of different rock images are clustered using a self-organizing map to obtain a background category for each rock image. The corresponding rock images are pseudo-labeled based on the background category. A preconfigured deep learning network is trained for classification based on rock images labeled with background categories or crack labels. The trained deep learning network is then used to perform crack detection on the target rock image to be detected, yielding a detection result. This embodiment of the present invention adds background classification. By increasing the number of background classifications related to non-crack categories, the convolutional neural network is driven to learn non-crack features and improve crack detection performance. This embodiment of the present invention enables accurate rock slope joint crack detection.

[0195] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0196] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0197] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0198] Those skilled in the art will appreciate that all or some of the steps, devices, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0199] The terms "first," "second," "third," "fourth," and the like (if any) in the description of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0200] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0201] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0202] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0203] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0204] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store programs.

[0205] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the invention is not limited thereby. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the invention should be within the scope of the invention.

Claims

1. A method for detecting cracks in rock slope joints, characterized in that: The method comprises the following steps: Acquire a rock image, and preprocess each channel of the rock image; The channels of the preprocessed rock image are stacked and input into a preset feature extractor for feature extraction, and a feature vector is output; the feature extractor includes an encoder, and the encoder is trained based on a cascaded decoder; Clustering the feature vectors of different rock images through self-organizing mapping to obtain a background category of each rock image; and pseudo-labeling the corresponding rock image based on the background category; Performing classification training on a preconfigured deep learning network based on the rock image marked with the background category or crack label; The method further comprises the following steps: Determining a classification category based on the total number of the background categories; wherein the classification category is one more than the total number of the background categories, and the extra category represents a crack category; Configuring the deep learning network based on the classification categories in combination with a deep CNN architecture; The trained deep learning network is used to perform crack detection on the target rock image to obtain a detection result.

2. The method according to claim 1, characterized in that The preprocessing of each channel of the rock image comprises the following steps: Performing filtering processing on each channel of the rock image by using a Wiener filter; Normalization is applied to each channel of the rock image after the filtering process, and the maximum intensity and the minimum intensity of each channel are adjusted.

3. The method according to claim 1, characterized in that The method further comprises the following steps: Constructing a CAE architecture through the encoder and the decoder; The encoder includes a first encoder layer and a second encoder layer, each of which is configured with filters; the number of filters in the first encoder layer is smaller than the number of filters in the second encoder layer; the decoder includes a hierarchical structure symmetrical to the encoder and a Sigmoid function layer; Using preset image data in combination with a preset loss function to train the CAE architecture to minimize the difference between input and output images, and then optimizing and adjusting parameters of the encoder based on the training results to obtain the feature extractor; The loss function includes mean absolute error and mean square error.

4. The method according to claim 3, characterized in that Before the step of building the CAE architecture, the method further includes the following steps: Inserting a convolutional layer after the second encoder layer and inserting the convolutional layer before the first layer of the decoder; And / or, configuring a rectified linear unit activation function and batch normalization in each layer in the encoder and the decoder.

5. The method according to claim 1, wherein The classification training of a preconfigured deep learning network based on the rock image marked with the background category or crack label comprises the following steps: Inputting the rock image into the deep learning network and outputting a classification result; Parameters of a deep learning network are adjusted based on the category result and the loss result of the background category or the crack label marked in the rock image.

6. The method according to claim 1, characterized in that The method of using the trained deep learning network to perform crack detection on the target rock image to be detected to obtain a detection result includes the following steps: Using the trained deep learning network to perform crack detection on the target rock image to be detected, and outputting classification category probabilities; the classification category probabilities include the probabilities of each of the background categories and the probabilities of the crack categories; Threshold processing is performed on the classification category probability to obtain a detection result of the target rock image.

7. A rock slope joint crack detection device, characterized in that: The device comprises: The first module is used to acquire a rock image and preprocess each channel of the rock image; The second module is configured to stack the channels of the preprocessed rock image and input the stacked channels into a preset feature extractor for feature extraction, thereby outputting a feature vector; the feature extractor includes an encoder, which is trained based on a cascaded decoder; The third module is configured to cluster the feature vectors of different rock images through self-organizing mapping to obtain a background category of each rock image; and pseudo-label the corresponding rock image based on the background category; A fourth module is configured to perform classification training on a preconfigured deep learning network based on the rock image marked with the background category or crack label; The device further comprises a model for performing the following steps: Determining a classification category based on the total number of the background categories; wherein the classification category is one more than the total number of the background categories, and the extra category represents a crack category; Configuring the deep learning network based on the classification categories in combination with a deep CNN architecture; The fifth module is used to use the trained deep learning network to perform crack detection on the target rock image to be detected to obtain the detection results.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images