Spatial domain and frequency domain combination-based crack image detection method
By combining the spatial and frequency domain features of the crack image detection method, the problems of insufficient preprocessing and topological map construction not suitable for complex cracks in the existing technology are solved, and crack detection with higher accuracy and adaptability is achieved.
Patent Information
- Application Number
- CN202511070838.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-09-12
AI Technical Summary
Existing technologies for pavement crack detection have problems such as preprocessing failing to adequately add gradient and edge information, topological map construction being unsuitable for cracks with complex orientations, and graph neural networks only considering a single frequency domain or spatial domain.
A crack image detection method based on the combination of spatial and frequency domains is adopted. By preprocessing the pavement crack image data, extracting time series features and constructing a topological map, node features are generated, and frequency and spatial domain features are extracted. Finally, feature fusion and image detection are performed.
It improves the accuracy and adaptability of crack detection, can better describe complex and changeable cracks, and enhances the model's performance in deep feature mining and generalization capabilities.
Smart Images
Figure CN120635722A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of pavement crack image detection, and in particular to a crack image detection method based on the combination of spatial domain and frequency domain. Background Art
[0002] In the field of pavement crack detection, traditional methods, such as machine learning, rely heavily on the algorithm designer's experience in classification models, resulting in cumbersome training processes and suboptimal efficiency and accuracy. Deep learning methods, on the other hand, can not only classify crack images but also locate and segment cracks. Consequently, a growing number of researchers are integrating deep learning with concrete crack detection. Existing technologies have explored preprocessing enhancement, topology map construction, and graph neural networks, but many challenges remain. First, preprocessing methods only consider raw image features, failing to incorporate gradient and edge information, which is more effective for crack detection in complex environments. Second, topology map construction methods based on graph neural networks typically consider a single topology map, making them unsuitable for cracks with complex orientations. Finally, when using graph neural networks, only single graph convolutions in the frequency or spatial domains have been considered. Summary of the Invention
[0003] In view of this, an object of the present invention is to provide a crack image detection method based on the combination of spatial domain and frequency domain to at least solve the above problems.
[0004] The technical solution adopted in the present invention is as follows: A crack image detection method based on spatial domain and frequency domain combination includes the following steps: Step 1: Collect pavement crack image data, divide it into first image data and second image data, and perform preprocessing; Step 2: Extract time series features from the first image data and the second image data, construct a topology graph, and generate node features; Step 3: Extract the node features in frequency domain and spatial domain, and fuse the frequency domain features and spatial domain features; Step 4: Perform image detection on the fused features and output the detection results.
[0005] Furthermore, the first image data in step 1 is time series data, representing multiple time series images of the crack location, including crack image data under cloudy, low fog, and sunny conditions; The second image data is single-frame data, representing a crack image at the crack position, and is a separately acquired image under standard lighting conditions.
[0006] Furthermore, the preprocessing in step 1 includes contrast enhancement and illumination normalization, and illumination normalization uses adaptive gamma correction:
[0007] in, represents the output image, represents the input image, represents the gamma correction parameter, Represents the image mean.
[0008] Furthermore, step 2 is specifically as follows: Extracting temporal enhancement features and crack confidence from the first image data using a recurrent neural network; The second image data is segmented into superpixels to generate superpixel regions. Each superpixel region is used as a graph node. The texture structure features are extracted through the SpixelFCN network, and the node features are generated by combining the temporal enhancement features. The weighted edge value is constructed in combination with the crack confidence.
[0009] Furthermore, the node features are defined as:
[0010] in, Indicates the node The mean color value of the region, Indicates the position coordinates of the node in the graph, Represents texture structural features, Indicates the timing enhancement feature.
[0011] Furthermore, step 3 is specifically as follows: The edge weights are converted into normalized adjacency matrices, and the node features are transformed into the frequency domain through adaptive graph Fourier transform combined with the normalized adjacency matrix to extract frequency domain features. The spatial features of the node features are extracted through a multi-scale convolutional network combined with the normalized adjacency matrix to generate spatial domain features. The frequency domain features and spatial domain features are fused to generate fused features.
[0012] Furthermore, the spatial features of node features are extracted through multi-scale convolutional networks as follows: The normalized adjacency matrix and node features are transformed linearly and activated nonlinearly to obtain the first-level features:
[0013] in, represents the single-hop neighborhood feature, represents the normalized adjacency matrix, Represents node features, represents the weight matrix; The normalized adjacency matrix is used to perform a double-hop adjacency matrix operation on the node features to obtain a double-hop adjacency matrix. The double-hop adjacency matrix and node features are transformed linearly and activated nonlinearly to obtain secondary features:
[0014] in, represents the double-hop neighborhood feature, represents the two-hop adjacency matrix, represents the weight matrix; The first-level features and the second-level features are combined and fused to obtain the spatial features:
[0015] in, Represents the spatial characteristics, represents the weight matrix.
[0016] Furthermore, the frequency domain features and spatial domain features are fused to generate fusion features, specifically including: Step 31: Combine frequency domain features and spatial domain features:
[0017] in, Represents the features after splicing, represents the frequency domain characteristics, Represents airspace characteristics; Learning concatenated features via Transformer encoder Global dependency of attention-enhanced features ; Step 32: Generate gate values by concatenating features and attention-enhanced features:
[0018] in, represents the gate value, represents the sigmoid activation function, represents the weight matrix, represents the bias term; Step 33: Use the gate value to perform weighted fusion of the concatenated features and the attention-enhanced features:
[0019] in, represents the fusion feature, represents the weight matrix.
[0020] Furthermore, step 4 is specifically as follows: inputting the fused features into the trained image detection and classification network model for image screening and classification; the image detection and classification network includes a graph-to-grid feature conversion layer, a convolution module, and a morphological classification head; the graph-to-grid feature conversion layer is used to convert the fused features into a grid feature map through Gaussian kernel mapping; the convolution module includes a 1×1 convolution and a 3×3 convolution, which are used to extract the features of the grid feature map and fuse them through channel attention; the morphological classification head is used to make the final classification decision on the fused features, and output the probability of each crack described in the image through the global average pooling and fully connected layer of the multi-scale classifier.
[0021] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a crack image detection method based on the combination of spatial and frequency domains. It takes into account the multi-scale characteristics of the image. By preprocessing the original RGB image with gradient and edge information, it fully explores the color, gradient, edge, position, texture, shape and other characteristics of the crack. In the topological map construction, it splices relatively robust gradient and edge information, which can more accurately describe the crack. The SpixelFCN algorithm is used to perform superpixel segmentation on the image and construct the topology map used in graph convolution, taking advantage of the convolutional network operation in the algorithm. This not only avoids the highly complex process of pixel composition and repeated image feature extraction, but also creates a unique topology map for each image, adapting to the complex and varied cracks in different images. Combining the features of frequency domain and spatial domain, global features are obtained through frequency domain to suppress high-frequency noise, enhance the overall structure of cracks and be robust to changes in crack direction. Local features are obtained through spatial domain to focus on crack edge details, capture crack directions and dynamically adjust the importance of neighbors, thereby improving the model's ability to mine deep features and generalize the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0023] Figure 1 It is a flowchart of the method provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0024] The principles and features of the present invention are described below with reference to the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.
[0025] Reference Figure 1The present invention provides a crack image detection method based on the combination of spatial domain and frequency domain, comprising the following steps: Step 1: Collect pavement crack image data, divide it into first image data and second image data, and perform preprocessing; The first image data is time series data, representing multiple time series images of the crack location, including crack image data under cloudy, low fog, and sunny conditions; the second image data is single frame data, representing the crack image of the crack location, which is a separately collected image under standard lighting conditions.
[0026] Preprocessing includes contrast enhancement and illumination normalization, and illumination normalization uses adaptive gamma correction:
[0027] in, represents the output image, represents the input image, represents the gamma correction parameter, Represents the image mean.
[0028] Step 2: Extract time series features from the first image data and the second image data and construct a topology graph to generate node features, specifically: The first image data is extracted using a recurrent neural network to extract temporal enhancement features and crack confidence. The second image data is segmented into superpixels to generate superpixel regions. Each superpixel region is used as a graph node. Texture structure features are extracted through the SpixelFCN network and combined with the temporal enhancement features to generate node features. The weighted edge value is constructed in combination with the crack confidence. The node feature is defined as:
[0029] in, Indicates the node The mean color value of the region, Indicates the position coordinates of the node in the graph, Represents texture structural features, Represents timing enhancement features Step 3: Extract the node features in the frequency domain and spatial domain, and fuse the frequency domain features and spatial domain features. Specifically: The edge weights are converted into a normalized adjacency matrix, and the node features are transformed into the frequency domain by combining the adaptive graph Fourier transform with the normalized adjacency matrix to extract the frequency domain features. The spatial features of the node features are extracted by combining the multi-scale convolutional network with the normalized adjacency matrix to generate spatial features, specifically: The normalized adjacency matrix and node features are transformed linearly and activated nonlinearly to obtain the first-level features:
[0030] in, represents the single-hop neighborhood feature, represents the normalized adjacency matrix, Represents node features, represents the weight matrix; The normalized adjacency matrix is used to perform a double-hop adjacency matrix operation on the node features to obtain a double-hop adjacency matrix. The double-hop adjacency matrix and node features are transformed linearly and activated nonlinearly to obtain secondary features:
[0031] in, represents the double-hop neighborhood feature, represents the two-hop adjacency matrix, represents the weight matrix; The first-level features and the second-level features are combined and fused to obtain the spatial features:
[0032] in, Represents the spatial characteristics, represents the weight matrix.
[0033] The frequency domain features and spatial domain features are fused to generate fusion features, including: Step 31: Combine frequency domain features and spatial domain features:
[0034] in, Represents the features after splicing, represents the frequency domain characteristics, Represents airspace characteristics; Learning concatenated features via Transformer encoder Global dependency of attention-enhanced features ; Step 32: Generate gate values by concatenating features and attention-enhanced features:
[0035] in, represents the gate value, represents the sigmoid activation function, represents the weight matrix, represents the bias term; Step 33: Use the gate value to perform weighted fusion of the concatenated features and the attention-enhanced features:
[0036] in, represents the fusion feature, represents the weight matrix.
[0037] Step 4: Perform image detection on the fused features and output the detection results, specifically: input the fused features into the trained image detection and classification network model for image screening and classification; the image detection and classification network includes a graph-to-grid feature conversion layer, a convolution module, and a morphological classification head; the graph-to-grid feature conversion layer is used to convert the fused features into a grid feature map through Gaussian kernel mapping; the convolution module includes a 1×1 convolution and a 3×3 convolution, which are used to extract the features of the grid feature map and fuse them through channel attention; the morphological classification head is used to make the final classification decision on the fused features, and output the probability of each crack described in the image through the global average pooling and fully connected layer of the multi-scale classifier.
[0038] For example, the formula for converting the fusion feature into a grid feature map through Gaussian kernel mapping is:
[0039] in, Indicates the number of fusion feature nodes, Indicates the location The value of the grid feature map at represents the standard deviation of the Gaussian kernel, Indicates the The location of the nodes; The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A crack image detection method based on the combination of spatial domain and frequency domain, characterized in that: The following steps are involved: Step 1: Collect pavement crack image data, divide it into first image data and second image data, and perform preprocessing; Step 2: Extracting time series features from the first image data and the second image data and constructing a topological graph to generate node features; Step 3: Extract the node features in frequency domain and spatial domain, and fuse the frequency domain features and spatial domain features; Step 4: Perform image detection on the fused features and output the detection results.
2. The crack image detection method based on spatial domain and frequency domain combination according to claim 1 is characterized in that: The first image data in step 1 is time series data, representing multiple time series images of the crack location, including crack image data under cloudy, low fog, and sunny conditions; The second image data is single-frame data, representing a crack image at the crack position, and is a separately acquired image under standard lighting conditions.
3. The crack image detection method based on spatial domain and frequency domain combination according to claim 2 is characterized in that: The preprocessing in step 1 includes contrast enhancement and illumination normalization, and illumination normalization uses adaptive gamma correction: in, represents the output image, represents the input image, represents the gamma correction parameter, Represents the image mean.
4. The crack image detection method based on spatial domain and frequency domain combination according to claim 3 is characterized in that: Step 2 is as follows: Extracting temporal enhancement features and crack confidence from the first image data using a recurrent neural network; Perform superpixel segmentation on the second image data to generate superpixel regions. Each superpixel region is used as a graph node. Texture structure features are extracted through the SpixelFCN network, and node features are generated by combining temporal enhancement features. The weighted boundary value is constructed in combination with the crack confidence.
5. The crack image detection method based on spatial domain and frequency domain combination according to claim 4 is characterized in that: The node characteristics are defined as: in, Indicates the node The mean color value of the region, Indicates the position coordinates of the node in the graph, Represents texture structural features, Indicates the timing enhancement feature.
6. The crack image detection method based on spatial domain and frequency domain combination according to claim 5 is characterized in that: Step 3 is as follows: The edge weights are converted into normalized adjacency matrices, and the node features are transformed into the frequency domain through adaptive graph Fourier transform combined with the normalized adjacency matrix to extract frequency domain features. The spatial features of the node features are extracted through a multi-scale convolutional network combined with the normalized adjacency matrix to generate spatial domain features. The frequency domain features and spatial domain features are fused to generate fused features.
7. The crack image detection method based on spatial domain and frequency domain combination according to claim 6 is characterized in that: The spatial features extracted from node features through multi-scale convolutional networks are specifically as follows: The normalized adjacency matrix and node features are transformed linearly and activated nonlinearly to obtain the first-level features: in, represents the single-hop neighborhood feature, represents the normalized adjacency matrix, Represents node features, represents the weight matrix; The normalized adjacency matrix is used to perform a double-hop adjacency matrix operation on the node features to obtain a double-hop adjacency matrix. The double-hop adjacency matrix and node features are transformed linearly and activated nonlinearly to obtain secondary features: in, represents the double-hop neighborhood feature, represents the two-hop adjacency matrix, represents the weight matrix; The first-level features and the second-level features are combined and fused to obtain the spatial features: in, Represents the spatial characteristics, represents the weight matrix.
8. The crack image detection method based on spatial domain and frequency domain combination according to claim 7 is characterized in that: The fusion of frequency domain features and spatial domain features to generate fusion features specifically includes: Step 31: Combine frequency domain features and spatial domain features: in, Represents the features after splicing, represents the frequency domain characteristics, Represents airspace characteristics; Learning concatenated features via Transformer encoder Global dependency of attention-enhanced features ; Step 32: Generate gate values by concatenating features and attention-enhanced features: in, represents the gate value, represents the sigmoid activation function, represents the weight matrix, represents the bias term; Step 33: Use the gate value to perform weighted fusion of the concatenated features and the attention-enhanced features: in, represents the fusion feature, represents the weight matrix.
9. The crack image detection method based on spatial domain and frequency domain combination according to claim 8, characterized in that: Step 4 is as follows: Input the fused features into the trained image detection and classification network model for image screening and classification; the image detection and classification network includes a graph-to-grid feature conversion layer, a convolution module, and a morphological classification head; the graph-to-grid feature conversion layer is used to convert the fused features into a grid feature map through Gaussian kernel mapping; the convolution module includes a 1×1 convolution and a 3×3 convolution to extract the features of the grid feature map and fuse them through channel attention; The morphological classification head is used to make the final classification decision on the fused features, and outputs the probability of each crack described in the image through the global average pooling and fully connected layer of the multi-scale classifier.