A crack identification method, system, device and storage medium

Abstract

The application discloses a crack identification method, system and device and a storage medium. The method comprises the following steps: acquiring a crack image to be identified; inputting the crack image into a multi-layer feature extraction layer to perform a feature extraction operation, and obtaining a first feature of multiple scales; combining small-scale features to facilitate detection of large-size cracks and large-scale features to facilitate representation of detailed information; performing compression operation on the first feature of each layer of the feature extraction layer through a convolution kernel to obtain a second feature of multiple scales; reducing data complexity through feature compression to avoid subsequent calculation redundancy; inputting the second feature of each scale into a curve structure restriction module to perform a constraint and limitation operation to obtain a third feature of multiple scales; highlighting the curve structure feature; performing up-sampling on the third feature of each scale based on a preset scale to obtain a fourth feature; and inputting the fourth feature into a classifier to perform a classification operation to obtain a predicted image. The application can accurately realize crack identification and can be widely applied to the technical field of image processing.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a crack recognition method, system, device, and storage medium. Background Technology

[0002] Cracks typically appear as curved structures with constantly changing angles or directions, and are one of the early signs of pavement performance degradation. The spatial characteristics of cracks are an important indicator for assessing buildings or infrastructure. They are generally inferred from the morphological and locational features of the cracks, as well as the degree of internal damage. Early detection of cracks allows for effective preventative measures, ensuring the safety and durability of the structure. Currently, manual inspection is the most common method for crack detection. However, this method is only suitable for small-scale inspections, and given the wide variety and number of cracks, the process of manually observing, describing, and marking cracks is time-consuming and relies on the inspector's experience and knowledge. The evaluation of the results also has a certain degree of subjectivity. Therefore, there is a need to develop objective and effective automated crack detection methods to improve this situation.

[0003] Automatic crack detection methods can be broadly categorized into two main types: traditional image processing methods and deep learning methods. The detection process typically involves image acquisition, preprocessing, image processing, and crack detection. Traditional image processing methods include using local and global brightness thresholds based on the characteristic of crack brightness, but this method is only applicable when the crack is singular and the background is simple. Another method involves meshing the crack image, designing specific filters to apply to each mesh, and locking onto meshes containing cracks. While effective, this method's recognition results are limited to the mesh level. Furthermore, penetration models have been applied to crack detection, achieving good results, but this method requires multiple traversals of global or local pixels and is time-consuming, making it unsuitable for real-time detection. To improve the adaptability of traditional image processing, machine learning methods have been applied to automatic crack detection technology. However, these methods are generally only used for image classification and are inevitably susceptible to human error in feature selection, affecting detection performance. Moreover, cracks in real-world engineering environments are unevenly distributed, and their backgrounds are complex, including uneven lighting, shadow interference, and environmental noise. This presents significant challenges to machine learning-based automatic crack detection. The existing methods for crack identification and detection have the aforementioned drawbacks. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a crack identification method, system, device, and storage medium that can efficiently and accurately identify cracks.

[0005] On one hand, embodiments of the present invention provide a crack identification method, including:

[0006] Obtain the image of the crack to be identified;

[0007] The crack image is input into a multi-layer feature extraction layer for feature extraction to obtain multi-scale first features. Each feature extraction layer outputs first features at different scales. The input data of each feature extraction layer in the multi-layer feature extraction layer includes the result of max pooling the first features output by the previous feature extraction layer.

[0008] The first features of each feature extraction layer are compressed using convolutional kernels to obtain multi-scale second features.

[0009] The second features at each scale are input into the curve structure constraint module for constraint operations to obtain the multi-scale third features; the constraint operations include partial derivative calculation, second-order partial derivative solving, check point calculation, and significance value calculation.

[0010] Based on a preset scale, the third feature at each scale is upsampled to obtain the fourth feature;

[0011] The fourth feature is input into the classifier for classification to obtain the predicted image.

[0012] Optionally, in the step of inputting the crack image into a multi-layer feature extraction layer for feature extraction to obtain multi-scale first features, each feature extraction layer performs feature extraction through at least two convolution operations.

[0013] Optionally, the first features of each feature extraction layer are compressed using convolutional kernels to obtain multi-scale second features, including:

[0014] The first features of each feature extraction layer are compressed using convolution kernels of a preset size to obtain multi-scale second features with a uniform number of channels.

[0015] Optionally, in the step of inputting the second features at each scale into the curve structure constraint module for constraint operation to obtain the multi-scale third features, the constraint operation performed by the curve structure constraint module includes:

[0016] By calculating the partial derivatives of the second feature, a set of partial derivatives is obtained; the set of partial derivatives is used to characterize the local straight line direction of each pixel of the two-dimensional curve of the image.

[0017] The second-order partial derivatives of the target partial derivatives in the partial derivative set are solved to obtain the eigenvectors; the eigenvectors are used to characterize the directions perpendicular to the line.

[0018] Verification point data is obtained by calculating verification points based on feature vectors.

[0019] When the verification point data is within the preset range, the target eigenvalue of the feature vector on the second directional derivative is calculated as the salient value to obtain the third feature.

[0020] Optionally, checkpoints are calculated based on feature vectors to obtain checkpoint data, including:

[0021] The verification parameters are calculated based on the eigenvectors and the set of partial derivatives.

[0022] Based on the feature vector, the check points are calculated in combination with the check parameters to obtain the check point data.

[0023] The expression for the verification point data is as follows:

[0024] (p x p y )=(t·n x ,t·n y )

[0025] In the formula, (p x p y ) represents the test point data, (n x n y ) represents the feature vector, and t represents the verification parameter;

[0026] The expression for the test parameter is:

[0027]

[0028] In the formula, CF x CF y CF xx CF xy and CF yy This represents the partial derivatives in the set of partial derivatives.

[0029] Optionally, based on a preset scale, the third feature at each scale is upsampled to obtain the fourth feature, including:

[0030] Based on the scale of the third feature corresponding to the first feature extraction layer in the multi-layer feature extraction layer as the preset scale, the third features at other scales are upsampled to obtain a fourth feature with multiple unified scales.

[0031] Optionally, the fourth feature is input into a classifier for classification to obtain a predicted image, including:

[0032] The fourth feature map is superimposed along the channel direction and input into the classifier, and the predicted image is obtained by at least two convolutional module fusion operations; wherein an activation function is set between the convolutional module fusion operations.

[0033] On the other hand, embodiments of the present invention provide a crack identification system, comprising:

[0034] The first module is used to acquire images of the cracks to be identified;

[0035] The second module is used to input the crack image into a multi-layer feature extraction layer for feature extraction to obtain multi-scale first features. Each feature extraction layer outputs first features at different scales. The input data of each feature extraction layer in the multi-layer feature extraction layer includes the result of max pooling the first features output by the previous feature extraction layer.

[0036] The third module is used to compress the first features of each feature extraction layer through convolution kernels to obtain multi-scale second features.

[0037] The fourth module is used to input the second features at each scale into the curve structure constraint module for constraint operations to obtain the multi-scale third features; the constraint operations include partial derivative calculation, second-order partial derivative solving, check point calculation, and saliency value calculation.

[0038] The fifth module is used to upsample the third feature at each scale based on a preset scale to obtain the fourth feature;

[0039] The sixth module is used to input the fourth feature into the classifier for classification to obtain the predicted image.

[0040] On the other hand, embodiments of the present invention provide a crack identification device, including an operator and a memory;

[0041] Memory is used to store programs;

[0042] The operator execution program implements the method described above.

[0043] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a program that is executed by an operator to implement the method described above.

[0044] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. An operator of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0045] This invention first acquires an image of the crack to be identified; then, the crack image is input into a multi-layer feature extraction layer for feature extraction to obtain multi-scale first features; each feature extraction layer outputs first features at different scales, and the input data for each feature extraction layer includes the result of max pooling the first features output by the previous feature extraction layer; this invention leverages the advantages of small-scale features for detecting large cracks and large-scale features for representing detailed information; the first features of each feature extraction layer are compressed using convolution kernels to obtain multi-scale second features; feature compression reduces data complexity and avoids redundant subsequent calculations; each scale of the second features is input into a curve structure constraint module for constraint operations to obtain multi-scale third features; the constraint operations include partial derivative calculation, second-order partial derivative solving, checkpoint calculation, and saliency value calculation; through various calculations in the curve structure constraint module, the curve structure features are fully highlighted; based on a preset scale, each scale of the third features is upsampled to obtain a fourth feature; the fourth feature is input into a classifier for classification to obtain a predicted image. This invention can efficiently and accurately identify cracks. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a crack identification method provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the overall architecture of a crack identification method provided in an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of the process architecture of the curve structure limiting module provided in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the classifier's process architecture provided in an embodiment of the present invention;

[0051] Figure 5 A schematic diagram illustrating the recognition results of different recognition methods provided in embodiments of the present invention on the CrackForest image dataset;

[0052] Figure 6 This diagram illustrates the recognition results of different recognition methods provided in embodiments of the present invention on the CrackInWorld image dataset. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] On the one hand, such as Figure 1 As shown, an embodiment of the present invention provides a crack identification method, including:

[0055] S100, Obtain the image of the crack to be identified;

[0056] Specifically, images of building facilities such as road surfaces, bridge decks, or walls that may have cracks need to be identified and detected; to facilitate subsequent processing, the crack images can be cropped to a uniform size.

[0057] S200. Input the crack image into a multi-layer feature extraction layer to perform feature extraction operations and obtain the first feature at multiple scales.

[0058] It should be noted that each feature extraction layer outputs a first feature at a different scale. The input data of each feature extraction layer in the multi-layer feature extraction layer includes the result of max pooling the first feature output by the previous feature extraction layer. In some embodiments, each feature extraction layer performs feature extraction through at least two convolution operations.

[0059] Specifically, in some specific embodiments, the multi-layer feature extraction layer of the present invention can be implemented based on a VGG16 convolutional neural network (including 13 convolutional layers), such as... Figure 2 As shown, the convolutional layer is re-divided into five feature extraction layers of different scales based on the location of the max pooling layer. Figure 2 The blue blocks represent feature blocks output from the first 13 convolutional layers of the VGG16 network baseline, while the feature extraction layers output feature F... i (i.e., the first characteristic) is:

[0060] F i =Cony i (F i-1 θ i ), i = 1, 2, 3, 4, 5

[0061] In the formula, F i Indicates the current feature; F i-1 Indicates the features of the previous layer; Conv i (*) indicates the current feature extraction layer; θ i The parameter representing the current feature; when i = 1, F i-1 This is the input image I.

[0062] The parameter configurations of the 13 convolutional layers in the multi-layer feature extraction layer of this invention and the positional relationships of each max pooling layer are shown in Table 1 below, where C, H and W represent the number of channels, height and width of the feature map, respectively.

[0063] Table 1

[0064] Module parameter Output feature scale information Convolutional layer 1 3x3 kernel, 64 channels C = 64, H, W Convolutional layer 2 3x3 kernel, 64 channels C = 64, H, W Max pooling layer 1 C = 64, H / 2, W / 2 Convolutional layer 3 3x3 kernel, 128 channels C = 128, H / 2, W / 2 Convolutional layer 4 3x3 kernel, 128 channels C = 128, H / 2, W / 2 Max pooling layer 2 C = 128, H / 4, W / 4 Convolutional layer 5 3x3 kernel, 256 channels C = 256, H / 4, W / 4 Convolutional layer 6 3x3 kernel, 256 channels C = 256, H / 4, W / 4 Convolutional layer 7 1x1 kernel, 256 channels C = 256, H / 4, W / 4 Max pooling layer 3 C = 256, H / 8, W / 8 8 convolutional layers 3x3 kernel, 512 channels C = 512, H / 8, W / 8 Convolutional layer 9 3x3 kernel, 512 channels C = 512, H / 8, W / 8 Convolutional layer 10 1x1 kernel, 512 channels C = 512, H / 8, W / 8 Max pooling layer 4 C = 512, H / 16, W / 16 Convolutional layer 11 3x3 kernel, 512 channels C = 512, H / 16, W / 16 Convolutional layer 12 3x3 kernel, 512 channels C = 512, H / 16, W / 16 Convolutional layer 13 1x1 kernel, 512 channels C = 512, H / 16, W / 16

[0065] S300. The first features of each feature extraction layer are compressed using convolutional kernels to obtain multi-scale second features:

[0066] It should be noted that in some embodiments, the method includes: compressing the first features of each feature extraction layer using convolution kernels of a preset size to obtain multi-scale second features with a uniform number of channels.

[0067] Specifically, in some specific embodiments, such as Figure 2 As shown, the feature block F output by each feature extraction layer i It is compressed into a compressed feature map CF with 1 channel by using a 1×1 convolution kernel (to achieve channel compression). i (i.e., the second feature), expressed as:

[0068]

[0069] In the formula, Compress(*) represents the compression operation. This represents the parameters used in the compression operation. Specifically, it includes the compression feature map CF of the shallow layers (the first two layers). i (i≤2) Contains more texture detail information, deep (last three layers) compressed feature maps CF i (i≥3) contains rich semantic information. Among them, small-scale features (deep compressed feature maps) have a large receptive field, which is convenient for detecting large cracks, while large-scale features (shallow compressed feature maps) have a small receptive field, which is convenient for detecting microcracks.

[0070] S400. Input the second features at each scale into the curve structure constraint module for constraint operation to obtain the multi-scale third features:

[0071] It should be noted that the constraint operation includes partial derivative calculation, second-order partial derivative solving, checkpoint calculation, and saliency value calculation. In some embodiments, the curve structure constraint module performs constraint operations, including: calculating the partial derivative of the second feature to obtain a set of partial derivatives; the partial derivative set is used to represent the local straight line direction of each pixel of the two-dimensional curve of the image; solving the second-order partial derivative of the target partial derivative in the set of partial derivatives to obtain a feature vector; the feature vector is used to represent the direction perpendicular to the straight line; calculating checkpoints based on the feature vector to obtain checkpoint data; when the checkpoint data is within a preset range, calculating the target feature value of the feature vector on the second-order directional derivative as the saliency value to obtain the third feature. By performing the above process steps to constrain the second feature at each scale, the corresponding third feature at each scale is obtained.

[0072] In some embodiments, verification point calculation based on feature vectors to obtain verification point data includes: calculating verification parameters based on feature vectors and partial derivative sets; and calculating verification points based on feature vectors and verification parameters to obtain verification point data. The expression for the verification point data is as follows:

[0073] (p x p y )=(t·n x ,t·n y )

[0074] In the formula, (p x p y ) represents the test point data, (n x n y ) represents the feature vector, and t represents the verification parameter;

[0075] The expression for the test parameter is:

[0076]

[0077] In the formula, CF x CF y CF xx CF xy and CF yy This represents the partial derivatives in the set of partial derivatives.

[0078] S500: Based on a preset scale, the third feature at each scale is upsampled to obtain the fourth feature;

[0079] It should be noted that in some embodiments, the method includes: using the scale of the third feature corresponding to the first feature extraction layer in the multi-layer feature extraction layers as a preset scale, upsampling the third features at other scales to obtain multiple fourth features with uniform scales. It is easy to understand that when the scale of the third feature corresponding to the first feature extraction layer is used as the preset scale, the third feature corresponding to the first feature extraction layer in the multi-layer feature extraction layers (i.e., the third feature obtained after compression and curve structure constraint by the first feature output from the first feature extraction layer) is directly used as the fourth feature; however, if other preset sizes are selected, all third features are upsampled using the corresponding preset size.

[0080] Specifically, in some embodiments, the compressed feature map CF is highlighted through the Curve Structure Restriction (CSR) module. i The local and global curve structure features are analyzed to suppress the expression of other structural features. This is achieved through the Curve Structure Restriction (CSR) module for each compressed feature map (CF). i Constraints are applied and upsampling is used to obtain a constrained feature map (RF) of the same size as the input image. i (i.e., the fourth characteristic), expressed as:

[0081] RF i =Up(CSR(CF) i ), I)

[0082] In the formula, CSR(*) represents the constraint operation of the curve structure constraint module, Up(*) represents the upsampling operation, and I represents the input image.

[0083] Among them, such as Figure 3 As shown, the Curve Structure Restriction (CSR) module is described as follows: A crack exists as a curved structure in an image. Assume there is a two-dimensional curve c(t) on the compressed feature map CF. The grayscale change of the profile image along the normal n(t) of this curve is an arched bridge line, with its first-order directional derivative disappearing and its second-order directional derivative having a significant absolute value. This is achieved by calculating the partial derivatives CF of the feature map. x CF y CF xx CF xy and CF yy The local straight line direction of each feature image pixel p(x, y) is determined (to calculate partial derivatives), and the direction of the maximum absolute value of the second-order directional derivative of the cross-section (CF) is taken as the direction of n(t). The value of this direction can be determined by calculating the eigenvalues ​​and eigenvectors of the Hessian matrix (by solving the second-order partial derivatives of the Hessian matrix). Let the absolute value of the maximum eigenvalue and the corresponding eigenvector (i.e., the direction perpendicular to the line) be (n x n y), and satisfy the 2-norm of 1, and use a quadratic polynomial to determine along (n x n y If the first derivative in the direction vanishes within the current pixel, then that point is defined (verification point calculation, corresponding to...). Figure 3 The calculation of t is as follows:

[0084] (p x p y )=(t·n x ,t·n y )

[0085] in:

[0086]

[0087] When (p) x p y ) satisfy in Within the range, it is restricted to points on the curved structure. At this point, (n x n y The largest eigenvalue on the second directional derivative is considered a significant point on the curve, thus obtaining the curve structure constraint feature map (i.e., the third feature), which is further obtained through upsampling. Figure 3 (Not shown) can then be used to obtain the fourth feature (corresponding to) Figure 3 (RF in the middle).

[0088] S600. Input the fourth feature into the classifier for classification to obtain the predicted image;

[0089] It should be noted that the classifier includes an activation function and at least two convolutional modules; in some embodiments, it includes: superimposing the fourth feature map along the channel direction and inputting it into the classifier, and obtaining the predicted image through at least two convolutional module fusion operations; wherein, an activation function is set between the convolutional module fusion operations.

[0090] Specifically, in some specific embodiments, by overlaying different restricted feature maps along the channel direction and inputting them into the classifier Cls, a predicted image P with a foreground of 1 and a background of 0 is finally obtained, represented as:

[0091] P = Cls(Concat(RF) 1 RF 2 RF 3 RF 4 RF 5 ), σ)

[0092] In the formula, Concat represents the feature channel merging operation; σ represents the classifier parameters. For example... Figure 4As shown, the classifier Cls consists of a 3×3 convolutional kernel with three output channels, a ReLU activation function, and a 1×1 convolutional kernel with one output channel. The restricted feature maps (RF) of different layers... i The images are superimposed along the channel direction and input into the classifier Cis. After being fused by two convolutional modules of different sizes, the predicted image P is finally obtained.

[0093] To further illustrate the technical effects of the embodiments of the present invention, explanations are provided in conjunction with comparative experimental data from some specific embodiments:

[0094] This invention's technical solution researches and develops a crack detection program based on a Python machine learning library. Key parameters are set as follows: a sample size of 2 per iteration; a preset learning rate of 0.05, adjusted via cosine annealing; an Adam optimizer; initialization parameters for the feature extraction layer using the parameters of the VGG16 pre-trained model, and initialization parameters for other layers using PyTorch default values; a period of 50; and a loss function combining cross-entropy loss and Dice loss. Model parameters are updated and saved when the loss on the current validation set is less than the historical minimum loss on the validation set.

[0095] Experiments on crack image recognition were conducted using the publicly available CrackForest and CrackInWorld image datasets. The CrackForest dataset contains 118 crack images of size 480×320 pixels and corresponding pixel-level annotations. The cracks in this dataset have diverse shapes, including single cracks, branching cracks, and network cracks, and are affected by complex textures such as paint and oil stains in the background. The CrackInWorld dataset contains 100 crack images collected by smartphones, and the cracks are embedded in different natural scenes, involving interference from factors such as cars, pedestrians, manhole covers, and trench covers. Specifically, the performance comparison of different crack recognition methods on the CrackForest dataset is shown in Table 2.

[0096] Table 2

[0097] Performance indicators D <![CDATA[P r ]]> <![CDATA[R c ]]> F T(s) Number of parameters (MB) Method of the present invention 0.6741 0.6523 0.7201 0.6741 0.5877 56 Unet 0.6034 0.6417 0.6128 0.6034 0.5259 132 DCNet 0.5848 0.6156 0.6274 0.5848 0.5125 62 FCNCNet 0.6738 0.6179 0.7697 0.6738 0.4973 94 FPHBNet 0.6538 0.6112 0.7459 0.6538 0.4948 60

[0098] Table 2 compares the performance metrics of the method of this invention with those of different crack identification methods on the CrackForest image dataset, including the Dice value (D) and accuracy (P). r Recall rate R c The measured value F, the recognition time T, and the number of network parameters are expressed as follows:

[0099]

[0100]

[0101] In the formula, T p This represents the number of pixels where both the actual and predicted values ​​are defects; F P This represents the number of pixels whose true value is the background but whose predicted value is a defect; F N The value represents the number of pixels where the true value is a defect but the predicted value is the background; D represents the overlap rate between the true and predicted values; P represents the number of pixels where the true value is a defect but the predicted value is the background. r R represents the ratio of the number of truly correct defect pixels to the total number of predicted defect values. c F represents the ratio of the number of truly correct defective pixels to the total number of true defective pixels; it is used to balance P. r and R c .

[0102] For the index D, P r R c For F, a larger value indicates a better recognition result from the network; for the metrics T and the number of parameters, a smaller value indicates less runtime and memory consumption by the network. The values ​​in Table 2 show that the corresponding methods achieve the best results on these metrics. From D, P r A comparison of performance indicators such as F and parameter quantity shows that the method of the present invention has significant advantages in crack identification. The designed network is relatively lightweight (with 56MB of parameters) and the detection time is 0.5877 seconds, which can meet the requirements of online detection.

[0103] like Figure 5 As shown, Figure 5 The recognition results of different recognition methods on the CrackForest image dataset are presented, among which... Figure 5 (a) shows the original sample images of the CrackForest image dataset. It can be seen from the image that the cracks in the sample images have various shapes and that the background of the cracks is affected by uneven lighting and paint. Figure 5 (b) presents the actual labeling results for different crack sample images. From Figure 5 As can be seen from (c1) and (d1), the Unet and DCNet methods cannot detect finer cracks. From... Figure 5 As can be seen from (e4), (e5), (f4), and (f5), the FCNCNet and FPHBNet methods misclassify paint chipping and surface weathering textures as cracks, resulting in a high number of false positives. From Figure 5 As can be seen from (e5), (f5), and (g5), the FCNCNet method, the FPHBNet method, and the method of this invention generally have poor performance in identifying low-contrast cracks and may result in missed detections.

[0104] like Figure 6 As shown, Figure 6The results of different recognition methods on the CrackInWorld image dataset are presented, and the original sample images contain relatively high levels of recognition. Figure 5 More complex background interference, such as paint, manhole covers, drain covers, human feet, shadows, and vehicles, etc. A direct visual comparison reveals that the Unet method and the method of this invention perform better than other methods, but they still... Figure 6 The shadows and manhole cover edges in (a3) ​​and (a4) are identified as cracks, mainly because the data used to train the network lacks images of the corresponding type. For other recognition methods, the DCNet method may miss detections, while the FCNCNet and FPHBNet methods produce more false positives, such as identifying the holes between drainage ditch covers as cracks.

[0105] In summary, this invention focuses on pavement cracks and designs a deep network based on curve structure constraints for real-time pixel-level recognition of pavement cracks. Five feature extraction layers at different scales of the VGG16 convolutional neural network baseline are used to extract crack image features. Large-scale features have smaller receptive fields, facilitating the detection of microcracks, while small-scale features have larger receptive fields, facilitating the detection of large cracks. To achieve good detection results with fewer training samples, a designed curve structure constraint module is embedded into the network, and a pre-trained parameter fine-tuning mechanism is used, enabling effective and rapid recognition of concrete pavement cracks. Furthermore, this invention utilizes the VGG16 convolutional neural network baseline to extract effective features from crack images in complex backgrounds. Embedding the designed curve structure constraint module into the network and using prior knowledge to constrain the output multi-scale features reduces the network size while maintaining good detection performance. The experimental results show that this patented method can identify crack regions in unevenly lit and complex textured backgrounds, exhibiting good robustness and meeting online detection requirements.

[0106] On the other hand, embodiments of the present invention provide a crack recognition system, comprising: a first module for acquiring a crack image to be identified; a second module for inputting the crack image into a multi-layer feature extraction layer for feature extraction to obtain multi-scale first features; wherein each feature extraction layer outputs first features at different scales, and the input data of each feature extraction layer in the multi-layer feature extraction layer includes the result of max pooling the first features output by the previous feature extraction layer; a third module for compressing the first features of each feature extraction layer through convolution kernels to obtain multi-scale second features; a fourth module for inputting the second features at each scale into a curve structure constraint module for constraint operation to obtain multi-scale third features; wherein the constraint operation includes partial derivative calculation, second-order partial derivative solving, checkpoint calculation, and saliency value calculation; a fifth module for upsampling the third features at each scale based on a preset scale to obtain fourth features; and a sixth module for inputting the fourth features into a classifier for classification to obtain a predicted image.

[0107] The content of the method embodiments of the present invention is applicable to the system embodiments. The specific functions implemented in the system embodiments are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0108] Another aspect of the present invention provides a crack identification device, including an operator and a memory;

[0109] Memory is used to store programs;

[0110] The operator execution program implements the method described above.

[0111] The content of the method embodiments of the present invention is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0112] Another aspect of this invention provides a computer-readable storage medium storing a program that is executed by an operator to implement the method described above.

[0113] The content of the method embodiments of the present invention is applicable to the computer-readable storage medium embodiments. The specific functions implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0114] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. An operator of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0115] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0116] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0117] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution means, apparatus, or device (such as a computer-based device, an apparatus including an operator, or other means that can fetch and execute instructions from, or in conjunction with, an instruction execution means, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution means, apparatus, or device.

[0119] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise manipulating it as necessary, and then stored in computer memory.

[0120] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0121] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0122] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0123] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A crack identification method, characterized in that, include: Obtain the image of the crack to be identified; The crack image is input into a multi-layer feature extraction layer for feature extraction to obtain multi-scale first features; wherein, each feature extraction layer outputs first features at different scales, and the input data of each feature extraction layer in the multi-layer feature extraction layer includes the result of max pooling operation on the first features output by the previous feature extraction layer; The first features of each feature extraction layer are compressed using convolution kernels to obtain multi-scale second features. The second feature at each scale is input into the curve structure constraint module for constraint operation to obtain the multi-scale third feature; wherein, the constraint operation includes partial derivative calculation, second-order partial derivative solution, check point calculation and saliency value calculation; In the step of inputting the second features at each scale into the curve structure constraint module for constraint operation to obtain the multi-scale third features, the constraint operation of the curve structure constraint module includes: By calculating the partial derivatives of the second feature, a set of partial derivatives is obtained; the set of partial derivatives is used to characterize the local straight line direction of each pixel of the two-dimensional curve of the image; The second-order partial derivatives of the target partial derivatives in the set of partial derivatives are solved to obtain the eigenvectors; the eigenvectors are used to characterize the direction perpendicular to the line. Based on the feature vector, checkpoints are calculated to obtain checkpoint data; When the verification point data is within a preset range, the target feature value of the feature vector on the second-order directional derivative is calculated as a significant value to obtain the third feature; The step of calculating checkpoints based on the feature vector to obtain checkpoint data includes: The verification parameters are calculated based on the eigenvectors and the set of partial derivatives. Based on the feature vector and the verification parameters, the verification points are calculated to obtain the verification point data. The expression for the verification point data is as follows: In the formula, Indicates the test point data. Represents the eigenvector. Indicates the verification parameters; The expression for the test parameter is: In the formula, , , , and Represents each partial derivative in the set of partial derivatives; Based on a preset scale, the third feature at each scale is upsampled to obtain the fourth feature; The fourth feature is input into the classifier for classification to obtain the predicted image.

2. The crack identification method according to claim 1, characterized in that, In the step of inputting the crack image into a multi-layer feature extraction layer for feature extraction to obtain multi-scale first features, each feature extraction layer performs the feature extraction operation through at least two convolution operations.

3. The crack identification method according to claim 1, characterized in that, The step of compressing the first features of each feature extraction layer through a convolutional kernel to obtain multi-scale second features includes: The first features of each feature extraction layer are compressed using convolution kernels of a preset size to obtain multi-scale second features with a uniform number of channels.

4. The crack identification method according to claim 1, characterized in that, The fourth feature is obtained by upsampling the third feature at each preset scale to obtain the fourth feature, including: Based on the scale of the third feature corresponding to the first feature extraction layer in the multi-layer feature extraction layers as a preset scale, the third features at other scales are upsampled to obtain multiple fourth features with unified scales.

5. The crack identification method according to claim 1, characterized in that, The step of inputting the fourth feature into the classifier for classification to obtain the predicted image includes: The fourth feature map is superimposed along the channel direction and input into the classifier, and the predicted image is obtained by at least two convolutional module fusion operations; wherein, an activation function is set between the convolutional module fusion operations.

6. A crack identification system, characterized in that, include: The first module is used to acquire images of the cracks to be identified; The second module is used to input the crack image into a multi-layer feature extraction layer for feature extraction to obtain multi-scale first features; wherein, each feature extraction layer outputs first features at different scales, and the input data of each feature extraction layer in the multi-layer feature extraction layer includes the result of max pooling operation on the first features output by the previous feature extraction layer; The third module is used to compress the first features of each feature extraction layer through a convolution kernel to obtain multi-scale second features. The fourth module is used to input the second feature at each scale into the curve structure constraint module for constraint operation to obtain the multi-scale third feature; wherein, the constraint operation includes partial derivative calculation, second-order partial derivative solution, check point calculation and saliency value calculation; In the step of inputting the second features at each scale into the curve structure constraint module for constraint operation to obtain the multi-scale third features, the constraint operation of the curve structure constraint module includes: By calculating the partial derivatives of the second feature, a set of partial derivatives is obtained; the set of partial derivatives is used to characterize the local straight line direction of each pixel of the two-dimensional curve of the image; The second-order partial derivatives of the target partial derivatives in the set of partial derivatives are solved to obtain the eigenvectors; the eigenvectors are used to characterize the direction perpendicular to the line. Based on the feature vector, checkpoints are calculated to obtain checkpoint data; When the verification point data is within a preset range, the target feature value of the feature vector on the second-order directional derivative is calculated as a significant value to obtain the third feature; The step of calculating checkpoints based on the feature vector to obtain checkpoint data includes: The verification parameters are calculated based on the eigenvectors and the set of partial derivatives. Based on the feature vector and the verification parameters, the verification points are calculated to obtain the verification point data. The expression for the verification point data is as follows: In the formula, Indicates the test point data. Represents the eigenvector. Indicates the verification parameters; The expression for the test parameter is: In the formula, , , , and Represents each partial derivative in the set of partial derivatives; The fifth module is used to upsample the third feature at each preset scale to obtain the fourth feature; The sixth module is used to input the fourth feature into the classifier for classification to obtain a predicted image.

7. A crack detection device, comprising an operator and a memory; The memory is used to store programs; The operator executes the program to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by an operator to implement the method as described in any one of claims 1 to 5.

Images