Dynamic crack detection and pixel-by-pixel object division method based on parameter redefinition

Abstract

The invention discloses a dynamic crack detection and pixel-by-pixel object division method based on parameter redefinition, and the method designs a parameter redefinition Transform architecture for feature extraction, can capture the context information of cracks in different scenes, and reduces the calculation amount; according to the method, rapid multi-scale attention is further added to enhance feature extraction of cracks of different sizes, a parameter redefinition decoder is introduced to further reduce the calculation amount of the model, and by means of the method, necessary information can be provided for accurate crack detection and pixel-by-pixel object division, intelligent traffic enhancement and urban development, and the method is suitable for large-scale popularization and application. Through the information, traffic facilities can be better monitored and maintained, and the safety of roads and bridges is ensured, so that a solid guarantee is provided for the progress of intelligent traffic and the sustainable development of cities.

Description

Technical Field

[0001] The present invention relates to the field of dynamic crack detection and pixel-by-pixel object division, and in particular to a method for dynamic crack detection and pixel-by-pixel object division based on parameter redefinition. Background Art

[0002] Civil infrastructure and transportation facilities based on materials such as asphalt, concrete and rock are vulnerable to corrosion, carbonization and human factors; surface cracks are often considered to be a sign of internal structural damage in these infrastructures. If surface cracks are not detected in a timely and accurate manner, the consequences can be catastrophic; in addition, if Figure 1 As shown, a is a fine-grained crack, b is a background noise image, and c is an irregular crack. From top to bottom are the original image, the result of lightweight YOLO-seg, and the result of the pixel-by-pixel object segmentation network with dynamic parameters to redefine the crack. The wide range of crack types further complicates the crack detection task, especially fine-grained cracks have a significant impact on the accuracy of the algorithm; therefore, timely and accurate detection of these cracks is crucial because it can prevent more serious disasters; manual detection of surface cracks is time-consuming and laborious, and the use of intelligent algorithms and optical sensors has become the mainstream method.

[0003] Existing crack detection methods include:

[0004] 1. Crack detection based on traditional image processing: Since computer vision was introduced into crack detection, research in this field has made certain progress. Early crack detection work was based on image morphological algorithms and time domain and frequency domain transformations.

[0005] 2. Crack detection based on deep learning: In recent years, deep learning has been widely used in the field of crack detection due to its excellent feature extraction capabilities. Early deep learning-based crack detection work used DNN and FCN architectures; such as Faster R-CNN, SSD, GoogLeNet, etc. These methods can detect crack bounding boxes or perform pixel-by-pixel object segmentation to a certain extent.

[0006] 3. Pixel-by-pixel object segmentation algorithm: The emergence of this algorithm provides a promising solution to the challenge of crack detection, and the continuous improvement of object detection algorithms has also laid a solid foundation for crack detection; crack detection and crack pixel-by-pixel object segmentation can be performed simultaneously in the future; Mask R-CNN and FCIS are built on the basis of two-stage target detectors FasterR-CNN and R-FCN; the crack pixel-by-pixel object segmentation performed by these two algorithms has normal detection accuracy. With the introduction of single-stage target detectors represented by YOLO, the pixel-by-pixel object segmentation algorithm has also entered a new era; in terms of segmentation accuracy and inference efficiency, SOTA instance segmentation methods such as YOLOv8-Seg and YOLOACT have made great progress; based on the YOLOACT proposed by Piyathilaka et al. as a specific crack detection and pixel-by-pixel object segmentation algorithm, crack detection and pixel-by-pixel object segmentation with real-time frame rate are achieved.

[0007] The existing crack detection methods specifically include the following defects:

[0008] 1. Early crack detection work suffers from poor robustness of morphological algorithms due to the complex variations in crack shape and size, making them more prone to errors or missed detections. In addition, the performance of morphological algorithms may degrade when processing images with complex backgrounds and uneven illumination, which poses a challenge to accurately identifying cracks. Time and frequency domain transforms are very sensitive to noise in images and are prone to misdetection of noisy images. Images may have broken edges or incomplete connections, which can affect accurate crack detection. Although these methods have been used to extract crack information, crack detection has not fully benefited from the above work due to technical limitations.

[0009] 2. Faster R-CNN, SSD, and GoogLeNet all have a common disadvantage: the algorithm requires too much computation, and too much redundant calculation causes the model to lose real-time performance.

[0010] 3. The inference efficiency of the two algorithms, Mask R-CNN and FCIS, is still far from real-time. The frame rate of the YOLOACT algorithm on the Tesla P100 is only 30FPS Nvidia Titan XP GPU. Large-scale deployment still requires hardware support with high computing power, which increases the deployment cost. Summary of the invention

[0011] The purpose of the present invention is: to address the above-mentioned problems, the present invention provides a dynamic crack detection and pixel-by-pixel object segmentation method based on parameter redefinition. The method of the present invention designs a parameter redefinition Transformer architecture for feature extraction, which can capture the contextual information of cracks in different scenarios and reduce the amount of calculation; the present invention also adds fast multi-scale attention to enhance the feature extraction of cracks of different sizes, and introduces a parameter redefinition decoder to further reduce the amount of calculation of the model.

[0012] The technical solution adopted by the present invention is as follows:

[0013] Dynamic crack detection and pixel-by-pixel object segmentation methods based on parameter redefinition, such as Figure 2 As shown, including:

[0014] Establish a dataset of target crack images;

[0015] Re-defining the Transformer architecture for the input parameters of the data set to extract context information and obtain semantic information of cracks in the data set;

[0016] The parameter redefinition Transformer architecture includes a RepConv module and a TC2F module. The RepConv module and the TC2F module are used as a double-layer training module to obtain high-level information of the target crack image in the data set, thereby realizing the positioning of the target crack and pixel-by-pixel object division.

[0017] Furthermore, the target crack image dataset includes but is not limited to building cracks, rural roads, and cracks LS315 public dataset.

[0018] Furthermore, the RepConv module and the TC2F module include:

[0019] The RepConv module combines 3×3 convolution, 1×1 convolution, and identity connection in a single convolutional layer to provide diversified gradients for mapping the input feature information of the dataset.

[0020] The TC2F module specifically introduces a multi-layer TransformerBlock in the C2F architecture, and the TransformerBlock converts the target crack image feature information of the input data set into Reshape into a flat 2D patch Where (H, W) is the resolution of the original image, C is the number of channels, and (P, P) is the resolution of each target crack image patch. After the flattened feature information X passes through the TransformerLayer architecture and multiple linear layers, a feature-rich target feature information X is obtained. 1 .

[0021] Furthermore, the TransformerLayer architecture divides the input feature information X into three components, query Q, key K and value V, and then passes through multiple linear layers in sequence; Q, K, V in the linear layer are aggregated in the multi-head attention MHA, and then summed with the divided original components Q, K, V, and finally pass through two linear layers to obtain the output feature information.

[0022] Furthermore, the output header in multi-head attention MHA is as follows:

[0023]

[0024] In the formula, i is the number of heads, head i is the output of the head, the matrix They represent the query transformation matrix, key transformation matrix, and value transformation matrix of the head respectively, and Attention represents the attention calculation function;

[0025] The Attention represents the attention calculation function as follows:

[0026]

[0027] Where, d k Represents the dimension of K vectors, and Softmax normalizes the weight of each K vector based on similarity.

[0028] Furthermore, the method also includes a fast multi-scale attention architecture, which captures target features of the input target crack image through two parallel convolutions, and the two parallel convolutions include a 1×1 convolution and a 3×3 convolution.

[0029] Furthermore, the method also includes a parameter redefinition decoder for reducing the amount of calculation of the method, including:

[0030] Combine the convolutional layer with the batch normalization BN layer. The BN formula is as follows:

[0031]

[0032] Convert equation (3) to Y = w BN +b BN in the form of Therefore, the combination of convolutional layer and BN layer is:

[0033]

[0034] The combined convolution parameters are as follows:

[0035]

[0036] The form is The BN layer calculation process is as follows:

[0037]

[0038] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0039] The parameter-redefinition-based dynamic crack detection and pixel-by-pixel object division method of the present invention proposes a new dynamic crack detection and pixel-by-pixel object division pipeline-dynamic parameter redefinition crack pixel-by-pixel object division network; benchmark test results show that the dynamic parameter redefinition crack pixel-by-pixel object division network extraction is better than other SOTA methods, especially when the contrast between the crack and the background in the image is low, the method of the present invention achieves great improvement; therefore, the dynamic parameter redefinition crack pixel-by-pixel object division network proposed by the present invention can be applied to crack detection challenges in various complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a display diagram of crack type detection in the dynamic crack detection and pixel-by-pixel object segmentation method based on parameter redefinition of the present invention;

[0041] Figure 2 It is a schematic diagram of the network structure of the dynamic crack detection and pixel-by-pixel object division method based on parameter redefinition of the present invention;

[0042] Figure 3 It is a structural schematic diagram of a parameter redefinition Transformer in a dynamic crack detection and pixel-by-pixel object segmentation method based on parameter redefinition of the present invention;

[0043] Figure 4 It is a structural schematic diagram of TC2F in the dynamic crack detection and pixel-by-pixel object segmentation method based on parameter redefinition of the present invention;

[0044] Figure 5 It is a heat map generated by different trunks on the building crack in the dynamic crack detection and pixel-by-pixel object segmentation method based on parameter redefinition of the present invention;

[0045] Figure 6 It is a structural schematic diagram of fast multi-scale attention in the dynamic crack detection and pixel-by-pixel object segmentation method based on parameter redefinition of the present invention;

[0046] Figure 7 This is a result diagram of a benchmark test performed on a crack LS315 during a simulation experiment of the dynamic crack detection and pixel-by-pixel object segmentation method based on parameter redefinition of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] This embodiment provides a method for dynamic crack detection and pixel-by-pixel object segmentation based on parameter redefinition, which specifically includes:

[0050] Dataset creation

[0051] This embodiment selects three data sets to evaluate the detection ability of the parameter redefinition Transformer proposed in this embodiment for cracks. The three data sets include building cracks, rural roads, and cracks LS315. These data sets are strictly labeled to provide a high-quality data basis for subsequent feature extraction and model training. Building cracks: This data set is taken by a drone 30P camera at a certain location. The drone is fully charged at a constant speed of 21.6 kilometers per hour. The camera takes 30 photos per second. In order to ensure the safety of drone flight and the accuracy of detection, the drone is equipped with a small edge computing card JestonNano. This edge computing card has only 128 Cuda cores. This data set focuses on cracks on objects under different angles, weather and construction conditions. It contains 1,487 images, depicting common crack types on the exterior of urban buildings, including straight cracks, curved cracks and branch cracks. Rural roads: This data set was captured in rural areas using a camera. It contains 9,500 images of rural concrete pavements in different environments, showing various complex cracks, including mesh, extension, branching and crossing. Crack LS315,This public dataset contains 315 asphalt pavement images taken by a line array camera,,containing a large number of fine-grained cracks.

[0052] The parameters redefine the Transformer architecture to extract the contextual information of the cracks in the crack image, including:

[0053] like Figure 3 As shown in the figure, the parameter redefinition Transformer architecture includes two parts: RepConv and TC2F. The RepConv module first provides the feature information of the input crack image to provide multiple gradient mapping channels, and then the TC2F module is used to transform the input feature information to obtain richer feature information.

[0054] Among them, RepConv combines 3×3 convolution, 1×1 convolution and identity connection in a single convolution layer; since the identity connection in RepConv destroys the residual in ResNet and the crosstalk in DenseNet, it provides more diverse gradients for different feature maps; this embodiment develops two training strategies, namely using RepConv and TC2F to process a large number of input feature information mapping channels; therefore, the parameter redefinition Transformer architecture in this embodiment obtains high-level information in the training step, and can be simplified to 3×3 convolution in the reasoning step, thereby improving the reasoning efficiency.

[0055] Among them, the structure of TC2F from temporal convolution to fully connected layer is as follows Figure 4 As shown, compared with the original step-by-step refinement method C2F architecture, this embodiment introduces a multi-layer TransformerBlock; the standard TransformerBlock receives a 1D label embedding sequence as input; in order to process 2D images, TransformerBlock converts the input image Reshaped into a series of flat 2D patches Where H×W is the resolution of the original image, C is the number of channels, and P 2 is the resolution of each image patch; after the flattened feature information X passes through multiple linear layers and the TransformerLayer architecture, a feature-rich X 1 ;

[0056] Among them, the TransformerLayer architecture divides the input feature X into three components: query Q, key K and value V, and then passes through the linear layer in sequence; Q, K, V passing through the linear layer are aggregated in the multi-head attention MHA, and then summed with the original components Q, K, V, and finally pass through two linear layers to get the output; the linear layer is an important part of the transformer structure, which is actually a linear transformation of the input feature information X, as described in the following formula (7), where Y n×o is the output matrix, with a size of n×o, representing n samples, each with o output neurons; W is the parameter that the model needs to learn, and the dimension of W is W i×o ; b is the 0-dimensional vector bias, n is the number of rows in the input vector, i is the number of input neurons, and o is the number of output neurons;

[0057] Y n×o =X n×i W i×o +b (7)

[0058] Among them, MHA extends the attention mechanism to multiple heads, improving the model's ability to focus on various features; the calculation of the output header of each head is shown as follows:

[0059]

[0060] In the formula, the matrix They represent the query transformation matrix, key transformation matrix and value transformation matrix of the head respectively, and Attention represents the attention calculation function. In MHA, the self-attention mechanism is usually used to calculate the attention. The calculation formula of the self-attention mechanism is as follows:

[0061]

[0062] Where, d k represents the dimension of K vector. Softmax normalizes the weight of each K vector based on similarity. Next, the normalized weight is multiplied by the corresponding V vector. Finally, the weighted sum is performed to generate the attention output.

[0063] The principle of MHA is to perform a weighted sum of the key vector based on the similarity between the above Q vector and K vector, and then multiply the resulting weighted sum by the V vector to generate the output; commonly used similarity calculation methods include dot product or bilinear calculation; the multi-head mechanism of MHA enhances the expressive power of the model, enabling it to learn diverse and complex features; the input sequence data is divided into multiple heads, each head performs independent calculations, produces different outputs, and then combines them into the final output.

[0064] The parameter redefinition Transformer architecture of this embodiment can accurately and quickly capture the semantic information of cracks, thereby achieving efficient crack positioning and pixel-by-pixel object segmentation in various scenarios. This embodiment selects three characteristic crack images for feature information graph visualization, such as Figure 5 As shown in the figure, from top to bottom are the original image, the heat map generated based on the original backbone, and the heat map generated based on the parameter-redefined Transformer.

[0065] This example introduces fast multi-scale attention to capture the long-term dependencies of cracks, including:

[0066] The fast multi-scale attention architecture uses two parallel convolutions to capture features, one branch is a 1×1 convolution and the second branch is a 3×3 convolution; bidirectional convolution parallel calculation can capture multi-scale feature information, greatly improving the calculation speed; in addition, the one-dimensional global average pooling in the fast multi-scale attention is used to encode global information along the horizontal dimension, the process is as follows:

[0067]

[0068] In the formula, Represents the output value of feature z at channel c and position H, which is a value after aggregation or averaging operation. is an averaging operation, where represents the coefficient of the average, that is, dividing the sum by W to calculate the average, Represents the input feature x c Sum all i values ​​at channel c and position H, where i ranges from 0 to W, indicating a summation operation along the width dimension, x c (H,i) represents the input feature value at channel c and position H and i, and W represents the length in the width direction (or window size) which is used to determine the range of the averaging operation.

[0069] By computing convolutional layers of different sizes in parallel, Fast Multi-Scale Attention can accurately capture long-range dependencies in the horizontal direction and preserve precise position information in the vertical direction. The structure of Fast Multi-Scale Attention is shown in Figure 2. Figure 6 shown.

[0070] This embodiment also introduces parameters to redefine the decoder to reduce the amount of model calculation, including:

[0071] Parameter redefinition involves the fusion of parameters of certain structures, which improves the inference efficiency without sacrificing accuracy. It is proposed to combine the convolution layer with the batch normalization BN layer. The formula of the BN layer is as follows:

[0072]

[0073] In the formula, is the normalized output value, x i is the original input value, μ is the mean of the batch data, usually calculated for a batch of data, σ 2 is the variance of the batch data, which is also calculated on a batch of data. γ is a learnable scaling parameter that adjusts the scale of the output value during training. β is a learnable translation parameter that adjusts the translation of the output value during training. ε is a very small constant that is used to prevent division by zero and ensure numerical stability.

[0074] The above formula can be regarded as Y = w BN +b BN in the form of Then the combination of convolutional layer and BN layer is:

[0075]

[0076] In the formula, is the output value after processing by the convolution layer and batch normalization layer, w BN is the scaling parameter of the batch normalization layer, which is learned through training in the batch normalization layer and is used to adjust the scale of the normalized output value. BNis the shift parameter of the batch normalization layer, which is also learned through training and is used to adjust the offset of the normalized output value, w conv is the weight parameter of the convolutional layer, which is equivalent to the filter of the convolutional layer and is used to extract features from the input data. conv It is the bias parameter of the convolution layer, which is used to add a fixed offset after the convolution operation. x is the input data, which may be an image or a feature map.

[0077] The combined convolution parameters are as follows:

[0078]

[0079] Where w is the effective weight after fusion, which is the scaling parameter w of batch normalization BN With the original convolution weight b conv The product of, b is the effective bias after fusion, which is the original convolution bias b conv Multiply by w BN After that, add the batch normalization bias b BN obtained;

[0080] The form is The BN layer can be regarded as a special 1×1 convolution, and the process is as follows:

[0081]

[0082] Formula (6) is similar to the structure of a matrix multiplication, which represents the calculation process of the fully connected layer in the neural network. Specifically, each part of the formula represents the following:

[0083] The right side is the input vector. This column vector represents the number of input channels, denoted as ch in Each element represents the value of the input data on different channels; for example, if this is the output of an image after a convolutional layer, then each value can be understood as the feature value on each input channel;

[0084] The middle part is the weight matrix, the shape of this matrix is ​​ch out ×ch in , where ch out Indicates the number of output channels; each row of the weight matrix corresponds to the weight of an output channel, and each column corresponds to the weight of an input channel. When performing matrix multiplication, the weight matrix performs a weighted summation on each channel in the input vector;

[0085] The left part is the output vector, which is a column vector indicating the number of output channels, denoted as ch out ; Each element is the product of the input vector and the weight matrix, that is, the output after the linear transformation in the neural network;

[0086] A 1×1 convolution multiplies a value by all pixels in a channel. The processing within the channel is the same, so the channel wh is simplified to a □, which is a layer of the image.

[0087] For example, if the input channels are 4 and the output channels are 3, the 12 boxes in the weights can be seen as multiple 1×1 convolution slices, and the 4-layer image is divided into 3 layers of images, corresponding to 12; BN normalizes on each channel, and each channel is independent of each other, so the weight is a diagonal array. The BN layer can be seen as a 1×1 convolution with the same number of input and output channels, and they are one-to-one corresponding. Among them, since the convolution weight is approximately ch out ×ch in ×ks×ks structure, and the BN layer has one weight for each input channel, so the length is ch in One-dimensional vector of ; ch of BN layer in Equal to the convolution ch out ; In order to realize the matrix multiplication of the two, w conv Convert to ch out ×ch in ×ks×ks, w BN Convert to ch out ×ch out , the process is as follows:

[0088]

[0089] Finally, the model of this embodiment is trained and evaluated, including:

[0090] Combining the parameter redefinition Transformer architecture, the fast multi-scale attention architecture, and the parameter redefinition decoder, the method of this embodiment constructs a complete dynamic crack detection and pixel-by-pixel object segmentation pipeline; in order to improve the pipeline's ability to detect different cracks in different scenarios, a heuristic training strategy is adopted, such as image enhancement, hyperparameter tuning, etc.; the four benchmarks are divided into training sets, validation sets, and test sets in a ratio of 8:1:1. After a large number of experiments, the hyperparameters for training the dynamic parameter redefinition crack pixel-by-pixel object segmentation network are finally determined; for the dynamic parameter redefinition crack pixel-by-pixel object segmentation network, batchsize=8,epcoh=200,optimizer=SGD,iou=0.7,learning rate=0.01,weight decay=0.0005,image size=640×640.

[0091] The simulation experiment of the present invention is as follows:

[0092] The simulation conditions are shown in Table 1:

[0093] Table 1 Partial environment configuration of the code of the present invention

[0094]

[0095]

[0096] The experimental comparison example is as follows:

[0097] As shown in Table 2, the parameter redefinition Transformer method proposed in this embodiment surpasses other SOTA methods with fewer parameter values ​​and achieves the SOTA performance in crack identification and pixel-by-pixel object segmentation (97.1% mAP50 Box ,94.5%mAP50 Mask ); Benchmark test results show that in the crack detection and pixel-by-pixel object segmentation challenges, utilizing common lightweight convolutions (FasterNet) and simplifying the output (RepCol) cannot improve the crack detection accuracy while being lightweight; One way to improve crack detection accuracy is to increase the model computation; however, this will inevitably lead to the model losing its real-time detection capability.

[0098] Table 2 Performance of parameter redefinition Transformer method compared with other methods on the building crack validation set

[0099]

[0100] In order to further improve the pipeline's ability to extract different types of crack features, this embodiment adopts multiple attention mechanisms; the benchmark test results are shown in Table 3. Fast multi-scale attention produces better pixel-level attention on high-level feature maps; therefore, in the crack detection and pixel-by-pixel object segmentation challenges, fast multi-scale attention can achieve better pixel-level attention with less computation (96.6% mAP50 Box ,93.8%mAP50 Mask ) outperforms other SOTA attention methods.

[0101] Table 3 Performance of the fast multi-scale attention method compared with other methods on the building crack validation set

[0102]

[0103] In the parameter redefinition decoder experiment, not only the VOV architecture using fused convolution is compared, but also two new FPN paradigms: Bifpn and Gfpn. The parameter redefinition decoder architecture we proposed combines convolutional layers and BN layers in the upsampling process; the experimental results are shown in Table 4; this method not only reduces the computational complexity of upsampling, but also retains all features without any loss; therefore, the parameter redefinition decoder proposed in this embodiment achieves SOTA performance (97.0% mAP50) in the crack detection and pixel-by-pixel object segmentation challenges. Box ,94.4%mAP50-95 Mask ).

[0104] Table 4 Performance of parameter redefinition decoder compared with other methods on building crack validation set

[0105]

[0106] The simulation process is as follows:

[0107] In the building crack dataset, in order to more fully verify that the dynamic parameter redefinition crack pixel-by-pixel object segmentation network is superior to other SOTA methods in the crack detection challenge, multiple SOTA segmentation algorithms "n" (nanometer) to "x" (extra large) models were selected for multiple comparisons; Tables 5 and 6 record the experimental results. From the benchmark test results, it can be seen that with the increase in the amount of model calculation, the pixel-by-pixel object segmentation accuracy of multiple SOTA methods has improved; YOLOv6 / 7 achieves the inversion of YOLOv8 on the 'l' and 'x' models, and the amount of calculation is twice that of YOLOv8; compared with the traditional simple increase in model calculation, the pipeline based on parameter redefinition and Transformer architecture proposed in this embodiment can effectively capture the global context information and long-range dependencies of cracks with minimal parameters, and ultimately achieve accurate pixel-by-pixel object segmentation of cracks.

[0108] Table 5 Comparison with YOLOv6 / 7 / 8 models with different computational loads on the building crack validation set

[0109]

[0110] Table 6 Comparison with mainstream object segmentation detectors on the building crack validation set

[0111]

[0112]

[0113] Rural roads, this dataset covers many types of cracks; more importantly, the length and width of the cracks have a large span. The fast multi-scale attention introduced in this embodiment can handle this situation well and can effectively capture the local information of long-range dependencies and long-range cracks; therefore, the pipeline proposed in this embodiment can also be used in various crack pixel-by-pixel object segmentation challenges (96.7% mAP50 Box ,92.8%mAP50 Mask ) achieves SOTA performance, and Table 7 shows the benchmark results.

[0114] Table 7 Comparison with mainstream object segmentation detectors on the rural road validation set

[0115]

[0116] As can be seen from Table 8, the dynamic parameter redefinition crack pixel-by-pixel object segmentation network achieves excellent results on the crack LS315, and has a significant improvement in mAP50Box (+11.9% mAP50) compared to the baseline YOLOv8s. Box , +8.3% mAP50 Mask , -2.4GFLOPs), since the crack LS315 contains a large number of fine-grained cracks and the contrast between the cracks and the image background is low. It is extremely difficult to perform accurate pixel-by-pixel object segmentation; the pipeline proposed in this embodiment can extract higher-level crack semantic information with fewer parameters; therefore, when segmenting low-contrast crack images, the cracks can be better distinguished from the background, and pixel-by-pixel object segmentation can be performed; the prediction results are as follows Figure 7 As shown, Figure 7 From top to bottom in the figure are the original image, ground truth, the result of YOLOv8s and the result of the proposed dynamic parameter redefinition crack pixel-by-pixel object segmentation network.

[0117] In order to enable the pipeline of this embodiment to be better deployed on edge computing facilities, TensorRT is used to quantize the trained model into three different modes: single-precision FP32, half-precision FP16, and INT8. Affected by factors such as image size and clarity, the same model has different inference efficiency for different images; in addition, there are differences in detection accuracy. In this paper, all images in each dataset are tested and the results are averaged. The benchmark test results are shown in Table 9.

[0118] Table 9 Benchmark test results in different modes on edge computing cards

[0119]

[0120] In three different modes, FP32 floating point numbers are represented using 32 bits, providing higher precision and dynamic range. The advantage of INT8 over FP16 and FP32 is that the amount of data calculated is relatively small, achieving faster calculation speed; however, the corresponding detection accuracy is also the lowest; FP16 is more accurate than INT8 and faster than FP32; it is a balance mode between detection speed and accuracy; after testing and considering the actual application scenarios of crack detection, this embodiment selects the FP16 mode; compared with FP32, the detection accuracy of FP16 is only reduced by less than 1%. However, it can significantly reduce storage space and computing overhead.

[0121] The principles and implementation methods of the present invention are described in this article using specific embodiments. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for dynamic crack detection and pixel-by-pixel object segmentation based on parameter redefinition, characterized in that: include: Establishing a dataset of target crack images; Re-defining the Transformer architecture for the input parameters of the data set to extract context information and obtain semantic information of cracks in the data set; The parameter redefinition Transformer architecture includes a RepConv module and a TC2F module. The RepConv module and the TC2F module are used as a double-layer training module to obtain high-level information of the target crack image in the data set, thereby realizing the positioning of the target crack and pixel-by-pixel object division.

2. The method for dynamic crack detection and pixel-by-pixel object segmentation based on parameter redefinition according to claim 1, characterized in that: The target crack image dataset includes but is not limited to building cracks, rural roads, and cracks LS315 public dataset.

3. The method for dynamic crack detection and pixel-by-pixel object segmentation based on parameter redefinition according to claim 1, characterized in that: The RepConv module and the TC2F module include: The RepConv module combines 3×3 convolution, 1×1 convolution, and identity connection in a single convolutional layer to provide diverse gradients for mapping the input feature information of the dataset; The TC2F module specifically introduces a multi-layer TransformerBlock in the C2F architecture, and the TransformerBlock converts the target crack image feature information of the input data set into Reshape into a flat 2D patch Where H×W is the resolution of the original image, C is the number of channels, and P 2 For the resolution of each target crack image patch, the flattened feature information X is passed through the TransformerLayer architecture and multiple linear layers to obtain a feature-rich target feature information x1.

4. The method for dynamic crack detection and pixel-by-pixel object segmentation based on parameter redefinition according to claim 3, characterized in that: The TransformerLayer architecture divides the input feature information X into three components, query Q, key K and value V, and then passes through multiple linear layers in sequence; Q, K, V in the linear layer are aggregated in the multi-head attention MHA, and then summed with the divided original components Q, K, V, and finally pass through two linear layers to obtain the output feature information.

5. The method for dynamic crack detection and pixel-by-pixel object segmentation based on parameter redefinition according to claim 4, characterized in that: The output header in the multi-head attention MHA is specifically as follows: In the formula, i is the number of heads, head i is the output of the head, the matrix They represent the query transformation matrix, key transformation matrix, and value transformation matrix of the head respectively, and Attention represents the attention calculation function; The Attention represents the attention calculation function as follows: Where, d k Represents the dimension of the K vector, and Softmax normalizes the weight of each K vector based on similarity.

6. The method for dynamic crack detection and pixel-by-pixel object segmentation based on parameter redefinition according to claim 1, characterized in that: The method also includes a fast multi-scale attention architecture; the fast multi-scale attention architecture captures target features of an input target crack image through two parallel convolutions, the two parallel convolutions include a 1×1 convolution and a 3×3 convolution, and is used to perform enhanced feature extraction after parameter redefinition of the Transformer architecture extracts context feature information.

7. The method for dynamic crack detection and pixel-by-pixel object segmentation based on parameter redefinition according to claim 6, characterized in that: The method also includes a parameter redefining decoder for reducing the amount of computation of the method after the fast multi-scale attention architecture captures the target features of the input target crack image, including: Combine the convolutional layer with the batch normalization BN layer. The BN formula is as follows: In the formula, is the normalized output value, x i is the original input value, μ is the mean of the batch data, σ 2 is the variance of the batch data, γ is a learnable scaling parameter, β is a learnable translation parameter, and ε is a constant; Convert equation (3) to Y = w BN +b BN in the form of Therefore, the combination of convolutional layer and BN layer is: In the formula, is the output value after processing by the convolution layer and batch normalization layer, w BN is the scaling parameter of the batch normalization layer, b BN is the translation parameter of the batch normalization layer, w conv is the weight parameter of the convolutional layer, b conv is the bias parameter of the convolutional layer, and x is the input data; The combined convolution parameters are as follows: Where w is the effective weight after fusion and w is the scaling parameter of batch normalization. BN With the original convolution weight b conv The product of, b is the effective bias after fusion, b conv is the original convolution bias; The form is The BN layer calculation process is as follows:

Images