Detection method and device for airfield pavement crack segmentation

Abstract

The invention discloses a detection method and device for airport pavement crack segmentation. The method comprises the following steps: acquiring an airport pavement crack image; training the airfield pavement crack segmentation model, and generating a crack segmentation prediction result for the airfield pavement crack image based on the trained model; and forming a crack pavement safety state detection report based on a prediction result. The device comprises a processor and a memory. According to the method, a new loss function is introduced into the airfield pavement crack segmentation model, so that the migration generalization ability of the model among different airfield pavement scenes is improved, and the detection precision of the airfield pavement cracks is improved.

Description

Technical Field

[0001] The invention relates to the field of airport pavement detection, and in particular to a detection method and device for airport pavement crack segmentation. Background Art

[0002] As the main carrier of aircraft ground operation, airport pavement is an important infrastructure to ensure the safe operation of airports. Cracks are the most common pavement surface disease and the initial form of pavement structural damage. Therefore, the International Civil Aviation Organization (ICAO) has made crack detection a mandatory regulation to ensure the healthy operation of airports. In the past, crack detection tasks were completed manually, which was easily affected by subjective factors and caused missed detection, false detection, etc., and the labor cost was extremely high. Traditional manual inspections have been unable to meet the increasing detection requirements, so in recent years, automated crack detection technology has become the mainstream research direction. Thanks to its powerful feature extraction capabilities, deep learning-based methods have been widely used in the field of crack detection, achieving efficient crack detection and providing data reference for pavement structure safety monitoring. Due to long-term high-load operation, the pavement is seriously polluted by fuel stains, aircraft wheel marks, etc., making the background environment of cracks very complex. The proportion of positive sample pixels is much smaller than that of background pixels, and the data distribution is highly unbalanced, which brings difficulties to pixel-level fine detection of cracks.

[0003] Crack detection not only provides a scientific basis for early damage prevention and repair, but also reduces the risk of wider road damage caused by crack propagation, ensuring traffic safety and smoothness. In recent years, the application of deep learning technology has significantly improved the accuracy and efficiency of crack segmentation.

[0004] However, when deploying models to practical applications, despite outstanding performance on standardized datasets, the generalization performance of these models in complex and changing real-world environments is often unsatisfactory. In practical applications, crack segmentation models often show high false positive and missed positive rates due to insufficient generalization ability, and are difficult to cope with complex and changing road environments and cracks with complex topological structures. Limited by the amount and diversity of training data, overfitting becomes a common problem, resulting in a significant decrease in segmentation accuracy when the model faces unfamiliar new scenarios, reflecting the urgent need to improve its generalization performance through various technologies. Summary of the invention

[0005] The present invention provides a detection method and device for airport pavement crack segmentation. The present invention introduces a new loss function into the airport pavement crack segmentation model to improve the migration and generalization ability of the model between different airport pavement scenes, thereby improving the detection accuracy of cracks on the airport pavement. The details are described below:

[0006] In a first aspect, a detection method for airport pavement crack segmentation comprises:

[0007] Acquire crack images of airport pavement;

[0008] Train the airport pavement crack segmentation model, and generate crack segmentation prediction results for the airport pavement crack image based on the trained model;

[0009] The cracked pavement is repaired based on the prediction results.

[0010] Among them, the airport pavement crack segmentation model is obtained by training a U-Net neural network model with a training data set consisting of tunnel crack images labeled at the pixel level. The U-Net neural network model consists of an encoder and a decoder, and the features of the encoder and the decoder are fused through jump connections.

[0011] The training of the airport pavement crack segmentation model is as follows:

[0012] An active fault-tolerant loss calculation method is proposed to act on the loss calculation stage during model training, including: neighborhood spatial information modeling and overall Gaussian smoothing;

[0013] The Markov random field is used to model the correlation of pixels in the neighborhood of the label center point and construct an active fault-tolerant loss;

[0014] The active fault-tolerant loss takes the crack segmentation prediction results output by the U-Net segmentation model and its corresponding label image as input. The domain space information modeling part is responsible for modeling the relative offset probability in the label neighborhood space corresponding to any pixel point in the crack segmentation prediction result, and the Gaussian smoothing part is responsible for smoothing the neighborhood space offset probability of all pixels.

[0015] Among them, the Markov random field is used to model the correlation of pixels in the neighborhood of the label center point, and the active fault tolerance loss is constructed as:

[0016] A neighborhood window W of size ω is set, ω=2n+1, to constrain the scope of the Markov random field; the relative offset probability based on the correlation within the neighborhood is calculated within the window; finally, it is integrated into a metric score for loss calculation.

[0017] Among them, the potential energy function of the Markov random field is expressed as:

[0018]

[0019] Where x represents the potential possibility of deviation between label pixels and predicted segmentation, V represents nodes, E represents edges, s and t represent predicted segmentation and label respectively, and ω represents the size of the neighborhood window; Φ(·) is the unary potential energy function of the node. In the neighborhood window, the deviation probability is calculated using the segmentation prediction and label pixels output by the segmentation model; Ψ(·) is the point pair potential energy function of the edge, which is used to smooth the spatial correlation between different neighborhood windows. i The impact of the drift probability.

[0020] Furthermore, the unary potential energy function of the node is:

[0021] Φ(x i ,s i ,t |ω| )=-ln p(x i | i ,t |ω| >)

[0022] Among them, p(x i | i ,t i >) indicates that when the segmentation prediction is s i The probability of a relative shift in the neighborhood space of the corresponding label is modeled using the following formula:

[0023]

[0024] in, is an XOR operation, ω is the size of the neighborhood window, d(s i ,t j ) means s i With t j The distance between.

[0025] Among them, the point pair potential energy function of the edge is:

[0026]

[0027] where μ(·) is the difference between the associated point pairs in the offset matrix x, that is, μ(x i ,x j )=|x i -x j |,s i With s j It is the corresponding value of the associated point pair in the segmentation prediction. The exponential part takes the difference between adjacent pixel labels as a variable to increase the perception of the local area. σ is a hyperparameter used to adjust the impact of the prediction deviation in the neighborhood window on the Markov random field.

[0028] Among them, the loss function of the joint Markov random field is: ​​

[0029]

[0030] Where N is the size of the image data, s i is the pixel value in the segmentation prediction output by the model, t i is the pixel value in the label. i is the relative offset probability corresponding to the point, that is, the result calculated by the active fault tolerance loss.

[0031] In a second aspect, a detection device for airport pavement crack segmentation, the device comprises: a processor and a memory, the memory stores program instructions, and the processor calls the program instructions stored in the memory to make the device execute any one of the methods described in the first aspect.

[0032] In a third aspect, a computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes any one of the methods described in the first aspect.

[0033] The beneficial effects of the technical solution provided by the present invention are:

[0034] 1. The present invention proposes an improved loss function, which uses Markov random fields to model the active fault tolerance mechanism. By combining pixel similarity and spatial information, the correlation between the center point and each point in the neighborhood is constructed, thereby improving the migration performance of the pre-trained crack segmentation model in actual scenes. This improved loss function, PFT Loss, not only provides a more reasonable supervision signal, alleviates the shortcomings of traditional loss functions in single-pixel sensitivity, but also enhances the model's perception of contextual information;

[0035] 2. In the crack detection task, the model needs to accurately identify and segment the crack area in the image; however, the traditional loss function often only focuses on the prediction results of a single pixel, which is far from the active fault tolerance mechanism in the human visual system; PFT Loss effectively simulates the active fault tolerance characteristics of the human visual system by predicting the correlation between segmentation and label pixel neighborhood space through Markov random field modeling; this enables the model to identify cracks more robustly in the face of complex scenes such as noise, shadows, and lighting changes, without being disturbed by the errors of a single pixel;

[0036] 3. In addition, PFT Loss has shown a high degree of universality on different data sets and network frameworks, enabling it to work stably on various road types and changing environmental conditions. In rigorous experiments on different road types and complex environmental conditions, the model trained with the improved loss function has shown significantly improved generalization ability. Not only has the accuracy on multiple standard test sets been maintained or improved, but more importantly, in real road environments, the model has shown more robust detection performance.

[0037] 4. This improved generalization and robustness greatly enhances the value of the model in practical applications; by using PFT Loss, the model can more effectively cope with various uncertainties in real-world scenarios, such as different road materials (such as asphalt, cement), climate conditions (such as rainy days, sunny days), and lighting changes (such as daytime, nighttime). This performance improvement makes the model have broad application prospects and important practical significance in actual road monitoring, maintenance and management. Ultimately, this improvement not only improves the accuracy and reliability of crack detection, but also helps to extend the service life of infrastructure and reduce maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of active fault-tolerant loss calculation of the present invention;

[0039] Figure 2 A predicted segmentation map output by the model of the embodiment of the present invention during a certain iteration process;

[0040] Figure 3 This is a diagram showing the effect of segmenting cracks using the present invention and models trained with different loss functions. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0042] Although many methods have been proposed to enhance the generalization of models, such as expanding data diversity, optimizing network structure, and adopting transfer learning, these strategies have their pros and cons. Increasing data diversity helps the model adapt to unseen scenarios, but it may still not cover all situations. Adjustments to the network architecture sometimes increase the complexity and computational burden of the model, and the effect of transfer learning may be limited by the differences between the source task and the target task. Overall, although these methods have made progress in some aspects, they have not yet solved the problem of model generalization.

[0043] Therefore, the embodiment of the present invention improves the model's ability to migrate and generalize between different airport pavement scenarios by introducing a new loss function into the airport pavement crack segmentation model. The embodiment of the present invention found during the analysis process that active fault-tolerance mechanisms are prevalent in the human perspective. In the data annotation stage, due to the complex background and inevitable subjective misjudgment, the annotators retain a certain tolerance interval for the segmentation of small and irregular targets. When performing crack segmentation, in order to be able to divide the cracks as much as possible, the annotators will divide a part of the background pixels into cracks, thereby forming a circle of fuzzy boundaries around the main body of the crack. In the model evaluation stage, considering that the width of the cracks is relatively small, many previous works generally set a tolerance of 2 pixels for the segmentation results, and this fault-tolerance mechanism during evaluation has been used in some recent works.

[0044] Through further exploration, the embodiments of the present invention found that the role of the active fault-tolerant mechanism in artificial crack segmentation is highly similar to the loss function used in the crack segmentation model based on deep learning. This inspires the embodiments of the present invention to think about two questions: Does an overly strict supervision signal limit the learning of the small crack segmentation model? Can the active fault-tolerant mechanism in human vision be explicitly injected into the model learning process to enhance the generalization ability of the model?

[0045] Subsequent in-depth analysis of the loss function found that it has a significant impact on the generalization performance during the model training process. In the field of crack segmentation, cross entropy, Dice Loss or Focal Loss are generally used as loss functions, and previous related work has also proved the effectiveness of these loss functions. However, cracks are typical irregular and small targets with random ductile topological structures, and their internal features are very sparse. In addition, when analyzing the above-mentioned loss function calculation process, the embodiment of the present invention found that this method of only considering a single pixel for loss calculation is suboptimal for the segmentation of small cracks. The traditional loss function based on single-point calculation naturally filters the active fault tolerance interval and ignores the contextual information of the label pixel in the neighborhood space, which makes the crack segmentation model easy to overfit the label distribution, thereby affecting the generalization ability of the model in actual scenarios.

[0046] Example 1

[0047] A detection method for crack segmentation in airport pavement, see Figure 1 , the method comprises the following steps:

[0048] 101: Acquire an airport pavement crack image;

[0049] The crack images of the airport pavement are obtained by using the RGB camera carried by the high-speed inspection robot. The image data needs to be as clear as possible to facilitate the subsequent crack segmentation processing.

[0050] 102: Using the trained airport pavement crack segmentation model to generate crack segmentation prediction results for the airport pavement crack image;

[0051] Among them, the airport pavement crack segmentation model is obtained by training the U-Net neural network model with a training data set consisting of pixel-level annotated tunnel crack images. The U-Net neural network model consists of an encoder (downsampling path) and a decoder (upsampling path), and the features of the encoder and decoder are fused through jump connections. The proposed active fault-tolerant loss calculation method acts on the loss calculation stage during model training, including: neighborhood spatial information modeling and overall Gaussian smoothing.

[0052] The embodiment of the present invention uses Markov random fields to model the correlation of pixels in the neighborhood of the label center point and construct an active fault-tolerant loss, such as Figure 1 shown.

[0053] In an embodiment, a training method for a crack segmentation model of a tunnel crack image is as follows:

[0054] First, a training data set is obtained. The training data set includes several groups of training data. Each group of training data includes an airport pavement crack image and a pixel-level label of the airport pavement crack image.

[0055] Finally, in an iterative cycle of training the U-Net segmentation model using the constructed dataset, in the loss calculation stage, the proposed active fault-tolerant loss calculation method uses the crack segmentation prediction results output by the U-Net segmentation model and its corresponding label image as input. Among them, the domain space information modeling part is responsible for modeling the relative offset probability in the label neighborhood space corresponding to any pixel point in the crack segmentation prediction result, and the Gaussian smoothing part is responsible for smoothing the neighborhood space offset probability of all pixels. Then the value calculated by the active fault-tolerant loss will be integrated into the overall loss function as the supervision signal of the U-Net segmentation model for reverse gradient update. The iterative cycle ends and enters the next iterative cycle until the end condition is met.

[0056] For example, the training of the crack segmentation model for the airport pavement crack image is explained in more detail as follows:

[0057] Step 1: Establish an airport pavement crack image dataset, which includes the following steps:

[0058] Step 1.1: Construct an airport pavement crack image dataset based on the airport pavement crack images taken by the airport pavement high-speed inspection robot, which contains 2000 airport pavement crack images with a fixed resolution of 1800×900.

[0059] Step 1.2: Use the Labelme annotation tool to annotate the airport pavement crack image at the pixel level. Pixel-level annotation here refers to annotating the category to which each pixel in the collected airport pavement crack image belongs. The annotated categories are two categories: crack pixels and non-crack pixels.

[0060] Step 1.3: In order to improve the robustness of the crack segmentation model of airport pavement crack images and enable it to adapt to airport pavement crack detection in complex environments, data enhancement operations are performed on the labeled airport pavement crack image dataset, that is, random adjustments to image brightness, contrast, image rotation, scale transformation, and noise addition are performed.

[0061] Step 1.4: Randomly divide the airport pavement crack images into training set and validation set in a ratio of 8:2 to complete the construction of the airport pavement crack image dataset.

[0062] Step 2: Construct an improved U-Net neural network and use the above-mentioned airport pavement crack image dataset for training. During the training process, the proposed active fault-tolerant loss function is used to calculate the loss and supervise the gradient optimization direction of the model.

[0063] Specifically, first, the embodiment of the present invention sets a neighborhood window W of size ω, ω=2n+1, to constrain the scope of the Markov random field; then, the relative offset probability calculation based on the correlation degree within the neighborhood is performed within the window; finally, it is integrated into a metric score for loss calculation.

[0064] First, the general process of Markov random field modeling is briefly described. The label T is a binary pixel matrix containing n elements, where 0 and 1 represent background and cracks respectively. The segmentation prediction S output by the model is a probability matrix. X is the offset matrix between any pair of points in T and S. For a given S and T, calculate any pixel s in S i Relative to the label pixel t i The probability P(x) of a shift occurring within the neighborhood window W i |s i ,t i ). The offset probability is then integrated into the loss calculation as a reference for proactive fault tolerance.

[0065] It should be noted that when considering the interaction between pixels, the image data can be naturally constructed as an undirected graph, and after adding the neighborhood window, it has the Markov property. The potential energy function of the Markov random field is expressed as:

[0066]

[0067] Where x represents the potential possibility of deviation between the label pixel and the predicted segmentation, V represents the node, E represents the edge, s and t represent the predicted segmentation and label respectively, and ω is the neighborhood window size. Φ(·) is the unary potential energy function of the node, which directly uses the segmentation prediction and label pixel output by the segmentation model to calculate the deviation probability. Ψ(·) is the point pair potential energy function of the edge, which is used to measure the spatial correlation between nodes in the neighborhood window of the label pixel to s. i Impact of offset probability. The optimization objective of the Markov random field model can be defined as minimizing the segmentation prediction offset probability.

[0068] 2.1 Neighborhood window setting. When applying Markov random fields in image segmentation tasks, the entire image is generally constructed as an undirected graph. If the global pixel relationship is considered, the computational complexity will be greatly increased. Therefore, the embodiment of the present invention uses a neighborhood window to constrain the scope of the Markov random field. The specific setting of the neighborhood window W is as follows:

[0069]

[0070] Among them, i and j are the offsets of each point in the window relative to the center point in the row direction and column direction respectively. Using relative position instead of sequential position encoding makes it easier to model the distance between points. At the same time, the relative position encoding method can maintain the rotation and translation invariance between pixels in the neighborhood. For example: when calculating an edge pixel x in the upper left corner of the neighborhood a,b With the center pixel x c,d If the sequential position encoding is used, the distance between the two points d = (ac) 2 +(bd) 2 , if relative position encoding is used, then d = |i+j|. Obviously, relative position encoding can retain the spatial distance measurement function and save computational cost.

[0071] 2.2 Node potential energy calculation. As an important part of the energy function, the unary potential energy function is responsible for predicting any pixel point s in the segmentation of the model output. i and pixel t in the label neighborhood window |ω| The relationship between is modeled, and the spatial position relationship and feature semantics are comprehensively considered to be the offset matrix corresponding to the point x i Assignment. The definition is as follows:

[0072] Φ(x i ,s i ,t |ω| )=-ln p(x i | i ,t |ω| >) (3)

[0073] Among them, p(x i ​| i ,t i >) indicates that when the segmentation prediction is s i The probability of a relative shift in the neighborhood space of the corresponding label. In order to compare with the label more intuitively, the embodiment of the present invention first converts s i Binarization is performed, and the threshold is set to 0.8. When the probability value is greater than 0.8, it can be considered that the segmentation result of the point is correct. The following formula is used for modeling:

[0074]

[0075] in, is an XOR operation, and ω is the size of the neighborhood window. The first term in equation (4) is used to control the effect of the Markov random field. Only the label node t i and predicted segmentation i When they are not the same, the result of the XOR operation is 1, and the label distribution in the neighborhood window needs to be examined. The numerator of the first term in equation (4) is used to integrate the spatial feature semantics in the domain window. When the model misclassifies a pixel, the label neighborhood window t |ω| Internal and predicted segmentation i Pixels with the same value t j All are likely to become i The relative offset point. Figure 1 For example, P1 and P2 in the neighborhood window of P1 have four points with the same label value as P1, and no point in the neighborhood window of P2 has the same value as P2. Therefore, P1 is more likely to be the point where the prediction offset occurs. The denominator of the first term in equation (4) is used to fuse the spatial distance information in the neighborhood window, d(s i ,t j ) means s i With t j The distance between them can be directly calculated with the help of the coordinates in the neighborhood window, and the offset probability is inversely proportional to the distance.

[0076] 2.3 Edge potential energy calculation. When the number of offset points in the neighborhood space is large, it indicates that the difference between the annotation and the actual crack in the local area is large, so a larger penalty is needed to promote model correction. In order to achieve this goal, the edge potential energy function must be able to effectively punish such local inconsistencies, thereby increasing the model's attention to ambiguous areas. Combined with prior knowledge, specifically, the embodiment of the present invention can define a binary potential energy function as follows:

[0077]

[0078] where μ(·) is the difference between the associated point pairs in the offset matrix x, that is, μ(x i ,x j )=|x​i -x j |.s i With s j It is the corresponding value of the associated point pair in the segmentation prediction. The exponential part takes the difference between adjacent pixel labels as a variable to increase the perception of the local area. σ is a hyperparameter used to adjust the impact of the prediction deviation in the neighborhood window on the Markov random field.

[0079] 2.4 Loss function. The potential energy function in the Markov random field is an estimate of the posterior probability that the predicted segmentation will shift within the neighborhood window, and performs local active fault tolerance on the traditional loss calculation. After completing the calculation of the Markov random field potential energy function, it is integrated into the final loss function of the model training. The embodiment of the present invention is improved on the basis of Dice Loss, and the specific expression is as follows:

[0080]

[0081] Where N is the size of the image data, s i is the pixel value in the segmentation prediction output by the model, t i is the pixel value in the label. i is the relative offset probability corresponding to the point, that is, the result calculated by the active fault tolerance loss.

[0082] The embodiment of the present invention injects the offset probability of the predicted segmented pixel in the neighborhood space into the numerator of the original Dice Loss expression to smooth the impact of the strict loss calculation. When the offset probability is large, the cost of incorrect prediction will be reduced, and the model will actively tolerate labeling errors.

[0083] Example 2

[0084] The feasibility of the solution in Example 1 is verified by combining specific experimental data, as described below:

[0085] The comparison experiment with the commonly used loss functions is shown in Table 1. Four evaluation indicators, namely precision (PR), recall (RE), F1 score and mean intersection over union (mIoU), are selected to comprehensively and deeply evaluate the performance of the model.

[0086] Table 1

[0087]

[0088] The experimental results show that compared with other loss functions, the U-Net model trained with PFT Loss has improved in the three evaluation indicators of Recall, F1 score and mIoU. Although the Precision indicator has declined, the difference between it and the Recall indicator is smaller, which proves that the model has a more balanced performance to a certain extent.

[0089] The embodiment of the present invention introduces a new loss function into the airport pavement crack segmentation model, thereby improving the model's migration and generalization capabilities between different airport pavement scenarios and improving the detection accuracy of cracks on airport pavements.

[0090] Example 3

[0091] A detection device for airport pavement crack segmentation, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to enable the device to execute the following method steps in embodiment 1:

[0092] Acquire crack images of airport pavement;

[0093] Train the airport pavement crack segmentation model, and generate crack segmentation prediction results for the airport pavement crack image based on the trained model;

[0094] The cracked pavement is repaired based on the prediction results.

[0095] Among them, the airport pavement crack segmentation model is obtained by training the U-Net neural network model with a training data set consisting of pixel-level annotated tunnel crack images. The U-Net neural network model consists of an encoder and a decoder, and the features of the encoder and decoder are fused through jump connections.

[0096] Among them, the airport pavement crack segmentation model is trained as follows:

[0097] An active fault-tolerant loss calculation method is proposed to act on the loss calculation stage during model training, including: neighborhood spatial information modeling and overall Gaussian smoothing;

[0098] The Markov random field is used to model the correlation of pixels in the neighborhood of the label center point and construct an active fault-tolerant loss;

[0099] The active fault-tolerant loss takes the crack segmentation prediction results output by the U-Net segmentation model and its corresponding label image as input. The domain space information modeling part is responsible for modeling the relative offset probability in the label neighborhood space corresponding to any pixel point in the crack segmentation prediction result, and the Gaussian smoothing part is responsible for smoothing the neighborhood space offset probability of all pixels.

[0100] Among them, the Markov random field is used to model the correlation of pixels in the neighborhood of the label center point, and the active fault-tolerant loss is constructed as:

[0101] A neighborhood window W of size ω is set, ω=2n+1, to constrain the scope of the Markov random field; the relative offset probability based on the correlation within the neighborhood is calculated within the window; finally, it is integrated into a metric score for loss calculation.

[0102] Among them, the potential energy function of the Markov random field is expressed as:

[0103]

[0104] Where x represents the potential possibility of deviation between label pixels and predicted segmentation, V represents nodes, E represents edges, s and t represent predicted segmentation and label respectively, and ω represents the size of the neighborhood window; Φ(·) is the unary potential energy function of the node. In the neighborhood window, the deviation probability is calculated using the segmentation prediction and label pixels output by the segmentation model; Ψ(·) is the point pair potential energy function of the edge, which is used to smooth the spatial correlation between different neighborhood windows. i The impact of the drift probability.

[0105] Furthermore, the unary potential energy function of the node is:

[0106] Φ(x i ,s i ,t |ω| )=-ln p(x i | i ,t |ω| .)

[0107] Among them, p(x i | i ,t i >) indicates that when the segmentation prediction is s i The probability of a relative shift in the neighborhood space of the corresponding label is modeled using the following formula:

[0108]

[0109] in, is an XOR operation, ω is the size of the neighborhood window, d(s i ,t j ) means s i With t j The distance between.

[0110] Among them, the point pair potential energy function of the edge is:

[0111]

[0112] ​​where μ(·) is the difference between the associated point pairs in the offset matrix x, that is, μ(x i ,x j )=|x i -x j |,s i With s j It is the corresponding value of the associated point pair in the segmentation prediction. The exponential part takes the difference between adjacent pixel labels as a variable to increase the perception of the local area. σ is a hyperparameter used to adjust the impact of the prediction deviation in the neighborhood window on the Markov random field.

[0113] Among them, the loss function of the joint Markov random field is:

[0114]

[0115] Where N is the size of the image data, s i is the pixel value in the segmentation prediction output by the model, t i is the pixel value in the label. i is the relative offset probability corresponding to the point, that is, the result calculated by the active fault tolerance loss.

[0116] It should be pointed out here that the device description in the above embodiment corresponds to the method description in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0117] The execution subjects of the above-mentioned processor and memory can be devices with computing functions such as computers, single-chip microcomputers, and microcontrollers. In specific implementation, the embodiments of the present invention do not limit the execution subjects and are selected according to the needs of actual applications.

[0118] The data signal is transmitted between the memory and the processor via a bus, which is not described in detail in the embodiment of the present invention.

[0119] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, the storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the method steps in the above embodiment.

[0120] The computer-readable storage medium includes but is not limited to a flash memory, a hard disk, a solid-state drive, and the like.

[0121] It should be pointed out here that the description of the readable storage medium in the above embodiment corresponds to the description of the method in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0122] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present invention are generated.

[0123] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be accessed by the computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium or a semiconductor medium, etc.

[0124] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.

[0125] Those skilled in the art will appreciate that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A detection method for airport pavement crack segmentation, characterized in that: The method comprises: Acquire airport pavement crack images; Train the airport pavement crack segmentation model, and generate crack segmentation prediction results for the airport pavement crack image based on the trained model; A crack pavement safety status inspection report is generated based on the prediction results.

2. A detection method for airport pavement crack segmentation according to claim 1, characterized in that: The airport pavement crack segmentation model is obtained by training a U-Net neural network model with a training data set consisting of tunnel crack images labeled at the pixel level. The U-Net neural network model consists of an encoder and a decoder, and the features of the encoder and the decoder are fused through a jump connection.

3. A detection method for airport pavement crack segmentation according to claim 1, characterized in that: The training of the airport pavement crack segmentation model is as follows: An active fault-tolerant loss calculation method is proposed to act on the loss calculation stage during model training, including: neighborhood spatial information modeling and overall Gaussian smoothing; The Markov random field is used to model the correlation of pixels in the neighborhood of the label center point and construct an active fault-tolerant loss; The active fault-tolerant loss takes the crack segmentation prediction results output by the U-Net segmentation model and its corresponding label image as input. The domain space information modeling part is responsible for modeling the relative offset probability in the label neighborhood space corresponding to any pixel point in the crack segmentation prediction result, and the Gaussian smoothing part is responsible for smoothing the neighborhood space offset probability of all pixels.

4. A detection method for airport pavement crack segmentation according to claim 3, characterized in that: The Markov random field is used to model the correlation of pixels in the neighborhood of the label center point, and the active fault-tolerant loss is constructed as: A neighborhood window W of size ω is set, ω=2n+1, to constrain the scope of the Markov random field; the relative offset probability based on the correlation within the neighborhood is calculated within the window; finally, it is integrated into a metric score for loss calculation.

5. The detection method for airport pavement crack segmentation according to claim 3, characterized in that: The potential energy function of the Markov random field is expressed as: Where x represents the potential possibility of deviation between label pixels and predicted segmentation, V represents nodes, E represents edges, s and t represent predicted segmentation and label respectively, and ω represents the size of the neighborhood window; Φ(·) is the unary potential energy function of the node. In the neighborhood window, the deviation probability is calculated using the segmentation prediction and label pixels output by the segmentation model; Ψ(·) is the point pair potential energy function of the edge, which is used to smooth the spatial correlation between different neighborhood windows. i The impact of the drift probability.

6. A detection method for airport pavement crack segmentation according to claim 5, characterized in that: The unary potential energy function of the node is: Φ(x i ,s i ,t |ω| )=-ln p(x i |<s i ,t |ω| >) Among them, p(x i | i ,t i >) indicates that when the segmentation prediction is s i The probability of a relative shift in the neighborhood space of the corresponding label is modeled using the following formula:​ in, is an XOR operation, ω is the size of the neighborhood window, d(s i ,t j ) means s i With t j The distance between.

7. A detection method for airport pavement crack segmentation according to claim 5, characterized in that: The point pair potential energy function of the edge is: where μ(·) is the difference between the associated point pairs in the offset matrix x, that is, μ(x i ,x j )=|x i -x j |,s i With s j It is the corresponding value of the associated point pair in the segmentation prediction. The exponential part takes the difference between adjacent pixel labels as a variable to increase the perception of the local area. σ is a hyperparameter used to adjust the impact of the prediction deviation in the neighborhood window on the Markov random field.

8. The detection method for airport pavement crack segmentation according to claim 5, characterized in that: The loss function of the Markov random field is: Where N is the size of the image data, s i is the pixel value in the segmentation prediction output by the model, t i is the pixel value in the label. i is the relative offset probability corresponding to the point, that is, the result calculated by the active fault tolerance loss.

9. A detection device for airport pavement crack segmentation, characterized in that: The device comprises: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute any one of the methods of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 8.

Images