An asphalt pavement crack identification method based on joint optimization network

Abstract

The application provides an asphalt pavement crack identification method based on a joint optimization network, and relates to the technical field of intelligent pavement disease detection. The method first acquires an asphalt pavement disease image, acquires a label image corresponding to the number of images, and constructs an asphalt pavement crack image dataset; then an adaptive image denoising network is designed, including a multi-filter denoising module and a hyperparameter prediction module; a spatial convolution attention U-Net network is designed as a semantic segmentation network; the adaptive image denoising network and the semantic segmentation network are combined by using a joint optimization strategy to form an IDSNet network, the IDSNet network is optimized by using the training loss of the semantic segmentation network, the synchronization of image denoising and crack segmentation of the asphalt pavement crack image is realized; finally, crack morphology identification and parameter calculation are performed based on the crack segmentation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent detection of road surface diseases, and particularly relates to a method for identifying asphalt pavement cracks based on a joint optimization network. BACKGROUND

[0002] In highway maintenance management, cracks, as one of the most common diseases of the road surface, accurate detection is an important basis for evaluating the technical condition of the road surface. However, the traditional manual detection method is low in efficiency and strong in subjectivity, and is difficult to adapt to the needs of large-scale road surface detection. With the development of automatic detection technology of the road surface, the crack detection method based on deep learning has gradually become the mainstream, but it also faces multiple challenges. First, in the process of collecting road surface images, the imaging device of the automatic detection system of the road surface is disturbed by environmental light, which leads to the decline in image quality and introduces a large amount of noise. Secondly, due to the band-shaped gully shape and complex topological structure of the road surface cracks, the traditional semantic segmentation network is difficult to learn and extract the features of the small cracks. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a method for identifying asphalt pavement cracks based on a joint optimization network to solve the problems of the prior art.

[0004] To solve the above technical problems, the technical solution adopted by the present application is:

[0005] On the one hand, the present application provides a method for identifying asphalt pavement cracks based on a joint optimization network, comprising:

[0006] Obtaining an asphalt pavement disease image and a corresponding label image of the same number of images to construct an asphalt pavement crack image dataset;

[0007] Designing an adaptive image denoising network; the adaptive image denoising network comprises a multi-filter denoising module and a hyperparameter prediction module; the multi-filter denoising module comprises a plurality of filters, and the parameters of each filter are dynamically adjusted through a joint optimization strategy to realize adaptive denoising under different noise conditions; the hyperparameter prediction module predicts and optimizes the hyperparameters of each filter according to the information of the input image;

[0008] Designing a spatial convolution attention U-Net network as a semantic segmentation network; the spatial convolution attention U-Net network is realized by introducing a spatial and channel reconstruction convolution module into the Attention U-Net network;

[0009] The adaptive image denoising network and the semantic segmentation network are combined by using a joint optimization strategy to form an IDSNet network, the IDSNet network is optimized by the training loss of the semantic segmentation network, and the synchronization of the image denoising and the crack segmentation of the asphalt pavement cracks is realized.

[0010] Based on the crack segmentation result, crack morphology recognition and parameter calculation are performed.

[0011] Further, the multi-filter denoising module includes an exposure filter, a contrast filter, a tone filter, and a stripe filter, which are respectively used for exposure filtering, contrast filtering, tone filtering, and stripe filtering of the input image.

[0012] Further, the stripe filter uses the average value of each column of pixels in the asphalt pavement crack image to represent the pixel value of each column, and then maps the illumination value of all columns to the desired value of illumination uniformization through column illumination balance operation, as shown in the following formula:

[0013] (1);

[0014] wherein γ is a trainable hyperparameter; is the pixel gray value of the processed image; is the predetermined average gray value of each column after preprocessing; I ij is the gray value of the pixel in the i-th column and j-th row of the k-th asphalt pavement crack image; A i k is the weighted average value of the i-th column of pixels in the k-th asphalt pavement crack image, and its operation formula is shown in formula (2); k is the number of asphalt pavement crack images; M is the number of pixel columns of the asphalt pavement crack image; N is the number of pixel rows of the asphalt pavement crack image;

[0015] (2);

[0016] wherein represents a weighted variable for controlling the weight of the current image, which is also the latest measured illumination information; β represents an empirical constant; is the average pixel value of the i-th column of the k-th asphalt pavement crack image.

[0017] Further, the hyperparameter prediction module is a framework composed of 5 convolutional layers and 2 fully connected layers, which predicts and optimizes the hyperparameters required by the multi-filter denoising module according to the global content of the asphalt pavement crack image, and realizes adaptive denoising of the asphalt pavement crack image.

[0018] Further, the method uses a joint optimization strategy to optimize the trainable hyperparameters of each filter in the multi-filter denoising module, i.e., simultaneously optimizes the image denoising effect and the semantic segmentation effect based on the stochastic gradient descent optimizer and the binary cross-entropy loss function, specifically:

[0019] ​The super parameter prediction module preliminarily predicts the super parameters used by each filter according to the state of the input asphalt pavement crack image, the asphalt pavement crack image after noise reduction by each filter is subjected to semantic segmentation by a semantic segmentation network, and the super parameter prediction module adjusts the super parameters of each filter according to the feedback of the semantic segmentation result, so that a better semantic segmentation asphalt pavement crack binary image with a better noise reduction effect is further obtained.

[0020] Further, the crack morphology includes transverse cracks, longitudinal cracks and cracks; and the crack parameters include crack length, crack width and crack area.

[0021] Further, the specific method for crack morphology identification based on the crack segmentation result is as follows:

[0022] First, the semantic segmentation asphalt pavement crack binary image is subjected to connected domain extraction; and the morphology of the crack is determined by judging the similarity of the trend direction of each crack connected domain with 0°, 90°, 45° and 135°.

[0023] Further, the crack length is obtained by summing the relative distances of a plurality of pixel points on a crack skeleton curve; and the crack skeleton curve is a connected path composed of a series of single pixel points and is distributed along the central axis of the crack target, reflecting the topological structure and morphological characteristics of the crack.

[0024] In a second aspect, the present application provides a computer readable storage medium storing executable instructions, which, when executed, cause a processor to perform the asphalt pavement crack identification method based on the joint optimization network.

[0025] In a third aspect, the present application provides a computer program product comprising a computer program or instructions, which, when executed by a processor, implement the asphalt pavement crack identification method based on the joint optimization network.

[0026] The technical scheme has the following beneficial effects: the asphalt pavement crack identification method based on the joint optimization network provided by the present application has the following advantages: (1) In order to process the real noise in the asphalt pavement crack image, an adaptive image noise reduction network is designed, which comprises a multi-filter noise reduction module and a super parameter prediction module. In the multi-filter noise reduction module, a stripe filter is designed, which can effectively reduce the influence of stripe noise in the pavement image.

[0027] (2) In order to solve the problem of difficult identification of small cracks, a spatial and channel reconstruction convolution module, i.e. SCConv module, is introduced in the semantic segmentation network. The introduction of the module improves the network's ability to capture small cracks and reduces the risk of misidentification.

[0028] (3) To reduce the redundancy of the separate image denoising and crack detection processes and to objectively evaluate the denoising effect on real noise in the image, an IDSNet network integrating denoising and segmentation was proposed using a joint optimization strategy. Under the guidance of the joint optimization strategy, the network was trained with the loss of the crack segmentation result to improve the image denoising effect. At the same time, the denoised image will further improve the crack segmentation effect. Attached Figure Description

[0029] Figure 1 A flowchart of an asphalt pavement crack identification method based on a joint optimization network provided in Embodiment 1 of the present invention;

[0030] Figure 2 This is a partial image data set of the asphalt pavement crack noise reduction image dataset provided in Embodiment 1 of the present invention, wherein (a) is a partial image of asphalt pavement disease, and (b) is the label image corresponding to the image of asphalt pavement disease;

[0031] Figure 3 The image shows the filtering effect of the stripe removal filter provided in Embodiment 1 of the present invention, wherein (a) is the original image of asphalt pavement distress, and (b) is the illumination balance image after stripe removal filtering.

[0032] Figure 4 This is a schematic diagram of the asphalt pavement distress image noise reduction process provided in Embodiment 1 of the present invention;

[0033] Figure 5 This is a schematic diagram of the spatial convolutional attention U-Net network structure provided in Embodiment 1 of the present invention;

[0034] Figure 6 This is a schematic diagram of the SCConv module structure provided in Embodiment 1 of the present invention;

[0035] Figure 7 This is a schematic diagram of the crack length calculation method provided in Embodiment 1 of the present invention;

[0036] Figure 8 This is a crack morphology recognition result image of different asphalt pavement defects provided in Embodiment 1 of the present invention. Detailed Implementation

[0037] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0038] Example 1:

[0039] In this embodiment, an asphalt pavement crack identification method based on a joint optimization network is described, such as... Figure 1 As shown, it includes the following steps:

[0040] Step 1, acquire asphalt pavement disease image, and acquire label image corresponding to the number of images, thereby forming asphalt pavement crack denoising image dataset;

[0041] In this embodiment, more than 10,000 asphalt pavement images are collected by a road detection vehicle equipped with a linear array industrial camera and stored in a database. By screening the images in the database, 5350 asphalt pavement disease images are obtained, and the obtained asphalt pavement disease images are preprocessed by grayscale and size adjustment. At the same time, the cracks in the asphalt pavement images are labeled by artificial labeling, and the label images corresponding to the number of images are obtained, thereby forming an asphalt pavement crack denoising image dataset, which is abbreviated as APCD dataset. Some images in the dataset are shown in Figure 2

[0042] Step 2, design an adaptive image denoising network for adaptive denoising processing of asphalt pavement disease images;

[0043] In this embodiment, an adaptive image denoising network composed of a multi-filter denoising module and a hyperparameter prediction module is designed. The multi-filter denoising module includes exposure, contrast, hue, and stripe removal filters, and dynamically adjusts the filter parameters through a joint optimization strategy to achieve adaptive denoising under different noise conditions. The hyperparameter prediction module predicts and optimizes the hyperparameters of each filter according to the brightness, hue, exposure, and other information of the input image to improve the denoising effect.

[0044] Asphalt pavement images are mainly collected by a road detection vehicle equipped with a linear array industrial camera and a high-power laser illumination device. As shown in Figure 3 (a), due to the change of light conditions during the collection process, the collected pavement images show stripe-shaped exposure differences. This phenomenon in the image is called stripe noise. At the same time, there are a large number of overexposed and underexposed areas in the imaging effect of the pavement image, which seriously affects the accuracy of semantic segmentation results. Therefore, reducing the influence of these exposure differences is crucial for improving the accuracy of semantic segmentation. In order to reduce the influence of stripe noise, the present invention designs a stripe removal filter. As can be observed from Figure 3 (a), the change of image illumination causes the brightness level of different scanning columns to be uneven. However, the illumination of the same column remains relatively consistent. Therefore, the average value of each column of pixels can be used to represent the pixel value of each column, and then the illumination value of all columns is mapped to the desired value of illumination uniformization through a column-wise illumination balancing operation, which can be summarized as shown in the following formula:

[0045] (1);

[0046] wherein, gamma is a trainable hyperparameter; ​is the gray value of the pixel in the i-th row and j-th column of the k-th image; is the predetermined average gray value of each column after preprocessing; is the gray value of the pixel in the i-th row and j-th column of the k-th image; k is the gray value of the pixel in the i-th row and j-th column of the k-th image; i is the gray value of the pixel in the i-th row and j-th column of the k-th image; j is the gray value of the pixel in the i-th row and j-th column of the k-th image; is the weighted average value of the pixel in the i-th column of the k-th image, and its calculation formula is shown in formula (2); k is the number of images; i is the number of image columns; k is the number of image rows; M N

[0047] (2);

[0048] wherein, represents a weighted variable for controlling the weight of the current image, which is also the latest measured illumination information; β represents an empirical constant, which is set to 0.75; is the average pixel value of the i-th column in the k-th asphalt pavement crack image.

[0049] As shown in Figure 3 (b), the asphalt pavement image after the illumination balance operation, from which it can be observed that the overall brightness of the image has a certain level of improvement, and the brightness of the edge area of the image is obviously enhanced, making the crack information better exhibited. At the same time, the stripe noise in the image is effectively controlled, and the expected test target is achieved.

[0050] In addition, the multi-filter denoising module includes multiple differentiable filters, including some of the exposure, gamma, contrast, sharpening, hue, and white balance filters. These filters optimize the image quality by adjusting the exposure, contrast, and other parameters of the image, thereby strengthening the expression of the image to the real situation of the asphalt pavement. The filter will map the input pixel to the output pixel , wherein r , g and b represent the values of the red, green, and blue channels, respectively. Table 1 lists the mapping functions of each filter and their corresponding trainable hyperparameters. The differentiability of the filter is also a prerequisite for the adaptive image denoising network to be trained through backpropagation. With the training of the network, the hyperparameters in the filter will be dynamically adjusted until the image processed by the filter can achieve a relatively optimal image denoising effect.

[0051] Table 1 Filter mapping function;

[0052] ;

[0053] ​​The definition of the contrast filter mapping function is shown in equation (3). The represents a luminance function based on the sensitivity of the human eye to the three primary colors, represents an enhanced luminance function, the definition of which is shown in equations (4) and (5). The in the sharpening filter mapping function is an input image, represents a Gaussian filter, is a positive proportional factor. The sharpening operation process can be differentiated with respect to x and , and the sharpening degree is adjusted by optimizing ;

[0054] (3);

[0055] (4);

[0056] (5);

[0057] In this embodiment, it is determined through ablation test verification that the multi-filter denoising module is composed of four filters, namely, exposure, contrast, tone, and stripe removal (ECTD).

[0058] For complex and variable road surface image conditions, multiple filters need to work together when performing image denoising. Different asphalt pavement images have their unique image characteristics, such as exposure and contrast. Therefore, the hyperparameters of each filter need to be adjusted specifically to achieve the best denoising effect. Traditionally, the adjustment of filter hyperparameters needs to be completed manually by experienced engineers, which not only consumes time and effort, but also is difficult to cope with all possible image scenarios, resulting in low efficiency and high cost. In order to overcome this limitation, the present application introduces a hyperparameter prediction module before the multi-filter denoising module. The hyperparameter prediction module is a framework composed of 5 convolutional layers and 2 fully connected layers, which will predict and optimize the hyperparameters required by the multi-filter denoising module according to the global content of the image, i.e. brightness, tone, exposure, etc., to achieve the purpose of adaptive denoising. At the same time, in order to reduce the computational cost and improve the processing efficiency, the input size of the asphalt pavement image is down-sampled to 256 × 512 resolution. This operation not only greatly reduces the data amount processed by the network, but also preserves enough image information for the hyperparameter prediction module to accurately predict the hyperparameters. The denoising process of the asphalt pavement image is shown in Figure 4 , the image will be input into the hyperparameter prediction module and the multi-filter denoising module respectively, and the two modules will run gradually to complete the denoising task of the image together.

[0059] Step 3, a spatial convolution attention U-Net network is designed as a semantic segmentation network for feature extraction of the asphalt pavement disease image, and then segmentation and positioning of the crack are realized;

[0060] The spatial convolution attention U-Net network enhances the extraction ability of the network for the fine crack features by introducing a spatial and channel reconstruction convolution (SCConv) module in the Attention U-Net network, and reduces the misidentification and unidentified areas.

[0061] The denoised image is input into the spatial convolution attention U-Net network, and the image features are gradually extracted through the encoder-decoder structure of the Attention U-Net network;

[0062] The SCConv module is used to enhance the extraction ability of the fine crack features, and the crack segmentation result is output; the high-precision mapping of the crack position is realized through matrix coding, and the crack positioning map is generated.

[0063] In the embodiment, the spatial convolution attention U-Net network enhances the extraction ability of the network for the fine crack features by introducing a spatial and channel reconstruction convolution (SCConv) module in the Attention U-Net network, and reduces the misidentification and unidentified areas; the SCConv module processes the spatial information and channel information of the image through a spatial reconstruction unit (SRU) and a channel reconstruction unit (CRU) respectively, captures more rich feature information, and improves the feature representation ability.

[0064] The Attention U-Net is an encoder-decoder network; the core idea of the Attention U-Net is to introduce an attention gate mechanism in the decoder, through which the network can suppress the background area irrelevant to the target layer by layer, and improve the accuracy of the network. Although the decoder has been improved from this improvement, the encoder responsible for spatial information compression and crack feature extraction still has a lot of room for improvement. Therefore, the spatial convolution attention U-Net network is proposed, which embeds a spatial and channel reconstruction convolution (SCConv) module in each encoder layer, effectively enhances the extraction ability of the network for the pavement crack features. In addition, compared with the traditional Attention U-Net, the spatial convolution attention U-Net increases the number of network layers to 5 layers, reduces the risk of network degradation, and further improves the representation ability. The structure of the spatial convolution attention U-Net network is shown in Figure 5 .

[0065] SCConv is an efficient convolutional strategy that saves computational cost and storage space. It consists of two units: a Spatial Reconstruction Unit (SRU) and a Channel Reconstruction Unit (CRU). It aims to reduce spatial and channel redundancy in convolutional neural networks and improve feature learning capabilities. The detailed structure of the SCConv module is as follows: Figure 6 As shown.

[0066] The purpose of SRU is to reduce redundancy in the spatial dimension, and it consists of separation and reconstruction operations. Specifically, first, group normalization is used to evaluate the performance of the feature map. Second, the normalized relevance weights are used. W γ ∈ R C This demonstrates the importance of different feature maps. Then, it utilizes... Sigmoid The function, by setting a threshold, will reweight the results. W γ Mapped to the range (0,1). In the experiment, the threshold was set to 0.5, and weights above the threshold were set to 1 to obtain information weights. W 1, then set it to 0 to obtain the non-information weight. W 2. Obtain W The entire process can be represented by equation (6). The reconstruction operation adopts a cross-reconstruction strategy, which enhances information exchange while reducing spatial redundancy. However, spatial redundancy is effectively suppressed by SRU, and redundancy still exists on the channel;

[0067] (6);

[0068] The purpose of CRU is to reduce channel redundancy, and it consists of segmentation, transformation, and fusion operations. First, the input features are divided into two branches based on the number of channels. X up Include αC aisle, X low The branches contain (1- α ) C Channel. During the conversion phase, for X up Grouped convolution (GWC) and pointwise convolution (PWC) are used, which reduces computational cost while effectively extracting and fusing features, and forms a representative feature map through element-wise summation. Y 1. At the same time, X low By repeatedly applying pointwise convolution to extract hidden information, more feature maps are generated. These feature maps are then concatenated to form another representative feature map. Y2. During the fusion phase, global average pooling is used to collect global spatial information, including calculating the global average value for each channel. S m = Pooling ( Y m This achieves the goal of reflecting global information for each channel. Simultaneously, a soft attention mechanism is used to process channel statistics. S 1 and S 2. Integrate into feature vectors β 1 and β 2. Finally, by weighting the upper-layer features Y 1 and weighted lower-level features Y 2. Combining and refining the output channel features Y , obtain Y The entire process can be represented by formula (7):

[0069] (7);

[0070] in, It is the learnable weight matrix of GWC. and It is the learnable weight matrix of PwC. It is a connection operation.

[0071] Spatial and channel reconstruction convolution operations can effectively reduce the computation of irrelevant features by the network, significantly improving the accuracy and efficiency of crack segmentation. The core of this approach is to establish feature attention focusing in the spatial dimension, reducing redundant computation through compression and recombination techniques; and to implement dynamic feature selection in the channel dimension, using adaptive weighting to strengthen target features. Spatial and channel reconstruction convolutions enhance the network model's ability to extract key crack regions, suppressing background noise interference while improving the capture of subtle features, thus improving both computational efficiency and model robustness.

[0072] On the APCD dataset, the introduction of the SCConv module further improved the results of crack semantic segmentation, with mIoU and F1-Score increasing by 0.50% and 0.93%, respectively.

[0073] Step 4: The adaptive image denoising network and the semantic segmentation network are combined using a joint optimization strategy to form the IDSNet network. The training loss of the semantic segmentation network is used to optimize the IDSNet network, so that image denoising and crack segmentation can be performed simultaneously. Through joint optimization, the quality of the denoised image is improved, which further enhances the accuracy of crack segmentation.

[0074] The semantic segmentation network and the adaptive image denoising network are combined in a cascading manner to form an end-to-end network, and the overall IDSNet network is optimized according to the loss value of the semantic segmentation network;

[0075] The trainable hyperparameters of each filter in the multi-filter denoising module are optimized by using a joint optimization strategy, which optimizes the image denoising effect and the semantic segmentation effect based on the stochastic gradient descent (SGD) optimizer and the binary cross-entropy (BCE) loss function; specifically, the hyperparameter prediction module preliminarily predicts the hyperparameters used by each filter according to the state of the input asphalt pavement crack image, the asphalt pavement crack image denoised by each filter is subjected to semantic segmentation by the semantic segmentation network, and the hyperparameter prediction module adjusts the hyperparameters of each filter according to the feedback of the semantic segmentation result, thereby further obtaining an asphalt pavement crack image with better denoising effect;

[0076] The test results show that the reference of the adaptive image denoising network enables the crack semantic segmentation to achieve an mIoU of 54.60% and an F1-Score of 59.93%, which are 6.38% and 5.58% higher than those of the image without denoising. The IDSNet achieves the best crack segmentation effect on the APCD dataset, with an mIoU of 55.10% and an F1-Score of 60.86%, which are 6.88% and 6.51% higher than those of the traditional Attention U-Net.

[0077] Step 5: Crack morphology identification and parameter calculation based on crack segmentation results;

[0078] Based on the crack segmentation results, crack morphology identification (such as transverse cracks, longitudinal cracks, and cracks) and geometric parameter calculation (such as crack length, width, and area) are performed. According to the relevant provisions in the "Highway Technical Condition Evaluation Standard", the crack damage data is statistically analyzed to provide a scientific basis for the development of road maintenance strategies.

[0079] In the process of calculating the crack damage rate, the length, width, and area-related geometric parameters of the crack need to be extracted. For transverse cracks and longitudinal cracks, their influence width is calculated as 0.2m. Therefore, in actual engineering, only the length needs to be calculated, and the calculation of the crack length depends on the extraction and analysis of the crack skeleton curve. The skeleton curve is a continuous path composed of a series of single pixel points and distributed along the center axis of the crack target, which can accurately reflect the topological structure and morphological characteristics of the crack. By extracting these key pixel points, the center line of the asphalt pavement crack can be constructed, thereby revealing the spatial distribution law of the crack. Figure 7The skeleton curve of the local crack is shown in the figure, and the crack pixels on the skeleton curve are represented by black squares. When the relative positions of the crack pixels on the skeleton are A and B points as shown in the figure, the coordinates of the A point are assumed to be (0, 0) and the coordinates of the B point are assumed to be (1, 1), and the relative distance between the two points is x 0 ,y 0 x 1 ,y 1 d AB The relative distance can be represented by formula (8):

[0080] (8);

[0081] Therefore, the length of the entire crack can be obtained by summing the relative distances of multiple pixel points. Compared with the calculation of the crack length, the calculation method of the crack area is relatively more direct. The crack in the image is fundamentally composed of pixel points, and therefore, the area of the crack can be preliminarily determined by locating the area where the crack is located and counting the number of pixels in the area. However, the crack area obtained in this step is the total number of pixel points in the image, and to obtain the real area of the crack, the total number of pixels needs to be multiplied by the actual physical size corresponding to a single pixel.

[0082] The asphalt pavement cracks can be divided into transverse cracks, longitudinal cracks, cracks and block cracks, and different types of cracks have different proportions in the calculation of the crack damage rate. Therefore, the cracks need to be classified when calculating the damage rate. In practical applications, the distinction between transverse cracks and longitudinal cracks is relatively easy, while the distinction between cracks and block cracks is relatively difficult to achieve. In the classification of crack types in this embodiment, the cracks are only divided into transverse cracks, longitudinal cracks and cracks.

[0083] First, the connected domain extraction is performed on the binary image of the asphalt pavement crack after semantic segmentation; the shape of the crack is determined by judging the similarity of the trend direction of each crack connected domain to the four directions of 0°, 90°, 45° and 135°, and the recognition result is as shown in Figure 8 .

[0084] In order to verify the morphological recognition method, this embodiment uses the gray level co-occurrence matrix method to recognize the crack shape and count the results using the continuous 500 images in the APCD data set. At the same time, the crack categories of the same batch of images are manually recognized, and the error of the algorithm recognition is calculated taking the manual recognition result as the standard. As shown in Table 2, the crack shape recognition method proposed in this paper can achieve an accuracy of more than 92%, which has certain engineering significance.

[0085] Table 2 Crack category recognition statistics

[0086] .

[0087] Example 2:

[0088] The embodiment provides an electronic device, comprising one or more processors, and a memory for storing instructions which, when executed by the one or more processors, cause the one or more processors to perform the joint optimization network-based asphalt pavement crack identification method.

[0089] The electronic device can be a mobile phone, a computer, a tablet computer or the like, comprising a memory and a processor, and the memory stores a computer program which, when executed by the processor, implements the joint optimization network-based asphalt pavement crack identification method as described in the embodiments. It can be understood that the electronic device can further comprise an input / output (I / O) interface and a communication component.

[0090] The processor is configured to perform all or part of the steps of the joint optimization network-based asphalt pavement crack identification method as described in the above embodiments. The memory is configured to store various types of data, which can include, for example, instructions of any application program or method in the electronic device, and application program related data.

[0091] The processor can be an Application Specific Integrated Cricuit (ASIC), a Digital Signal Processor (DSP), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and is configured to perform the joint optimization network-based asphalt pavement crack identification method as described in the above embodiments.

[0092] Embodiment 3:

[0093] The embodiment provides a computer readable storage medium storing executable instructions which, when executed, can be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as an independent product.

[0094] The computer software product is stored in a storage medium and comprises a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the joint optimization network-based asphalt pavement crack identification method described in the embodiments of the present application.

[0095] The aforementioned storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD (Secure Digital Memory Card) or a DX (an abbreviation of Memory Data Register, MDR) memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an APP (an abbreviation of Application) application store, and the like, which can store a program check code, and stores a computer program thereon, the computer program being executable by a processor to implement each step of the asphalt pavement crack identification method based on a joint optimization network.

[0096] Embodiment 4:

[0097] The embodiment provides a computer program product, including a computer program or instructions, which are executable by a processor to implement the asphalt pavement crack identification method based on a joint optimization network.

[0098] Based on such an understanding, the technical solution of the present application, in essence, or the part of the technical solution that contributes to the prior art, or the part of the technical solution can be embodied in the form of a computer program product.

[0099] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the difference from other embodiments.

[0100] The scope of protection of the present application is not limited to the above-mentioned embodiments. Obviously, those skilled in the art can make various modifications and changes to the present disclosure without departing from the scope and spirit of the present disclosure. If these modifications and changes belong to the scope of the present application and its equivalent technology, the present application also includes these modifications and changes.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the present application.

Claims

1. A method for asphalt pavement crack identification based on a joint optimization network, characterized in that, The method comprises the following steps: acquiring asphalt pavement disease images and label images corresponding to the number of images, and constructing an asphalt pavement crack image dataset; designing an adaptive image denoising network; the adaptive image denoising network comprises a multi-filter denoising module and a hyperparameter prediction module; the multi-filter denoising module comprises multiple filters, and the parameters of the filters are dynamically adjusted through a joint optimization strategy to realize adaptive denoising under different noise conditions; the hyperparameter prediction module predicts and optimizes the hyperparameters of each filter according to the information of the input image; a spatial convolution attention U-Net network is designed as a semantic segmentation network; the spatial convolution attention U-Net network is realized by introducing a spatial and channel reconstruction convolution module into the Attention U-Net network; the adaptive image denoising network and the semantic segmentation network are combined by using a joint optimization strategy to form an IDSNet network, the IDSNet network is optimized by using the training loss of the semantic segmentation network, and the synchronization of image denoising and crack segmentation of the asphalt pavement crack image is realized; crack morphology recognition and parameter calculation are performed based on the crack segmentation result; the method optimizes the trainable hyperparameters of each filter in the multi-filter denoising module by using a joint optimization strategy, that is, the image denoising effect and the semantic segmentation effect are simultaneously optimized based on a stochastic gradient descent optimizer and a binary cross-entropy loss function, and the specific method is as follows: the hyperparameter prediction module preliminarily predicts the hyperparameters used by each filter according to the state of the input asphalt pavement crack image, the asphalt pavement crack image denoised by each filter is subjected to semantic segmentation by using the semantic segmentation network, and the hyperparameter prediction module adjusts the hyperparameters of each filter according to the feedback of the semantic segmentation result, so that a better semantic segmentation asphalt pavement crack binary image with better denoising effect is obtained.

2. The asphalt pavement crack identification method based on joint optimization network according to claim 1, characterized in that, the multi-filter denoising module comprises an exposure filter, a contrast filter, a tone filter and a stripe filter, which are respectively used for exposure filtering, contrast filtering, tone filtering and stripe filtering of the input image.

3. The asphalt pavement crack identification method based on joint optimization network according to claim 2, characterized in that, the stripe filter uses the average value of each column of pixels of the asphalt pavement crack image to represent the pixel value of each column, and then maps the illumination values of all columns to the expected value of illumination uniformization through a column-wise illumination balance operation, as shown in the following formula: ， （1）； wherein γ is a trainable hyper-parameter; is the pixel gray value of the processed image; is the predetermined average gray value of each column after preprocessing; ij is the gray value of the pixel in the i-th column and j-th row of the k-th asphalt pavement crack image; i k is the weighted average value of the i-th column pixel in the k-th asphalt pavement crack image, and its operation formula is shown in formula (2); k is the number of asphalt pavement crack images; M is the number of pixel columns of the asphalt pavement crack image; and N is the number of pixel rows of the asphalt pavement crack image. （2）； wherein, represents a weighting variable for controlling the weighting variable of the current image weight, which is also the latest measured illumination information; β represents an empirical constant; a i k is the average pixel value of the i-th column in the k-th asphalt pavement crack image.

4. The asphalt pavement crack identification method based on joint optimization network according to claim 1, characterized in that, the hyperparameter prediction module is a framework composed of 5 convolution layers and 2 fully connected layers, which predicts and optimizes the hyperparameters required by the multi-filter denoising module according to the global content of the asphalt pavement crack image, so as to realize adaptive denoising of the asphalt pavement crack image.

5. The asphalt pavement crack identification method based on joint optimization network according to claim 1, characterized in that, the crack morphology includes transverse cracks, longitudinal cracks and cracks; the crack parameters include crack length, crack width and crack area.

6. The method of claim 5, wherein the method is based on a joint optimization network. the specific method for recognizing the crack morphology based on the crack segmentation result is as follows: first, the asphalt pavement crack binary image after semantic segmentation is subjected to connected domain extraction; the morphology of the crack is judged by judging the similarity of the trend direction of each crack connected domain with 0°, 90°, 45° and 135°.

7. The method of claim 6, wherein the method is based on a joint optimization network. The crack length is obtained by summing relative distances of a plurality of pixel points on a crack skeleton curve; the crack skeleton curve is a series of single pixel points connected together and distributed along a central axis of the crack target, reflecting a topological structure and morphological characteristics of the crack. 8.A computer readable storage medium for performing the method of claim 1-7, wherein, The storage has executable instructions which, when executed, cause the processor to perform the joint optimization network-based asphalt pavement crack identification method.

9. A computer program product for performing a joint optimization network based asphalt pavement crack detection method according to any one of claims 1-7, characterized in that, The computer program or instructions are executed by the processor to implement the joint optimization network-based asphalt pavement crack identification method.

Images