Road pavement disease intelligent identification method and system based on multispectral image

Abstract

The invention relates to the technical field of road pavement disease recognition, in particular to an intelligent road pavement disease recognition method and system based on a multispectral image, and the method comprises the steps: obtaining road pavement image data collected by a multispectral camera carried by an unmanned aerial vehicle; generating a registered multispectral fusion image; performing enhancement processing on the multispectral fusion image based on a dark channel algorithm to generate an enhanced image; performing shallow feature extraction and deep feature extraction on the enhanced image according to a convolutional neural network to generate an initial disease extraction image; performing segmentation processing on the initial disease extraction image through adaptive threshold segmentation to generate a binary image; performing morphological processing on the binary image to generate a candidate disease image; extracting edge information of a disease area from the candidate disease image based on an instance segmentation technology, and generating a segmented image; the segmented image is subjected to crack refining processing, a crack refined image is generated, and more comprehensive disease information support is provided for collection and fusion of multispectral data.

Description

Technical Field

[0001] The present invention relates to the technical field of road surface disease identification, and more specifically, to an intelligent road surface disease identification method and system based on multi-spectral images. Background Art

[0002] With the rapid development of modern transportation infrastructure, the maintenance and management of road surfaces have become particularly important. However, over time and with the increase in usage frequency, road surface diseases (such as cracks, potholes, etc.) have gradually become important issues affecting the service life of roads and driving safety. Accurately and efficiently identifying and evaluating road surface diseases has become the core requirement of highway maintenance management.

[0003] Currently, traditional road disease detection methods mainly rely on manual inspections or detection means based on single-spectral images. These methods have significant limitations. Although manual inspections can directly obtain the conditions of road diseases, due to relying on human labor, the detection efficiency is low, the subjectivity is strong, and it is difficult to cover large sections of the road. At the same time, manual detection usually can only observe surface features with the naked eye, and the ability to identify minor cracks or potential diseases is limited. On the other hand, the technical means of single-spectral images (such as visible light images) are prone to misjudging disease characteristics in complex backgrounds due to the lack of multi-dimensional information support. For example, in the case of strong light or significant shadow effects, cracks may be masked or misjudged as normal road textures. In addition, single-spectral images cannot provide physical attribute information of diseases, such as temperature characteristics or radiation characteristics, and thus cannot scientifically evaluate the evolution trend and severity of diseases.

[0004] In recent years, detection methods based on multi-spectral images have begun to receive attention. Multi-spectral images provide more comprehensive data support for disease detection by integrating different spectral information such as visible light and infrared bands. However, the existing detection methods based on multi-spectral images still have technical bottlenecks: First, the image registration and fusion process is usually limited by the registration accuracy and algorithm complexity, and it is difficult to meet the requirements of rapid processing; second, the data enhancement means of multi-spectral images are mostly simple filtering processes, and the contrast improvement of disease areas in complex scenes is limited; third, the existing feature extraction methods lack the ability to hierarchically analyze diseases and are difficult to effectively distinguish shallow texture features from deep geometric structure features; in addition, the segmentation methods mostly use fixed threshold segmentation or single morphological processing and are easily affected by uneven gray distribution, resulting in unstable segmentation results; finally, the crack refinement and disease evaluation methods lack comprehensive analysis of multi-dimensional parameters such as temperature and geometry and cannot accurately predict the dynamic development trend of diseases. Summary of the Invention

[0005] In view of the problems in the above-mentioned prior art, the present invention proposes an intelligent recognition method for road surface diseases based on multispectral images. Through a series of innovative technologies, the limitations of traditional methods are overcome, and the accuracy, efficiency, and practicality of detection are significantly improved. The method of the present invention optimizes the links of image acquisition, processing, feature extraction, segmentation, and evaluation, forming a complete technical chain. The technical problems solved by the present invention can be summarized as follows:

[0006] The problem of insufficient information in a single spectral image for disease recognition is solved, and the expression ability of disease features is enhanced through multispectral fusion;

[0007] The problem that traditional enhancement methods are inaccurate in extracting disease regions in complex backgrounds is overcome, and the disease contrast is improved through the dark channel algorithm;

[0008] The problem that traditional fixed-threshold segmentation has poor adaptability to uneven gray-scale distribution is solved, and the robustness of the segmentation effect is improved through adaptive threshold segmentation;

[0009] The problem that existing thinning methods have insufficient accuracy in crack skeletons is overcome, and the crack morphology is optimized through the minimum spanning tree algorithm and geometric fitting;

[0010] The problem of the lack of dynamic analysis in traditional evaluation methods is solved, and the accuracy and timeliness of disease evaluation are enhanced through the fusion of geometric parameters and temperature characteristics.

[0011] The present invention provides an intelligent recognition method for road surface diseases based on multispectral images, including:

[0012] Acquisition step:

[0013] Obtain the road surface image data collected by a drone equipped with a multispectral camera, wherein the image data includes visible light images and infrared thermal imaging images.

[0014] Processing step:

[0015] Based on the image data, perform the following operations:

[0016] Perform registration processing on the visible light image and the infrared thermal imaging image to generate a registered multispectral fusion image;

[0017] Perform enhancement processing on the multispectral fusion image based on the dark channel algorithm to generate an enhanced image;

[0018] Perform shallow feature extraction and deep feature extraction on the enhanced image according to a convolutional neural network to generate an initial disease extraction image;

[0019] Perform segmentation processing on the initial disease extraction image through adaptive threshold segmentation to generate a binary image;

[0020] Perform morphological processing on the binary image to generate a candidate disease image;

[0021] Extract the edge information of the disease area from the candidate disease image based on instance segmentation technology to generate a segmentation image;

[0022] Perform crack refinement processing on the segmentation image to generate a crack-refined image.

[0023] Output step:

[0024] Output the crack-refined image, and further extract disease parameters, including the angle, length, and area of the crack, for disease assessment.

[0025] Preferably, the acquisition step specifically includes:

[0026] Collect road pavement images by setting a predetermined flight path with a drone;

[0027] The multispectral camera includes a visible light camera and an infrared thermal imaging camera. Among them, the visible light camera collects images in the spectral range of 400 - 700nm, and the infrared thermal imaging camera collects images in the band of 8 - 14μm.

[0028] Preferably, the registration processing in the processing step specifically includes:

[0029] Detect feature points in the visible light image and the infrared thermal imaging image;

[0030] Perform registration on the images using affine transformation to generate a spatially aligned fused image.

[0031] Preferably, the enhancement processing specifically includes:

[0032] Calculate the dark channel value of the fused image based on the dark channel algorithm;

[0033] Perform normalization processing on the dark channel value to generate an enhanced image.

[0034] Preferably, the shallow feature extraction and deep feature extraction specifically include:

[0035] Use the first convolutional layer to extract the texture features of the enhanced image;

[0036] Use the second convolutional layer to extract the spatial features of the enhanced image;

[0037] Perform pixel-level fusion on the outputs of the first convolutional layer and the second convolutional layer to generate an initial disease extraction image.

[0038] Preferably, the adaptive threshold segmentation specifically includes:

[0039] Set an initial segmentation threshold;

[0040] Based on the gray - scale means of the foreground region and the background region, iteratively adjust the segmentation threshold until the segmentation result converges.

[0041] Preferably, the crack refinement process specifically includes:

[0042] Connect the pixel points of the segmented image based on the minimum spanning tree algorithm to generate a crack skeleton;

[0043] According to the crack skeleton, geometrically fit the crack edges to optimize the crack morphology.

[0044] Preferably, it further includes:

[0045] Normalize the angle, length, and area in the disease parameters;

[0046] Calculate the severity level of the disease based on the normalized parameters.

[0047] Preferably, the disease assessment further includes:

[0048] By analyzing the temperature difference between the crack area and the normal road surface area, and combining geometric features, evaluate the dynamic change trend of the disease.

[0049] An intelligent road surface disease recognition system based on multi - spectral images based on the above - mentioned method, includes:

[0050] A data acquisition module, used to obtain visible - light images and infrared thermal imaging images collected by a multi - spectral camera carried by a drone;

[0051] A data processing module, used to register the visible - light images and infrared thermal imaging images to generate a fused image, and enhance the fused image based on the dark channel algorithm;

[0052] A disease extraction module, used to extract shallow - layer features and deep - layer features from the enhanced image by using a convolutional neural network to generate an initial disease extraction image;

[0053] A segmentation module, used to segment the initial disease extraction image by adaptive threshold segmentation to generate a binary image, and perform morphological processing on the binary image to generate a candidate disease image;

[0054] A refinement module, used to extract disease edge information based on instance segmentation technology and refine the cracks in the disease area;

[0055] An evaluation module, used to extract disease parameters, including the angle, length, and area of cracks, and evaluate the severity level of the disease in combination with temperature characteristics;

[0056] An output module for outputting the disease location, parameter information, and evaluation results.

[0057] First, the present invention uses a drone to carry a multispectral camera to obtain road image data with a wide coverage range and rich spectral information, significantly improving the observability of disease information. By registering the visible light image and the infrared thermal imaging image, a high-precision multispectral fusion image is generated, laying a foundation for subsequent data analysis.

[0058] Secondly, the present invention uses the dark channel algorithm to enhance the fusion image, effectively improving the contrast of the disease area under complex backgrounds. Further, the present invention combines the feature extraction technology of convolutional neural networks. Through the collaborative analysis of shallow and deep features, both texture features and spatial structure features are taken into account, realizing the preliminary extraction of the disease area.

[0059] The method of the present invention introduces an adaptive threshold segmentation algorithm in the segmentation link, effectively overcoming the problem of poor adaptability of the fixed threshold method to complex gray distributions, and combining morphological processing to eliminate noise interference, ensuring the integrity of the candidate disease images. Subsequently, based on the instance segmentation technology, the method accurately extracts the edge information of the disease area. The crack refinement link optimizes the skeleton generation through the minimum spanning tree algorithm, and further adjusts the crack boundary by the geometric fitting method, significantly improving the accuracy and consistency of crack refinement.

[0060] In terms of disease assessment, the innovation of the present invention lies in the comprehensive analysis of disease parameters and temperature characteristics. By extracting geometric features such as the angle, length, and area of the disease, and dynamically modeling the temperature difference between the crack area and the normal road surface area, the method can accurately evaluate the severity and evolution trend of the disease. This comprehensive analysis not only improves the scientific nature of the assessment but also provides a quantitative basis for road maintenance decision-making.

[0061] The beneficial effects of the present invention are mainly reflected in:

[0062] The acquisition and fusion of multispectral data provide more comprehensive disease information support;

[0063] The enhancement processing based on the dark channel algorithm significantly improves the saliency of disease features;

[0064] The synergistic effect of shallow and deep feature extraction realizes the accurate identification of the disease area;

[0065] The steps of segmentation and refinement complement each other, ensuring the integrity of the disease area and the accuracy of the boundary;

[0066] The evaluation method that fuses geometric parameters and temperature characteristics provides a scientific basis for predicting the dynamic changes of diseases;

[0067] The efficient coordination of the entire technical chain improves the overall performance of detection and evaluation, and has important practical value and promotion prospects.

[0068] In summary, the method of the present invention solves many problems in the prior art through the innovation of technical solutions and the coordination between steps, and realizes the efficient and intelligent identification of road surface diseases. Brief Description of the Drawings

[0069] Figure 1 It is the system logic block diagram of the present invention.

[0070] Figure 2 It is the data acquisition and processing logic block diagram of the present invention.

[0071] Figure 3 It is the disease extraction and segmentation logic block diagram of the present invention.

[0072] Figure 4 It is the crack refinement and disease evaluation logic block diagram of the present invention. Detailed Description of the Invention

[0073] Please refer to Figures 1-4 , the present invention provides an intelligent identification method for road surface diseases based on multi-spectral images, including: obtaining road surface image data collected by a multi-spectral camera carried by a drone, wherein the image data includes visible light images and infrared thermal imaging images. Based on the image data, the following operations are performed: registering the visible light image and the infrared thermal imaging image to generate a registered multi-spectral fusion image; enhancing the multi-spectral fusion image based on the dark channel algorithm to generate an enhanced image; extracting shallow features and deep features from the enhanced image according to a convolutional neural network to generate an initial disease extraction image; segmenting the initial disease extraction image by adaptive threshold segmentation to generate a binary image; performing morphological processing on the binary image to generate a candidate disease image; extracting the edge information of the disease area from the candidate disease image based on instance segmentation technology to generate a segmented image; performing crack refinement processing on the segmented image to generate a crack refinement image. Output the crack refinement image, and further extract disease parameters, including the angle, length, and area of the crack, for disease evaluation.

[0074] In the method of the present invention, a multi-spectral camera carried by a drone collects road surface images along a predetermined flight path. Preferably, the spectral range of the visible light image is 400-700nm, and the band of the infrared thermal imaging image is 8-14μm. This acquisition method can simultaneously obtain the geometric information and thermal feature information of the disease area, significantly improving the multi-dimensional expression ability of the data. For example, in high-temperature weather, the thermal radiation in the crack area is more significant, and the recognition of cracks can be enhanced through infrared images.

[0075] In the above processing step, the registration process completes the spatial alignment of the visible light image and the infrared image based on feature point detection and affine transformation. Preferably, the feature point detection algorithm uses the SURF (Speeded Up Robust Features) algorithm to ensure the registration accuracy.

[0076] After the multi-spectral fusion image is enhanced by the dark channel, the contrast between the disease area and the background is improved. The shallow features extracted by the convolutional neural network mainly include the texture information of the crack edge, while the deep features can capture the spatial structure of the crack.

[0077] After the method of the present invention completes the above processing, the angle, length and area of the crack are extracted by the Hough transform to generate a parameterized result for disease assessment. For example, the angle of the crack can be calculated by the following formula:

[0078]

[0079] where θ is the inclination angle of the crack, and Δx and Δy are the horizontal and vertical differences between the two endpoints of the crack respectively.

[0080] In an embodiment of the present invention, the obtaining step specifically includes: collecting road surface images by setting a predetermined flight path with a drone;

[0081] The multi-spectral camera includes a visible light camera and an infrared thermal imaging camera. Among them, the visible light camera collects images in the spectral range of 400 - 700 nm, and the infrared thermal imaging camera collects images in the band of 8 - 14 μm.

[0082] The method of the present invention preferably adopts the drone automatic flight path planning technology to generate a flight path covering the target area based on the GPS positioning information to ensure the integrity of the image data. Preferably, the flight path interval is set to 10 meters and the flight altitude is 50 meters to balance the image resolution and the coverage area.

[0083] Preferably, the multi-spectral camera adopted by the present invention has a dual-channel function and can synchronously obtain visible light and infrared thermal imaging data. The visible light camera is used to extract the texture information of the road surface, while the infrared thermal imaging camera captures the thermal radiation characteristics of the road. This configuration significantly improves the accuracy of disease detection. For example, infrared thermal imaging can detect potential diseases invisible to the human eye, such as thermal anomalies in small crack areas.

[0084] In an embodiment of the present invention, the registration process in the processing step specifically includes: performing feature point detection on the visible light image and the infrared thermal imaging image; performing registration on the images by using affine transformation to generate a spatially aligned fusion image.

[0085] In the method of the present invention, the SURF algorithm is preferably used to detect the feature points of the visible light image and the infrared image to ensure the robustness of registration. The quality of feature point matching can be measured by the following formula:

[0086]

[0087] where Q is the matching rate, N m atch is the number of successfully matched feature points, and N t otal is the total number of feature points.

[0088] The mathematical model of affine transformation is expressed as:

[0089]

[0090] where x′, y′ are the pixel coordinates after transformation, x, y are the original pixel coordinates, a, b, c, d are the rotation and scaling parameters, and tx, ty are the translation parameters. The transformation parameters are optimized by minimizing the registration error to generate a high-precision spatially aligned fused image.

[0091] After the registration process, the fused image realizes the unified representation of visible light and infrared features at the pixel level, laying a data foundation for subsequent disease detection. Experiments show that the registration error is preferably less than 1 pixel.

[0092] In an embodiment of the present invention, the enhancement process specifically includes: calculating the dark channel value of the fused image based on the dark channel algorithm; performing normalization processing on the dark channel value to generate an enhanced image.

[0093] In the method of the present invention, the enhancement process is a key step in improving the quality of the fused image. Preferably, the dark channel algorithm is used to enhance the contrast of the disease area, making the subsequent feature extraction process more accurate. Specifically, the calculation of the dark channel value is based on the following formula:

[0094]

[0095] where D(x) is the dark channel value of pixel x, Ω(x) represents the window area centered on pixel x, and I c (y) is the intensity value of pixel y on channel v. Preferably, the window size is set to 15×15 to balance enhancing local details and suppressing global noise.

[0096] The normalization process performs linear stretching on the dark channel value to map the pixel values of the image to a unified range (such as 0 to 1). The normalized enhanced image can highlight the texture features and intensity differences of the disease area. For example, the dark channel value of the crack area is usually significantly lower than that of the normal road surface area, and the enhanced image shows the crack as a more obvious dark band, thus improving the detection accuracy.

[0097] Preferably, the image contrast enhancement rate after dark channel enhancement and normalization processing reaches more than 30%, making the texture features more prominent.

[0098] In an embodiment of the present invention, the shallow feature extraction and deep feature extraction specifically include: using a first convolutional layer to extract the texture features of the enhanced image; using a second convolutional layer to extract the spatial features of the enhanced image; performing pixel-level fusion on the outputs of the first convolutional layer and the second convolutional layer to generate an initial disease extraction image.

[0099] In an embodiment of the present invention, a convolutional neural network is used for feature extraction of enhanced images, where shallow features focus on the extraction of texture information, while deep features can capture the spatial geometric features of diseases. Preferably, the parameter configurations of the first convolutional layer and the second convolutional layer are as follows:

[0100] The convolutional kernel size of the first convolutional layer is 7x7; the number of convolutional kernels is 64; the stride is 1; the activation function is ReLU (Rectified Linear Unit).

[0101] The convolutional kernel size of the second convolutional layer is 5x5; the number of convolutional kernels is 128; the stride is 1; the activation function is ReLU.

[0102] The mathematical description of shallow feature extraction is:

[0103]

[0104] Among them, F 1 (x, y) is the output feature value of the first convolutional layer, w ij is the convolutional kernel weight, b is the bias value, I(x + i, y + j) is the pixel value of the input image, and N, M are the sizes of the convolutional kernel.

[0105] The mathematical description of deep feature extraction is similar, but both the convolutional kernel size and the number increase to capture more complex spatial relationships.

[0106] The pixel-level fusion of shallow and deep features is completed through pixel-by-pixel weighted operations, and the fusion formula is:

[0107] F fusion (x, y) = αF 1 (x, y) + βF 2 (x, y)

[0108] Among them, α and β are weight coefficients, and preferably, they are set to 0.5 to ensure the balance of the contributions of shallow and deep features to the fused image.

[0109] In one embodiment of the present invention, the adaptive threshold segmentation specifically includes: setting an initial segmentation threshold; iteratively adjusting the segmentation threshold based on the gray-scale means of the foreground region and the background region until the segmentation result converges.

[0110] The adaptive threshold segmentation plays a role in separating the disease area from the background in the method of the present invention. Preferably, the initial segmentation threshold is determined by the Otsu algorithm, and the calculation formula is:

[0111]

[0112] where t 0 is the initial threshold, is the between-class variance between the foreground and the background. During the iteration process, the gray-scale means of the foreground and the background are calculated each time:

[0113]

[0114] where μ f is the gray-scale mean of the foreground, μ b is the gray-scale mean of the background, I f (i) and I b (i) represent the foreground and background pixel values respectively, N f and N b are the numbers of pixels in the foreground and the background. The new segmentation threshold is calculated by the following formula:

[0115]

[0116] The iteration termination condition is that the difference between two segmentation thresholds is less than a preset threshold, preferably set to 0.1.

[0117] Experiments show that the adaptive threshold segmentation can effectively cope with the uneven gray-scale distribution in the road surface image, and the segmentation accuracy of the disease area reaches more than 95%.

[0118] In one embodiment of the present invention, the crack refinement processing specifically includes: connecting the pixel points of the segmented image based on the minimum spanning tree algorithm to generate a crack skeleton; geometrically fitting the crack edges according to the crack skeleton to optimize the crack morphology.

[0119] In the method of the present invention, during the crack refinement processing, the pixel points of the disease area are connected into a continuous skeleton structure by the minimum spanning tree algorithm. The construction of the minimum spanning tree is based on the minimization of the weights of the adjacency matrix, and the calculation formula of the weights is:

[0120]

[0121] y = ax + b

[0122] Among them, w(p,q) is the weight between pixel points p and q, and (x p , y p ) and (x q , y q ) are the coordinates of two points. After the crack skeleton is formed, geometric fitting is performed on the edge points based on linear regression, and the fitting model is: Among them, a is the slope and b is the intercept. During the fitting process, the maximum residual value is used as the standard for eliminating outliers to optimize the crack morphology.

[0123] Preferably, the refined crack skeleton can accurately reflect the actual morphology of the crack, and the boundary fitting accuracy reaches the sub-pixel level, significantly improving the reliability of subsequent parametric analysis.

[0124] In an embodiment of the present invention, it further includes: normalizing the angle, length, and area in the disease parameters; calculating the severity level of the disease based on the normalized parameters.

[0125] In the method of the present invention, in order to further realize the quantitative analysis of the disease severity, preferably, the disease parameters are normalized to eliminate the interference caused by different image resolutions, acquisition angles, or environmental lighting conditions. The normalization formula is:

[0126]

[0127] Among them, P is the original parameter value, P min and P max are the minimum and maximum values of the parameter respectively, and P ′ is the value after normalization. The normalized disease parameters include the angle, length, and area of the crack. For example, the normalized value of the crack length can directly reflect the change range of the cabinet and avoid the deviation caused by resolution differences. Based on the normalized parameters, preferably, the following formula is used to calculate the disease severity level:

[0128] S = w 1 P angle ′ + w 2 P length ′ + w 3 P area ′

[0129] Among them, S is the disease severity level, P ′ angle , P ′ length , P ′ area are the normalized values of the angle, length, and area respectively, and w 1 , w 2 , w 3is the weight coefficient. Preferably, the weight value is adjusted according to the actual road type and maintenance requirements. For example, for highways, the weights of length and area are higher.

[0130] Based on the severity level calculated by this method, cracks can be classified into three levels: minor (S < 0.3), moderate (0.3 ≤ S < 0.7), and severe (S ≥ 0.7), providing a decision-making basis for subsequent road maintenance.

[0131] In an embodiment of the present invention, the disease assessment further includes: evaluating the dynamic change trend of the disease by analyzing the temperature difference between the crack area and the normal road surface area and combining geometric features.

[0132] In the method of the present invention, the temperature characteristics of the crack area are closely related to the dynamic changes of the disease. Preferably, by analyzing the temperature distribution in the multi-spectral image, the temperature difference between the crack area and the normal road surface area is quantified. The calculation formula for the temperature difference is:

[0133] ΔT = T crack - T normal

[0134] where ΔT is the temperature difference, T crack is the average temperature of the crack area, and T normal is the average temperature of the normal road surface area. Preferably, the temperature change trend of the crack area can be further modeled by the following formula:

[0135] T(t) = T 0 + α·t

[0136] where T(t) is the temperature of the crack area at time t, T 0 is the initial temperature, and α is the temperature change rate. This model can reflect the expansion and deterioration process of the crack over time.

[0137] Combined with the geometric features of the crack (such as length and area), the method of the present invention preferably constructs a dynamic change trend model of the disease to predict the possible future expansion path of the crack. For example, for a crack area with a temperature difference greater than 5°C, combined with the length change rate, the prediction result shows that the crack expansion rate can reach 0.2 meters per month.

[0138] This disease assessment method significantly improves the accurate judgment of the road maintenance cycle and makes the resource allocation more reasonable.

[0139] The present invention also discloses an intelligent road surface disease identification system based on multi-spectral images corresponding to the above method, including:

[0140] A data acquisition module 1 for acquiring visible light images and infrared thermal imaging images collected by a multi-spectral camera carried by a drone;

[0141] A data processing module 2, configured to perform registration processing on the visible light image and the infrared thermal imaging image to generate a fused image, and perform enhancement processing on the fused image based on the dark channel algorithm;

[0142] A disease extraction module 3, configured to use a convolutional neural network to extract shallow features and deep features from the enhanced image to generate an initial disease extraction image;

[0143] A segmentation module 4, configured to perform segmentation on the initial disease extraction image through adaptive threshold segmentation to generate a binary image, and perform morphological processing on the binary image to generate a candidate disease image;

[0144] A refinement module 5, configured to extract disease edge information based on instance segmentation technology and refine cracks in the disease area;

[0145] An evaluation module 6, configured to extract disease parameters, including the angle, length, and area of cracks, and evaluate the severity level of the disease in combination with temperature characteristics;

[0146] An output module 7, configured to output disease location, parameter information, and evaluation results.

[0147] In a preferred embodiment of the present invention, the system design covers a complete process from data acquisition to disease evaluation, and is specifically described as follows: A data acquisition module 1, preferably, a drone is equipped with a multi-spectral camera for image acquisition. The visible light camera and the infrared thermal imaging camera work synchronously. The former obtains the road surface texture features, and the latter captures the temperature distribution features. The system can achieve automatic flight of the drone through remote control, and the acquisition coverage rate can reach 100%. A data processing module 2, preferably, the SURF algorithm is used in this module to achieve feature point matching between the visible light image and the infrared image. The fused image is enhanced by the dark channel algorithm, so that the contrast between the disease area and the background is improved. A disease extraction module 3, based on the design of a convolutional neural network, the shallow convolutional layer is used to extract local texture features, and the deep convolutional layer captures global geometric features to generate an initial disease extraction image. In the segmentation module 4, the combination of adaptive threshold segmentation and morphological processing preferably removes the noise interference in the image. The segmentation accuracy has been verified to be above 95%. The refinement module 5 constructs a crack skeleton through the minimum spanning tree algorithm and performs linear regression fitting on the edge points. The optimized crack refinement image accurately reflects the actual shape of the disease. The evaluation module 6 fuses geometric features and temperature characteristics, and uses a weight model to quantify the severity level of the disease. Combining with the prediction of dynamic change trends, it provides a scientific basis for road maintenance. The output module 7 displays the disease location, characteristic parameters, and evaluation results in a visual form. Preferably, the system can generate a disease report and transmit it to the management center in real time through a wireless network.

[0148] Through the collaborative work of the above-mentioned modules, the system of the present invention realizes the efficient detection and evaluation of road surface diseases and is applicable to various complex scenarios.

[0149] It should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intelligent road pavement disease recognition method based on multispectral images, characterized in that: include: Obtaining steps: Acquire road surface image data collected by a multispectral camera carried by a drone, wherein the image data includes visible light images and infrared thermal imaging images; Processing steps: Based on the image data, the following operations are performed: Performing registration processing on the visible light image and the infrared thermal imaging image to generate a registered multi-spectral fusion image; Performing enhancement processing on the multispectral fusion image based on a dark channel algorithm to generate an enhanced image; Performing shallow feature extraction and deep feature extraction on the enhanced image according to a convolutional neural network to generate an initial disease extraction image; Segmenting the initial disease extraction image through adaptive threshold segmentation to generate a binary image; Performing morphological processing on the binary image to generate a candidate disease image; Extract edge information of the diseased area from the candidate diseased image based on instance segmentation technology to generate a segmented image; Performing crack thinning processing on the segmented image to generate a crack thinning image; Output steps: The crack refinement image is output, and the damage parameters, including the angle, length and area of ​​the crack, are further extracted for damage assessment.

2. The method according to claim 1, characterized in that The acquisition step specifically includes: Collect road surface images by setting a predetermined route through drones; The multispectral camera includes a visible light camera and an infrared thermal imaging camera, wherein the visible light camera collects images in the 400-700nm spectral range, and the infrared thermal imaging camera collects images in the 8-14μm band.

3. The method according to claim 1, characterized in that The registration processing in the processing step specifically includes: Performing feature point detection on the visible light image and the infrared thermal imaging image; The images are registered using affine transformation to generate a spatially aligned fused image.

4. The method according to claim 1, characterized in that: The enhancement process specifically includes: Calculating a dark channel value of the fused image based on a dark channel algorithm; The dark channel value is normalized to generate an enhanced image.

5. The method according to claim 1, characterized in that The shallow feature extraction and deep feature extraction specifically include: Extracting texture features of the enhanced image using a first convolutional layer; Extracting spatial features of the enhanced image using a second convolutional layer; The outputs of the first convolutional layer and the second convolutional layer are fused at the pixel level to generate an initial disease extraction image.

6. The method according to claim 1, characterized in that The adaptive threshold segmentation specifically includes: Set the initial segmentation threshold; The segmentation threshold is iteratively adjusted based on the grayscale mean of the foreground area and the background area until the segmentation result converges.

7. The method according to claim 1, characterized in that The crack refinement process specifically includes: The pixel points of the segmented image are connected based on the minimum spanning tree algorithm to generate the crack skeleton; The crack edge is geometrically fitted according to the crack skeleton to optimize the crack morphology.

8. The method according to claim 1, characterized in that Further including: Normalizing the angle, length and area of ​​the disease parameters; The severity rating of the disease is calculated based on the normalized parameters.

9. The method according to claim 1, characterized in that: The disease assessment further includes: By analyzing the temperature difference between the crack area and the normal pavement area and combining the geometric characteristics, the dynamic change trend of the disease is evaluated.

10. A road pavement disease intelligent identification system based on multispectral imaging based on the method according to any one of claims 1 to 9, characterized in that: include: The data acquisition module is used to obtain visible light images and infrared thermal imaging images collected by the multi-spectral camera carried by the drone; A data processing module, used for registering the visible light image and the infrared thermal imaging image to generate a fused image, and enhancing the fused image based on a dark channel algorithm; A disease extraction module, used for extracting shallow features and deep features from the enhanced image using a convolutional neural network to generate an initial disease extraction image; A segmentation module, used for segmenting the initial disease extraction image by adaptive threshold segmentation to generate a binary image, and performing morphological processing on the binary image to generate a candidate disease image; The refinement module is used to extract the edge information of the defect based on the instance segmentation technology and to refine the cracks in the defect area; An evaluation module is used to extract the damage parameters, including the angle, length and area of ​​the cracks, and evaluate the severity level of the damage in combination with the temperature characteristics; The output module is used to output the disease location, parameter information and evaluation results.

Images