A dual-spectrum-based road surface crack detection device

Abstract

The application discloses a kind of based on dual-spectrum road surface crack detection equipment, comprising: equipment design module, including camera module, support structure, inclination monitoring module, image processing unit and mobile device;Dual-spectrum fusion detection module, including heat diffusion enhancement dual-spectrum multiscale feature fusion image preprocessing module, the image collected by infrared thermal imaging camera and visible light camera is denoised and carries out multiscale feature fusion, and the detection result, category and location information are output by the unsupervised enhanced BiSeNet hybrid wavelet transform network to complete detection;Road surface quality evaluation module, the detection of road surface crack is realized by image stitching and BiSeNet network segmentation, and the road surface quality evaluation is realized by crack distribution map generation and area ratio calculation, and the display of detection result is realized by visual labeling.The application can not only greatly improve detection efficiency, reduce maintenance cost, but also provide strong support for timely maintenance of road.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of target detection, and particularly relates to a movable portable complete road section pavement crack detection and diagnosis equipment based on dual spectrum. BACKGROUND

[0002] Roads are an important part of transportation infrastructure and play a crucial role in modern transportation, providing convenient and efficient paths for vehicles and public transportation, significantly reducing travel time and improving transportation efficiency. Cracks are one of the important safety hazards of roads, and if not discovered and handled in a timely manner, they can cause serious structural damage and safety accidents. The gradual expansion of cracks can affect the structural integrity of roads, and road cracks can also cause water seepage, corrosion, and potholes, affecting the durability and lifespan of the structure, increasing operating costs and economic losses in transportation.

[0003] In daily road maintenance work, the convenience of detection equipment is one of the key factors affecting detection efficiency. In recent years, multi-sensor fusion technology has received widespread attention, and the application of multi-sensor combined target detection technology can greatly reduce labor costs. How to combine the advantages and disadvantages of different sensors to accurately and effectively detect and locate has become an important direction for many scholars to study. Comprehensive detection of complete road sections can accurately determine the location of cracks and accurately determine their severity, which is of great significance for subsequent repair and maintenance. Therefore, the development and application of a portable, accurate, efficient, and full-coverage detection device can not only greatly improve detection efficiency and reduce maintenance costs, but also provide strong support for timely maintenance of roads, thereby effectively ensuring driving safety and promoting smooth and stable road transportation. SUMMARY

[0004] The purpose of the present application is achieved by the following technical solutions.

[0005] In order to overcome the defects and shortcomings of the prior art, the present application provides a movable portable complete road section pavement crack detection and diagnosis equipment based on dual spectrum.

[0006] The technical scheme of the present application is: a movable and portable complete road section pavement crack detection and diagnosis equipment based on dual-spectrum, comprising an equipment design module, a dual-spectrum fusion detection module, and a pavement quality evaluation module. The equipment design module realizes the free movement and high adaptability of an infrared thermal imaging camera and a visible light camera through a movable and portable equipment. The dual-spectrum fusion detection module comprises a thermal diffusion enhanced dual-spectrum multi-scale feature fusion image preprocessing module, which outputs detection results, categories, and location information through an unsupervised enhanced BiSeNet hybrid wavelet transform network to complete detection. The pavement quality evaluation module comprises accurate detection of pavement cracks, pavement quality evaluation through crack distribution map generation and area ratio calculation, and intuitive display of detection results through visual labeling.

[0007] Preferably, the equipment design module comprises a camera module, a support structure, an inclination monitoring module, an image processing unit, and a moving device. The camera module combines visible light and infrared thermal imaging technology to highlight pavement crack features. The support structure provides basic support through a stable triangular support. The inclination monitoring module ensures image shooting angle consistency through a mini digital inclinometer. The image processing unit realizes efficient image analysis using an NVIDIA Jetson Xavier NX and carries a visual screen to display detection conditions in real time. The moving device contains adjustable knobs and detachable small wheels to adapt to various carriers and realize free movement.

[0008] Further preferably, the movable and portable equipment realizes flexible movement and high efficiency adaptability of the detection equipment by integrating an infrared thermal imaging camera and a visible light camera. The equipment is equipped with a mini digital inclinometer, which can monitor and display the inclination angle in real time to ensure the stability of the detection process. A stable triangular support provides basic support, and the support bottom is equipped with adjustable knobs to flexibly adjust the base angle according to the needs of different carrier platforms, meeting various installation requirements. In addition, the equipment also integrates a detachable small wheel moving device, which facilitates quick installation or disassembly, further improving the portability and adaptability of the equipment. At the same time, the equipment is designed with fixing holes, and users can choose to install small wheels or fix the equipment on other platforms through screws according to specific needs, enhancing the versatility and flexibility of the equipment.

[0009] Further preferably, the image processing unit employs NVIDIA Jetson Xavier NX to process and analyze the images captured by the infrared thermal imaging camera and the visible light camera in real time. The powerful capabilities of Jetson Xavier NX in the field of edge computing are fully utilized, allowing the use of trained models directly without the need for lightweight versions of the framework. This ensures fast response and efficient execution of data processing, providing a high-performance and efficient solution for road crack detection. The device is equipped with a visual screen that can display the detection situation in real time.

[0010] Further preferably, the dual-spectrum fusion detection module includes a thermal diffusion enhanced dual-spectrum multi-scale feature fusion image preprocessing module. Non-local mean filtering (NLM) is used to denoise images of two different spectra; Sigmoid transformation is used to enhance the features of infrared thermal images; and Laplacian-based thermal diffusion processing is used for pseudo-three-dimensional data processing to enhance image features. This method further includes a multi-scale fusion step based on a Gaussian pyramid, which unifies the processing of crack sizes in visible light and infrared spectrum images and implements a weighted fusion strategy to enhance the visibility of cracks.

[0011] Further preferably, the unsupervised enhanced BiSeNet hybrid wavelet transform network constructs a customized dataset containing unlabeled images and a small number of labeled images for unsupervised pre-training and model fine-tuning. The network architecture combines the Deep InfoMax unsupervised learning mechanism and the BiSeNet dual-path network structure, initializes BiSeNet using the pre-trained DeepInfoMax encoder weights, and realizes efficient transfer learning. On this basis, the network embeds the HWD feature extraction module based on Haar wavelet transform into the spatial path of BiSeNet, replacing the traditional stride convolution operation, thereby significantly enhancing the feature learning ability and the accuracy of capturing image details. In addition, the network accurately predicts the crack type through the classification head and comprehensively evaluates the model through key performance indicators. Finally, the optimized model is deployed on the NVIDIA Jetson Xavier NX platform to realize efficient road crack detection and edge computing functions.

[0012] Further preferably, the image stitching, segmentation and detection determination method comprises collecting road section images and retaining 10% overlapping parts for seamless stitching, dividing the stitched images into appropriate blocks to retain crack details. The image of each block is input into an unsupervised enhanced BiSeNet hybrid network, thereby obtaining a crack segmentation mask and a classification label for each block. The segmentation mask represents a binary mask of the crack area, and the classification label can clearly distinguish between small cracks, large cracks and subsidence cracks, preparing for subsequent road surface quality determination. Finally, by stitching the segmentation masks of all blocks, a complete road section crack distribution map is constructed. This distribution map can accurately determine the location of the cracks, and the classification label can also lock the crack level at the same time, thereby providing a clear target for subsequent road maintenance.

[0013] Further preferably, the comprehensive evaluation system based on the segmentation mask and the classification label accurately analyzes the road crack condition. By calculating the crack area ratio and assigning weights to different crack types, the weighted ratio is used to quantify the severity of the cracks, which are divided into five levels: no cracks, mild, moderate, severe and extremely severe. On this basis, the system superimposes crack description information on the complete road section image after stitching to generate a visual image with crack annotations.

[0014] The advantages of the present application are:

[0015] 1. The movable portable complete road section pavement crack detection and diagnosis device based on dual spectrum provided by the present application solves the image deviation problem caused by environmental changes or unstable installation of traditional detection equipment by integrating a visible light camera and an infrared thermal imaging camera and combining a mini digital tilt angle instrument for real-time monitoring of the angle. This design not only ensures clear capture of crack features, but also compensates for the shortcomings of single imaging mode through dual spectrum imaging technology, further improving detection accuracy. The device uses the NVIDIA Jetson Xavier NX platform for real-time image processing and analysis, fully utilizes its powerful edge computing capability, directly runs the trained model without the need for a lightweight framework, thereby ensuring fast response and efficient execution of data processing, and displays the detection situation in real time through the visual screen. This technical choice not only reduces bandwidth consumption and operating costs, but also simplifies the detection process and improves detection efficiency.

[0016] 2. The application provides a kind of movable portable complete road section pavement crack detection and diagnosis equipment based on dual spectrum, in the mobility and adaptability of equipment, the application realizes the free movement and flexible fixation of equipment by designing a set of comprehensive support and mobile solution.The scheme takes stable triangular support as core support structure, equipped with adjustable knob, can be flexibly adjusted according to different mounting platform angle.In addition, the device integrates detachable small wheel moving device, is convenient for quick installation or unloading, further improves the portability of equipment.At the same time, the device is designed with fixing hole, and users can choose to install small wheel or fix the device on other platforms according to specific needs, to ensure its stability and adaptability in diversified environment.

[0017] 3. The application provides a kind of movable portable complete road section pavement crack detection and diagnosis equipment based on dual spectrum, through the heat diffusion enhanced dual spectrum multiscale feature fusion image preprocessing module in dual spectrum fusion detection module, by comprehensively utilizing visible light and infrared spectrum data and multiscale analysis method, to strengthen and highlight the features of road cracks and carry out image fusion, complete the effective image preprocessing of visible light and infrared spectrum data, prepare data set for subsequent crack detection.

[0018] 4. The application provides a kind of movable portable complete road section pavement crack detection and diagnosis equipment based on dual spectrum, through the unsupervised enhanced BiSeNet hybrid wavelet transform network in dual spectrum fusion detection module, the recognition ability of network to road crack feature is enhanced, the detection result, category and position information are output, the efficient crack detection is realized and the edge computing deployment is facilitated.

[0019] 5. The application provides a kind of movable portable complete road section pavement crack detection and diagnosis equipment based on dual spectrum, through road surface quality evaluation module, the road surface condition is comprehensively evaluated, the crack position is accurately and efficiently judged, and the intuitive display of detection result is realized through visual marking.Greatly facilitate the decision-making process of road maintenance personnel and engineers, improve work efficiency, reduce labor cost, and provide new ideas for monitoring and maintenance of road infrastructure. BRIEF DESCRIPTION OF DRAWINGS

[0020] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments with reference made to the accompanying drawings. The drawings are for purposes of illustration only and are not considered a limitation of the present application. Further, like reference numerals have been used throughout the several views to designate identical elements. In the drawings:

[0021] Figure 1 The structure flow chart of the dual spectrum based movable portable complete road section pavement crack detection and diagnosis equipment described in the application;

[0022] Figure 2 This is a schematic diagram of the device structure of the portable, dual-spectrum-based pavement crack detection and diagnosis device for complete road sections as described in this invention.

[0023] Figure 3 This is a schematic diagram of the device design module structure described in this invention;

[0024] Figure 4 This is a logic diagram of the thermal diffusion-enhanced dual-spectral multi-scale feature fusion image preprocessing module described in this invention.

[0025] Figure 5 This is a schematic diagram of the unsupervised enhanced BiSeNet hybrid wavelet transform network described in this invention.

[0026] Figure 6 This is a logic diagram of the road quality assessment method described in this invention. Detailed Implementation

[0027] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0028] This invention discloses a portable, mobile device for detecting and diagnosing pavement cracks across a complete road section, based on dual-spectrum imaging. To address the need for convenient detection, a novel portable device is designed, adaptable to various road conditions and different vehicles, thus improving detection efficiency. For the problem of detecting road cracks, long-wave infrared and visible spectrum images are integrated to enhance detection accuracy. Furthermore, to meet the requirements for pavement quality assessment and accurate crack location, a pavement quality evaluation method is standardized.

[0029] like Figure 1As shown, the present invention provides a portable, mobile, and complete road surface crack detection and diagnosis device based on dual-spectrum imaging. The device includes a device design module, a dual-spectrum fusion detection module, and a road surface quality assessment module. The device design module includes a support structure, a camera module, a tilt monitoring module, an image processing unit, and a mobile device. The portable device enables free movement and high adaptability of the infrared thermal imaging camera and the visible light camera. It is equipped with an edge computing device (e.g., NVIDIA Jetson Xavier NX) for efficient image analysis and a visual screen to display the detection status in real time. The dual-spectrum fusion detection module includes a thermal diffusion-enhanced dual-spectrum multi-scale feature fusion image preprocessing module. This module denoises the images acquired by the infrared thermal imaging camera and the visible light camera and performs multi-scale feature fusion. It outputs the detection results, category, and location information through an unsupervised enhanced BiSeNet hybrid wavelet transform network. The road surface quality assessment module achieves accurate detection of road surface cracks through image stitching and BiSeNet network segmentation, assesses road surface quality through crack distribution map generation and area ratio calculation, and provides an intuitive display of the detection results through visual annotation.

[0030] Specifically, the implementation process of the portable, dual-spectrum-based pavement crack detection and diagnosis device for complete road sections described in this invention includes the following:

[0031] 1. Equipment design module for a portable, dual-spectrum-based, full-section pavement crack detection and diagnosis device:

[0032] like Figure 2 As shown, the present invention provides a portable, mobile, and complete road surface crack detection and diagnosis device based on dual-spectrum imaging. The device design modules include a camera module, a support structure, a tilt monitoring module, an image processing unit, and a mobile device.

[0033] The camera module is equipped with both an infrared thermal imaging camera and a visible light camera. The presence of water or air at the crack causes differences in thermal conductivity, hindering or exacerbating heat conduction and thus affecting heat transfer. Furthermore, the crack area differs from the surrounding road surface in terms of structure, resulting in variations in illumination and ventilation. Therefore, the infrared thermal imaging camera can clearly capture the characteristics of the crack, complementing the images captured by the visible light camera.

[0034] The support structure is a stable triangular support as the core support structure, which provides a solid foundation for the equipment. The triangular design utilizes the stability principle of geometry to ensure the firmness of the entire structure, not only providing a solid foundation for the equipment, but also effectively dispersing and absorbing external impact forces, ensuring stable operation of the equipment in various complex environments. In addition, the adjustable design of the triangular support can adapt to different sizes and weights of equipment, further enhancing its versatility and practicality.

[0035] The inclination monitoring module selects a light and portable digital inclinometer. The device has a measurement range of 4x90°, an accuracy of ±0.2°, a resolution of 0.01°, and an EBTN display. The device size is 60x60x30mm, and a commonly used face magnetic base is designed for easy use and measurement.

[0036] The image processing unit uses edge computing devices (including but not limited to NVIDIA Jetson Xavier NX) to process and analyze the collected images in real time. This selection fully utilizes the powerful capabilities of edge computing, allowing the use of trained models without the need for lightweight versions of the framework, ensuring fast response and efficient execution of data processing. The device is equipped with a visual screen that displays the detection situation in real time, providing a high-performance and efficient solution for road crack detection.

[0037] The mobile device is composed of a knob, a detachable small wheel, and a fixed hole. The support bottom is equipped with an adjustable knob that allows flexible adjustment of the angle of the tripod base according to requirements, achieving the angle requirements when fixed with different mounting platforms. In addition, the device also integrates a detachable small wheel moving device, which facilitates quick installation or removal to adapt to different movement needs. At the same time, the device is designed with fixed holes, allowing users to freely choose to install the small wheel or fix the device to other platforms through screws according to specific needs.

[0038] As shown in Figure 3 , it is a structural schematic diagram of the movable portable complete road section pavement crack detection and diagnosis equipment based on dual-spectrum provided by the present application.

[0039] 2. Heat diffusion enhanced dual-spectrum multi-scale feature fusion image preprocessing module:

[0040] As shown in Figure 4 , it is a logic diagram of the heat diffusion enhanced dual-spectrum multi-scale feature fusion image preprocessing module. Heat diffusion enhanced dual-spectrum multi-scale feature fusion image preprocessing can be divided into four steps:

[0041] (1) Non-local means (NLM) filtering of complex cracks based on similarity principle.

[0042] Non-local means filter based on similarity principle is used to denoise the visible spectrum image and long-wave infrared spectrum image. By searching for similar pixel blocks in the whole image range and using these redundant information, NLM filter effectively suppresses the noise while preserving the image details, providing high-quality image data for the accurate extraction of crack features. The principle formula is as follows:

[0043] Assuming the noisy image is , the denoised image is , The pixel gray value at is calculated as follows:

[0044] ;

[0045] Where the weight represents the similarity between pixel points and , and its value is determined by the distance between the rectangular neighborhood , : , ;

[0046] ;

[0047] ;

[0048] ;

[0049] Where is the normalization coefficient, is the smoothing parameter, d is the edge length of the small neighborhood window, is the small search window area, and controls the attenuation degree of the Gaussian function. The larger the Gaussian function, the more gentle the change, and the higher the denoising level, but it will also cause the image to be more blurred; The smaller, the more edge details are preserved, but there will be too many noise points left; The specific value of

[0050] (2) Crack feature Sigmoid transformation based on logical function. After NLM processing, the long-wave infrared spectrum image is subjected to Sigmoid transformation, effectively enhancing the crack feature and making it more prominent in the image. The principle formula is as follows:

[0051] ;

[0052] Where, ​steepness of the control function, is the center point of the function. This function maps input values to the range (0, 1) with smooth transitions.

[0053] (3) Heat diffusion processing based on Laplacian operator. By simulating the diffusion process of heat in the image, the key information in the image is further improved, especially the visibility of crack features in visible light images and infrared images. The visible light image and the infrared image processed by the previous two steps are stacked into pseudo-three-dimensional data, so that the two images are processed simultaneously as different channels. Then, the heat diffusion algorithm is applied to iteratively calculate each pixel point, smooth the high-frequency noise and unnecessary details in the image, and thus enhance the prominence of important features such as cracks.

[0054] In the heat diffusion process, heat (or change) in the image will diffuse to surrounding pixels through the Laplacian operator, so that the mutation area of the image, i.e. the crack edge, is smoothed, while the smooth area remains stable. Through multiple iterations, the heat diffusion algorithm can effectively remove noise in visible light and infrared images and enhance the saliency of cracks and other features. Through the stacked pseudo-three-dimensional data, heat diffusion not only processes in a single image channel, but also shares information across channels, so that crack features can be complementary between visible light and infrared images, further improving the recognizability of cracks. The image after heat diffusion processing retains more key information, and the crack features are clearer, obtaining two independent data, further improving the image quality and the accuracy of feature extraction.

[0055] (4) Gaussian pyramid fusion of dual-spectrum images based on multi-scale image fusion. The Gaussian pyramid is used to unify the size of cracks in visible light and infrared spectrum images, ensuring that the features in the two images can be one-to-one. On this basis, a weighted fusion strategy is further implemented, giving higher weight to images with more obvious crack features. Through this fusion method, not only the key information in the dual-spectrum image is retained, but also the visibility of the crack is strengthened, providing more abundant and accurate image data for the accurate detection and analysis of cracks, completing the effective image preprocessing of visible light and infrared spectrum data, and preparing the data set for subsequent crack detection.

[0056] 3. Unsupervised enhanced BiSeNet hybrid wavelet transform network:

[0057] As Figure 5As shown, the unsupervised enhanced BiSeNet hybrid wavelet transform network schematic diagram is shown. The hybrid DeepInfoMax (DIM) unsupervised learning and BiSeNet dual-path network are mixed, and a full connection layer and a wavelet transform-based feature extraction module are further integrated on this basis. Not only the feature learning ability of the model is enhanced, but also the ability to capture image details is improved through the wavelet transform module, so as to realize higher precision and efficiency in the image segmentation task. The unsupervised enhanced BiSeNet hybrid wavelet transform network can be divided into 8 steps:

[0058] (1) Data set preparation. The data set is composed of two parts: one part is unlabelled images, which are used for unsupervised pre-training stage to enhance the network's ability to recognize road crack features; the other part is a small amount of labeled images, which are used for fine-tuning the segmentation model. The labeled images contain crack segmentation labels, indicating the presence or absence of cracks in the image, and crack classification labels, which distinguish the types of cracks.

[0059] (2) Deep InfoMax-based unsupervised task learning. Unlabeled images are used for Deep InfoMax unsupervised task learning training, and after the image feature learning is completed, the pre-trained encoder parameters can be obtained.

[0060] (3) Transfer learning. The DIM pre-trained encoder is used as the initialization weight of BiSeNet to adapt to specific downstream tasks, providing pre-trained feature representation, which helps to improve the performance and efficiency of new tasks, and migrates parameters to the spatial path and context path of BiSeNet network.

[0061] (4) HWD module based on Haar wavelet transform. The HWD (Haar Wavelet-based Downsampling) module based on Haar wavelet transform is integrated into the spatial path of BiSeNet network, replacing the original stride convolution. Not only the feature expression ability is enhanced, but also the low-frequency and high-frequency components of the image are effectively captured and preserved through Haar wavelet transform, so as to realize better performance in detail feature extraction and overall structure preservation, significantly improving the precision and efficiency of segmentation.

[0062] (5) Model fine-tuning. In order to further improve the precision of crack segmentation, a small amount of labeled data set is used to fine-tune the model. During the fine-tuning process, a loss function suitable for segmentation tasks such as Dice loss or cross-entropy loss is used to optimize the model performance. On the fine-tuned network model, road crack detection is carried out, and the model can realize automatic segmentation of crack areas in the image and generate crack position and size information for subsequent data analysis or system integration.

[0063] (6) Model classification. In order to further distinguish the crack types, a classification head-fully connected layer is added to the output layer to predict the type of each crack region, such as small crack, large crack, and subsidence crack.

[0064] (7) Model Evaluation. The model evaluation comprehensively utilizes key indicators such as F1 score, mean squared error (MSE), and recall to quantify the model's performance and reliability. The F1 score, through the harmonic mean of precision and recall, balances the model's accuracy and completeness in identifying positive samples; the mean squared error measures the deviation between predicted and true values, reflecting the model's prediction accuracy; and the recall rate evaluates the model's ability to detect positive samples. The comprehensive analysis of these indicators provides a scientific basis for model optimization.

[0065] (8) Model Deployment and Visualization. In order to achieve high-efficiency crack detection and facilitate edge computing deployment, the trained model is optimized and integrated into the edge computing device platform, equipped with a visual screen to display the detection status in real time, thus providing a high-performance and high-efficiency solution for road crack detection and providing strong technical support for on-site detection and immediate processing.

[0066] 4. Road surface quality assessment:

[0067] like Figure 6 The diagram shown is a logic diagram of the road surface quality assessment method. This invention proposes a road surface quality assessment method based on image analysis, aiming to assess road quality by calculating the proportion of cracks in the road surface. The road surface quality assessment method can be divided into four steps:

[0068] (1) Image stitching and segmentation. The image stitching and segmentation are achieved by acquiring images of the road segment, retaining a 10% overlap between images to achieve seamless stitching and form a continuous image of the road segment. Subsequently, the stitched image is divided into blocks of appropriate size to ensure that crack details are fully preserved.

[0069] (2) Acquisition of crack segmentation masks and classification labels. The image of each block is input into a self-supervised BiSeNet hybrid network to obtain the crack segmentation mask and classification label for each block. The segmentation mask represents the binary mask of the crack region, and the classification label can clearly distinguish between small cracks, large cracks, and subsidence cracks, preparing for subsequent pavement quality assessment. Finally, by stitching together the segmentation masks of all blocks, a complete crack distribution map of the road segment is constructed. This distribution map can accurately determine the location of cracks, and the classification label can also lock the crack level while locating the crack, thus providing a clear target for subsequent road maintenance.

[0070] (3) Based on the segmentation mask and the classification label, the application provides a method for calculating the crack area proportion of each target region. The application gives different weights to each type of crack, small crack: weight ; large crack: weight ; and subsidence crack: weight For each type of crack (assuming type i, i = 1, 2, 3), the area of each crack type is calculated by the segmentation mask . According to the defined weight coefficient and the area of the crack type, the weighted ratio can be calculated, and the formula is as follows:

[0071] ;

[0072] wherein is the weight coefficient of crack type i; is the area of the i-th type of crack; is the total area of the image. When , it is no crack or slight crack; when , it is mild crack; when , it is moderate crack; when , it is severe crack; and when , it is extremely severe crack.

[0073] (4) Visual display. In order to realize the visualization of the detection result, after the calculation and severity determination of the cracks, the device superimposes the text description of the cracks onto the spliced image, thereby generating a complete road section image with crack labeling, intuitively marking the crack position and the road crack condition, facilitating the decision-making process of road maintenance personnel and engineers, improving work efficiency, and reducing labor cost.

[0074] The portable complete road section pavement crack detection and diagnosis device based on dual-spectrum provided by the application not only greatly improves the detection efficiency and reduces the maintenance cost, but also provides strong support for the timely maintenance of the road, thereby effectively ensuring the driving safety and promoting the smoothness and stability of the road transportation.

[0075] The above is only a preferred specific embodiment of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be covered within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A road surface crack detection device based on dual spectrum, characterized in that, include: The equipment design module, the dual-spectrum fusion detection module, and the road surface quality assessment module; among them, The device design module includes a camera module, a support structure, a tilt monitoring module, an image processing unit, and a mobile device; The dual-spectral fusion detection module includes a thermal diffusion-enhanced dual-spectral multi-scale feature fusion image preprocessing module, which denoises the images acquired by the infrared thermal imaging camera and the visible light camera and performs multi-scale feature fusion. The detection results, category and location information are output through an unsupervised enhanced BiSeNet hybrid wavelet transform network to complete the detection. The road surface quality assessment module detects road surface cracks through image stitching and BiSeNet network segmentation, assesses road surface quality by generating crack distribution maps and calculating area proportions, and displays the detection results through visual annotation. The detection is completed by outputting detection results, category, and location information through an unsupervised enhanced BiSeNet hybrid wavelet transform network, including: A dataset containing unlabeled images and a small number of labeled images is constructed. The unlabeled images are used for unsupervised pre-training, and the labeled images are used to fine-tune the segmentation model. The labeled images contain crack segmentation labels, indicating the presence or absence of cracks in the image, and crack classification labels, distinguishing crack types. Based on DIM unsupervised task learning, unlabeled images are used for DIM unsupervised task learning training to obtain pre-trained encoder parameters. The DIM pre-trained encoder is used as the initial weights of BiSeNet for transfer learning, transferring the parameters to the spatial and contextual paths of the BiSeNet network. The HWD feature extraction module based on Haar wavelet transform is integrated into the spatial path of the BiSeNet network, replacing the preset stride convolution, thereby enhancing feature learning ability and image detail capture. The model was fine-tuned using the labeled images; The type of each crack region is predicted using a fully connected layer; Model evaluation is conducted using key indicators; The trained model is integrated into the image processing unit.

2. The road surface crack detection device based on dual spectrum according to claim 1, characterized in that, The camera module includes a visible light camera and an infrared thermal imaging camera; The tilt monitoring module is a digital tilt meter; The image processing unit uses an edge computing device and has a visual screen to display the detection status in real time; The mobile device includes an adjustable knob and detachable wheels.

3. The pavement crack detection device based on dual spectrum according to claim 1 or 2, characterized in that, The thermally diffused dual-spectral multi-scale feature fusion image preprocessing module includes: Nonlocal mean filtering of complex cracks based on the principle of similarity is performed on infrared and visible spectral images; Only the filtered infrared spectral image is subjected to a crack feature sigmoid transform based on a logistic function to enhance the infrared thermal image features; The processed infrared and visible spectral images are then subjected to pseudo-3D data processing using thermal diffusion based on the Laplacian operator to enhance image features. The Gaussian pyramid fusion of dual-spectral images based on multi-scale image fusion enhances the visibility of cracks by uniformly processing crack sizes in visible and infrared spectral images and implementing weighted fusion.

4. The pavement crack detection device based on dual spectrum according to claim 3, characterized in that, The thermal diffusion process smooths high-frequency noise and details in the image by iteratively calculating each pixel, thereby enhancing the prominence of crack features.

5. The pavement crack detection device based on dual spectrum according to claim 1, characterized in that, The unsupervised enhanced BiSeNet hybrid wavelet transform network combines DIM unsupervised learning and BiSeNet dual-path network, and integrates a fully connected layer and a wavelet transform-based feature extraction module.

6. The pavement crack detection device based on dual spectrum according to claim 5, characterized in that, The fine-tuning of the model using the labeled images includes employing a loss function suitable for the segmentation task to optimize model performance.

7. The pavement crack detection device based on dual spectrum according to claim 1, characterized in that, The image stitching includes: acquiring road segment images and retaining 10% overlap for seamless stitching, and dividing the stitched image into blocks of a preset size to preserve crack details.

8. The pavement crack detection device based on dual spectrum according to claim 7, characterized in that, The BiSeNet network segmentation includes: Each segmented block is input into an unsupervised augmented BiSeNet hybrid network to generate a crack segmentation mask and a classification label. The segmentation mask represents a binary mask of the crack region, and the classification label is used to distinguish between small cracks, large cracks, and subsidence cracks. By splicing and segmenting masks, a complete distribution map of road cracks is constructed, and the location and level of cracks are determined by combining classification labels.

9. The pavement crack detection device based on dual spectrum according to claim 8, characterized in that, The method of assessing road surface quality by generating crack distribution maps and calculating area proportions includes: Based on segmentation masks and classification labels, the severity of cracks is assessed by calculating the proportion of crack area and assigning weights to different crack types, and then using a weighted ratio.

Images