Road crack detection method based on thermo-optical matrix fusion

Abstract

The invention designs a road crack detection method based on thermo-optical matrix fusion, and the method comprises the steps: obtaining a temperature matrix and a visible light image based on a visible light-temperature matrix collection device, extracting crack edge features through temperature gradient analysis, and carrying out the binarization processing, thereby obtaining a temperature matrix binary image; a thermal-optical matrix image fusion module is provided, and a temperature matrix and a visible light image are effectively fused; a thermal-optical matrix fusion crack image obtained based on the fusion module is used as the input of an improved YOLOv8-AMPN network model, the semantic enhancement effect of a temperature matrix on a visible light image is further improved through a feature weight adaptive weighted fusion and region block level dynamic fusion method in the network, and crack features are highlighted. The method has the advantages that a temperature matrix branch is introduced, the limitation of a visible light image under environment change is effectively overcome, the problem that multi-modal image information is redundant and complex is solved, and the sensitivity of a model to crack characteristics and the detection response speed are improved through temperature information.

Description

Technical Field

[0001] The present invention relates to a road crack detection technology, in particular to an automated detection method based on thermal-optical matrix fusion. The main innovation is to use the dual-modal fusion of visible light images and temperature matrices to solve the problem of insufficient robustness of single-modal detection in complex road environments. The temperature matrix has stability under changing light conditions and can effectively distinguish the temperature difference information between cracks and normal road surfaces, while the visible light image has a higher resolution for crack details. By introducing an improved YOLOv8 network architecture, combined with the Adaptive Multimodal Pyramid Network (AMPN) and the adaptive weighted fusion of feature weights in the network and the regional block-level dynamic fusion method, the system realizes multi-scale feature enhancement and dynamic resolution adjustment, further enhances the semantic enhancement effect of the temperature matrix on visible light images, highlights crack features, adapts to a variety of detection scenarios, and significantly improves the accuracy and efficiency of crack detection, reduces manual participation, and reduces maintenance costs. Background Art

[0002] With the rapid development of road traffic, the aging of infrastructure and the increase in vehicle loads, road crack detection and maintenance have become extremely important. Cracks are a manifestation of early road damage. If not repaired in time, they will lead to further structural damage and safety hazards, significantly increasing repair costs and accident risks. Traditional manual inspection methods are inefficient and subjective, prone to missed inspections and false inspections, causing small cracks to deteriorate into large-scale damage, increasing resource consumption for later maintenance.

[0003] Existing crack detection methods are mainly based on single-modality images (such as visible light images) or a combination of visible light and infrared images. Although they can detect cracks to a certain extent, they are limited in robustness and accuracy due to the complex road environment and changes in lighting. This is especially true in complex environments such as insufficient lighting or surface reflections, which can easily lead to a decrease in detection accuracy. When dealing with complex scenarios, current detection methods often increase manual review costs and the burden of later repairs due to false detections and missed detections. Summary of the invention

[0004] To overcome these problems, the present invention proposes a road crack detection method based on thermal-optical matrix fusion. By combining the dual-modal information of visible light image and temperature matrix, the stability of temperature matrix under illumination changes and the high detail resolution advantage of visible light image are utilized to achieve accurate detection of cracks.

[0005] Specifically, the present invention provides a road crack detection method using thermal-optical matrix fusion, the method comprising: The temperature matrix and visible light image are acquired based on the visible light-temperature matrix acquisition device, the crack edge features are extracted through temperature gradient analysis, and binarization is performed to obtain a temperature matrix binarization image; The temperature matrix is ​​fused with the visible light image through the thermal-optical matrix image fusion module; The thermal-optical matrix fused crack image obtained based on the thermal-optical matrix image fusion module is used as the input of the improved YOLOv8-AMPN network model, and an enhanced thermal-optical matrix fused crack image is obtained through feature weight adaptive weighted fusion and regional block-level dynamic fusion in the network model; Crack identification is performed based on the enhanced thermal-optical matrix fusion crack image.

[0006] Furthermore, the visible light-temperature matrix acquisition device includes: a visible light camera, a coaxial alignment adjustment rod, a push handle, a synchronization trigger, an infrared thermal imaging camera, a level, a laser pointer, and a movable triangular stabilization bracket.

[0007] Furthermore, the optical axes of the visible light camera and the infrared thermal imaging camera are adjusted to be parallel through the coaxial alignment rod and the level; and the visible light camera and the infrared thermal imaging camera are adjusted to maintain a consistent field of view at different focal lengths through the laser pointer.

[0008] Furthermore, the visible light-temperature matrix acquisition device acquires the temperature matrix and the visible light image, extracts the crack edge features through temperature gradient analysis, and performs binarization processing to obtain a temperature matrix binarization map, including: Obtaining a visible light image, adjusting the visible light image to a preset resolution, and unifying the image size; Based on the visible light-temperature matrix acquisition device, the temperature matrix data is acquired, the grayscale image of the temperature matrix is ​​generated, and the temperature matrix gradient is calculated to segment the crack edge, and the temperature matrix binary image is further generated to correspond to the visible light image; Data enhancement is performed on the temperature matrix binary image and the visible light image.

[0009] Furthermore, the temperature matrix is ​​fused with the visible light image by the thermal-optical matrix image fusion module, including: (1) Preliminary feature alignment First, the temperature matrix features and visible light image features are input into the thermal-optical matrix image fusion module respectively, and the shared convolutional alignment layer is used for feature exchange and feature alignment; (2) Feature complementarity enhancement The similarity score is calculated through the similarity calculation layer, and the enhancement between the temperature matrix features and the visible light image features is adjusted by a learnable parameter: (3) Multi-layer refinement fusion The enhanced features of the visible light image and the temperature matrix are fused layer by layer, using channel-level and spatial-level attention mechanisms; (4) Thermal-optical enhancement module Through the preset convolution layer, the fused features are refined and extracted; The contribution of each modality to the final feature is calculated using a complementary weight layer, and a fused feature is obtained based on the contribution; The fused features are input into the saliency prediction layer to obtain a crack saliency map.

[0010] Furthermore, the thermal-optical matrix fusion crack image obtained based on the thermal-optical matrix image fusion module is used as the input of the improved YOLOv8-AMPN network model, and the enhanced thermal-optical matrix fusion crack image is obtained through the feature weight adaptive weighted fusion and regional block-level dynamic fusion in the network model, including: S41: Multi-scale convolution kernel fusion; S42: Dynamic resolution adjustment; S43: Global feature enhancement.

[0011] Furthermore, the multi-scale convolution kernel fusion includes: Introduce convolution kernels of different sizes into the convolution module of the YOLOv8-AMPN network model; In the YOLOv8-AMPN network model, the feature maps extracted by different convolution kernels are spliced ​​in different modes to achieve multi-scale fusion of visible light features and temperature matrix features; In the spliced ​​multi-scale features, according to the differences between the visible light features and the temperature matrix features, the feature contributions of the two modalities are adjusted through feature weight adaptive weighted fusion to achieve multi-modal feature expression; The feature weight adaptive weighted fusion includes: Feature segmentation: segment the visible light image and temperature matrix features into regional blocks; The adaptive weight matrix is ​​generated by calculating the average brightness and temperature difference of the area block: For each region block, the visible light image and temperature matrix feature blocks are weighted fused through an adaptive weight matrix; By fusing feature blocks of all regional blocks, a fused overall feature map is obtained.

[0012] Furthermore, the dynamic resolution adjustment includes: For each area block, a directional filter is used to perform edge detection and generate a direction matrix to determine the main direction of the cracks in the area block; According to the direction matrix, an adaptive convolution kernel is designed for each region block, which is applied to the visible light image and temperature matrix feature extraction respectively; in the design of the adaptive convolution kernel, the shape and size of the convolution kernel are dynamically adjusted according to the direction and width characteristics of the crack; In each area block, the adaptive fusion weight is calculated according to the temperature difference and edge definition; Based on the adaptive fusion weight, the visible light and temperature matrix features in each area block are fused to obtain a fused feature block: A dynamically fused overall feature map is obtained based on all region blocks.

[0013] Furthermore, the global feature enhancement includes: The overall feature map of the region block adaptive weighted fusion is spliced ​​with the overall feature map of the region block level dynamic fusion to form a multi-scale feature map; A global pooling operation is performed on the multi-scale feature map to generate a globally enhanced multi-scale feature map.

[0014] The advantages of the present invention are that the temperature matrix can effectively distinguish the temperature difference information between the crack area and the normal road surface, compensating for the shortcomings of the visible light image under complex lighting conditions; and the visible light image has a higher resolution for the fine structure of the crack. The combination of the two can effectively improve the robustness and accuracy of detection.

[0015] Furthermore, by introducing the improved YOLOv8 network architecture, combined with the Adaptive MultimodalPyramid Network (AMPN) and the multimodal feature fusion module, the present invention can adaptively fuse visible light and temperature matrix features, realize multi-scale feature enhancement and dynamic resolution adjustment, and adapt to the crack detection needs under different environments and conditions. This method significantly improves the efficiency and accuracy of crack detection, reduces manual participation, reduces maintenance costs, and extends the service life of roads. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 It is a flow chart of the road crack detection method of the present invention using thermal-optical matrix fusion.

[0017] Figure 2 It is a schematic diagram of the visible light-temperature matrix acquisition device of the present invention.

[0018] Figure 3Schematic diagram of the related preprocessing image of the present invention.

[0019] Figure 4 It is the visible light-temperature matrix binary feature fusion graph of the present invention. DETAILED DESCRIPTION

[0020] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0021] The present invention designs a road crack detection method that combines thermal and optical matrix to improve the road crack detection accuracy and efficiency. Figure 1 The figure is a flow chart of the road crack detection method of the present invention using thermal-optical matrix fusion. The specific description is as follows: S1: Data acquisition device construction Visible light (VIS) images and infrared (IR) thermal imaging temperature matrices of road cracks are acquired through dual-modal imaging equipment. The relevant information collected by each modality is stored in the database for subsequent multi-modal feature processing.

[0022] In the data collection part, innovative equipment construction methods and shooting processes are used to ensure the acquisition of accurate data of visible light (VIS) and infrared (IR) thermal imaging temperature matrices, so as to achieve high-quality multimodal information acquisition for the same crack.

[0023] S11: A visible light camera (i.e., visible light camera) and an infrared thermal imaging device (i.e., infrared camera) are installed on the same device to form a dual-mode device for synchronous imaging. These cameras have adjustable focal lengths and angles to ensure that crack images in the same field of view are captured simultaneously.

[0024] S12: To ensure the stability of the shooting, this device includes a triangular stabilization bracket with a shock-absorbing function that carries two shooting devices at the same time, ensuring that the devices are not affected by vibration or external interference during the shooting process. The bracket has the function of adjusting the height and angle to adapt to different road surfaces and crack positions. Figure 2 As shown, the visible light-temperature matrix acquisition device of the present invention comprises a visible light camera 1, a coaxial alignment adjustment rod 2, a hand push handle 3, a synchronization trigger 4, an infrared thermal imaging camera 5, a level 6, a laser pointer 7, and a movable triangular stabilizing bracket 8. The two cameras may also be referred to as cameras in the following description.

[0025] S13: In order to ensure that the visible light and infrared thermal imaging cameras can capture the overlapping area of ​​the same crack, the equipment uses a coaxial alignment system to adjust the optical axes of the two cameras to be parallel. Through the mechanical adjustment device, the two devices maintain the same field of view at different focal lengths.

[0026] In order to ensure the consistency of the field of view of visible light and infrared thermal imaging cameras at different focal lengths, the following calculation formula can be introduced to perform calibration based on the relationship between the field of view (FOV) and focal length (f). The calculation formula for field of view consistency is as follows:

[0027] The calculation formula of field of view width is: in, is the horizontal field of view of the camera, ω is the width of the sensor, and f is the focal length. This formula can be used to determine whether the field of view widths of visible light and infrared thermal imaging cameras are consistent.

[0028]

[0029] The calculation formula of field of view height is: in, is the vertical field of view of the camera, and h is the height of the sensor.

[0030] Field of view overlap calibration formula: In order to ensure the field of view overlap area of ​​visible light and infrared thermal imaging images at different focal lengths, the difference in field of view width and height can be used to calculate the overlap ratio (Overlap Ratio, OR):

[0031] in, and The horizontal field of view of the visible light and infrared thermal imaging cameras are used respectively. and are the vertical field of view of the visible light and infrared thermal imaging cameras, respectively. When OR is close to 1, the field of view overlap is optimal.

[0032] In the present invention, the visual field consistency mechanical adjustment device comprises: (1) Laser pointer A laser pointer is set on the top of the tripod device, pointing to the crack target area, to accurately calibrate the shooting position and ensure that the equipment is correctly aligned with the crack. The role of the laser pointer is to enable the visible light and infrared thermal imaging cameras to be accurately aligned before shooting, reducing subsequent alignment errors and improving the spatial consistency and detection accuracy of multimodal data.

[0033] (2) Mechanical structure adjustment parts Adjustment device: Add fine-tuning knobs and slide rails to the camera bracket to enable precise adjustment of the camera position and angle in the horizontal and vertical directions. The fine-tuning knob can finely adjust the optical axis of the camera so that the fields of view of the two cameras overlap on the calibration mirror mark. After completing the initial adjustment, the collimation mark on the calibration mirror can be observed by human eyes or auxiliary cameras to confirm whether the fields of view of the two cameras completely overlap. This step can be compared with the position of the mark in the camera field of view and the naked eye observation through the half mirror.

[0034] (3) Formula consistency check Field of view consistency formula check: The field of view consistency visible to the human eye is achieved through calibration, and the field of view consistency maintenance formula can be used for accuracy verification. For example: Width field of view overlap calculation:

[0035] Altitude field of view overlap calculation: The consistency of the two fields of view is determined by calculating that the overlap is close to 1.

[0036] (5) Long-term consistency test Regular image inspection: During the use of the equipment, regularly check whether there is any deviation by observing the image of the collimation mark taken to ensure long-term field of view consistency. If the collimation mark is offset, the bracket fine-tuning rod can be readjusted to restore the field of view consistency.

[0037] Mark alignment formula verification: After long-term use, if accuracy needs to be confirmed, the field of view overlap can be recalculated using the formula to ensure calibration accuracy at all focal lengths and in different environments.

[0038] S14: In order to capture visible light and infrared thermal imaging images at the same time, the device integrates a synchronization trigger. Each time a shot is taken, the trigger starts the two cameras at the same time, ensuring that the two images obtained are time synchronized, thus ensuring the consistency of crack information at the same moment.

[0039] S15: Before photographing the crack, install the laser pointer at the center of the base of the dual-mode device to calibrate the center position of the crack and ensure that the device is facing the crack area.

[0040] S16: During the acquisition process, the real-time monitoring function of the camera itself can be used to check at any time whether the quality of the acquired image meets the inspection requirements. If blur, offset or underexposure occurs, the operator can make adjustments.

[0041] S2: Preprocessing of visible light image and temperature matrix data S21: Adjust the visible light image to a suitable resolution and unify the image size.

[0042] S22: Based on the temperature matrix data acquired by the visible light-temperature matrix acquisition device, grayscale image generation and temperature matrix gradient calculation are performed on the temperature matrix to segment the crack edge, and a temperature matrix binary image is further generated to correspond to the visible light image. Figure 3 As shown, it includes temperature matrix grayscale image, temperature matrix gradient calculation segmentation map, temperature matrix binary image, and visible light image.

[0043] S23: Data Augmentation Data enhancement is performed through random cropping, flipping, rotating, and scaling in geometric transformations to enhance data geometry. This enhances data diversity and enables the model to adapt to cracks in different perspectives.

[0044] After the above data preprocessing and enhancement, the data distribution of the two modalities is ensured to be consistent and can correspond. The final output is a paired visible light image and a temperature matrix binary image to be input into the thermal-optical matrix fusion module.

[0045] S3: Thermal-Optical Matrix Fusion Module This module is mainly aimed at the fusion of thermal imaging matrix and visible light image, aiming to enable the thermal matrix and optical matrix to play their respective advantages in fusion through feature alignment, feature compensation and deep fusion. The module design focuses on improving the feature representation of the crack area and reflects the complementarity of temperature and optical information in the final output.

[0046] S31: Preliminary feature alignment First, the temperature matrix feature F _t and the visible light image F_ v The features are input into the module separately. To ensure that the two modal features maintain the same scale and feature resolution before fusion, a shared convolutional alignment layer (SCAL) is used for feature exchange.

[0047]

[0048] Among them, SCAL is a multi-level convolutional layer containing 3×3 and 1×1 convolutions, which is used to effectively refine and align the input visible light image and temperature matrix features.

[0049] S32: Feature complementation enhancement In order to understand the feature similarity between the visible light image and the temperature matrix, a similarity computation layer (SCL) is designed:

[0050] where ϵ is a value that prevents the denominator from being zero.

[0051] Based on the similarity score, a learnable parameter Adjust the enhancement between different modal features:

[0052] S33: Multi-layer refinement fusion The enhanced features of the visible light image and the temperature matrix are fused layer by layer, using channel-level and spatial-level attention mechanisms.

[0053]

[0054] in, Indicates channel-level feature concatenation and fusion.

[0055] S34: Thermal-Optical Enhancement Module By combining the uniqueness of visible light images and temperature matrix features, a dedicated convolution layer is designed to first perform a feature refinement extraction on the fused features:

[0056] A complementary weight layer is used to automatically calculate the contribution of each modality to the final features:

[0057] The final fusion features are:

[0058] S35: Final fusion features The crack saliency map is obtained through the saliency prediction layer:

[0059] Where σ represents the Sigmoid activation function, which is used to map the output to [0, 1] to represent the crack significance probability.

[0060] In the process of thermal-optical matrix fusion, the temperature matrix can effectively compensate for the deficiencies of the visible light image under changing ambient light. The temperature matrix provides temperature difference information of the crack area, which is still valid under changing lighting conditions, shadows or blur conditions, thereby enhancing the robustness of the model to crack features. This compensation feature enables the temperature matrix to play a key role in complex road environments, ensuring more stable and accurate crack detection.

[0061] Compared with visible light images, the temperature matrix is ​​more sensitive to cracks under temperature changes. Therefore, in the process of feature alignment and fusion, the shared convolutional alignment layer (SCAL) and feature complementary enhancement module are used to effectively extract the unique thermal contrast features in the temperature features, thereby improving the significance of the crack area. Figure 4 As shown, it is the visible light-temperature matrix binary feature fusion diagram of the present invention.

[0062] S4: Dual-modal multi-scale feature enhancement YOLOv8-Adaptive Multimodal Pyramid Network (YOLOv8-AMPN) According to the characteristics of actual road scenes, the YOLOv8 network structure is optimized and the Adaptive MultimodalPyramid Network (AMPN) is introduced to enhance the multi-scale crack feature extraction capability, so that the model can effectively handle large-scale cracks while maintaining sensitivity to small cracks. AMPN adaptively fuses visible light image and temperature matrix features at multi-modal and multi-scale levels, thereby improving the model's detection effect on complex crack features.

[0063] S41: Multi-scale convolution kernel fusion 1. In order to adapt to multi-scale features, we introduce convolution kernels of different sizes in the convolution module. For example, 1x1, 3x3 and 5x5 are used to capture small and large-scale crack features. 1x1 convolution kernel: used to integrate information between channels, compress high-dimensional features to lower dimensions, and avoid the loss of small feature information; 3x3 convolution kernel: used for general feature extraction, more sensitive to small and medium-scale features of cracks (such as edge texture); 5x5 convolution kernel: suitable for extracting large-scale crack features, and can capture the features of wider crack areas.

[0064] 2. Feature splicing: In AMPN, the feature maps extracted by different convolution kernels are spliced ​​in different modes to achieve multi-scale fusion of visible light features and temperature matrix features. The formula is as follows:

[0065] In the spliced ​​multi-scale features, AMPN adjusts the feature contributions of the two modalities through adaptive weights according to the differences between visible light features and temperature matrix features to achieve richer multimodal feature expression.

[0066] Among them, an innovative feature weight adaptive weighted fusion method is proposed, and the specific steps are as follows: Feature Blocking: Assume that and Divide into The area block, the size of each feature map is , we can get the following set of area blocks:

[0067] in, and Indicates location H is the height of the feature map, and W is the width of the feature map, representing the number of pixels in the vertical and horizontal directions respectively.

[0068] Weight generation: Generate an adaptive weight matrix by calculating the average brightness and temperature difference of the area block:

[0069] in, and Respectively indicated at the position The visible light image and the eigenvalues ​​of the temperature matrix.

[0070] Adaptive weighted fusion of regional blocks: For each regional block, we use the corresponding weight and Weighted fusion of visible light image and temperature matrix feature blocks:

[0071] Finally, the fused overall feature map Can be achieved by relying on all regional blocks get:

[0072] S42: Dynamic resolution adjustment strategy In order to adapt to the feature requirements of different scales in crack detection, AMPN designed a dynamic resolution adjustment strategy to flexibly process the resolution of feature maps at different levels of the feature pyramid to enhance the model's detection capabilities for large-scale and small cracks.

[0073] 1. Downsampling of large-scale feature maps: For large-scale crack feature extraction, a 2x2 downsampling convolution (stride 2) is used to reduce the feature map size to increase the receptive field.

[0074] 2. Upsampling of small-scale feature maps: Use an upsampling layer to restore the resolution of small crack features and retain the texture information of small cracks.

[0075] The steps for refining by combining the regional block-level dynamic fusion method are as follows: Direction detection calculation: For each area block, use a directional filter to perform edge detection and generate a direction matrix , which is used to determine the main direction of cracks within the regional block.

[0076]

[0077] Where DF stands for Direction Filter, which is a direction filter. Providing directional information is used to determine the shape of the strip convolution kernel suitable for the crack morphology, so that the filter can capture the continuous features along the crack direction.

[0078] Adaptive convolution kernel: according to the direction matrix , an adaptive convolution kernel is designed for each region block, which is applied to visible light image and temperature matrix feature extraction respectively:

[0079] Among them, AC stands for Adaptive Conv, which means that according to A convolution operation that dynamically adjusts the size and orientation of the convolution kernel.

[0080] In the design of the adaptive convolution kernel, the shape and size of the convolution kernel are dynamically adjusted according to the direction and width characteristics of the crack. For example: For the detected main crack direction as transverse cracks, the selected convolution kernel shape is 1×5 or 1×7 to ensure that the continuous crack features along the transverse direction are captured.

[0081] For vertical cracks or blocks with wider crack areas, the selected convolution kernel shape is 5×1 or 7×1 to enhance the feature capture in the vertical direction.

[0082] In crack areas with smaller widths, a smaller convolution kernel (such as 3×3) is selected to enhance the extraction of detail features.

[0083] This adaptive convolution kernel selection strategy ensures that the features of cracks of different shapes can be fully extracted in complex scenes, further enhancing the robustness of the model in crack detection.

[0084] Adaptive fusion weight calculation based on regional blocks: In each regional block, the adaptive fusion weight is calculated based on the temperature difference and edge clarity. .

[0085]

[0086] Among them, ES stands for Edge Sharpness, which is used to calculate the edge clarity of the visible light image features in the area block; TD stands for Temperature Difference, which is used to calculate the temperature difference of the temperature matrix features in the area block.

[0087] Dynamic fusion of regional blocks: based on adaptive fusion weights , the visible light and temperature matrix features in each area block are fused to obtain the fused feature block :

[0088] Finally, the fused overall feature map Can be achieved by relying on all regional blocks get:

[0089] AMPN's dynamic resolution adjustment strategy allows each layer of the feature pyramid to flexibly choose upsampling or downsampling according to the actual scale requirements of the cracks. In this way, the model can increase the receptive field when processing large-scale cracks while retaining detail resolution when detecting small cracks.

[0090] S43: Global feature enhancement Adaptive weighted fusion of regional blocks Dynamic integration with regional block level The global feature map is spliced ​​to form the final fusion feature map .

[0091]

[0092] In each area block, the multimodal feature information obtained by adaptive weighted fusion and dynamic fusion is retained at the same time, and the final fusion feature map is spliced.

[0093] Perform global pooling operations on multi-scale feature maps to generate globally enhanced multi-scale feature maps , so that the model can effectively aggregate global information in different modes and scales, thereby improving the crack detection accuracy and robustness to complex scenarios. Specifically: During crack detection, It is used to supplement the lack of environmental information in local features, thereby reducing false detection or missed detection due to changes in viewing angles or road conditions. After the global feature map is spliced ​​with the local area block features, the model's ability to identify cracks at all scales is improved, and the model's robustness in complex scenes (such as environments with a lot of background noise) is enhanced. Through global feature enhancement, the model can better understand the contrast between cracks and the surrounding background, ensuring higher accuracy and consistency when detecting crack edges.

[0094] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A road crack detection method using thermal-optical matrix fusion, characterized in that: The method comprises: acquiring a temperature matrix and a visible light image based on a visible light-temperature matrix acquisition device, extracting crack edge features through temperature gradient analysis, and performing binarization processing to obtain a temperature matrix binarization image; The temperature matrix binary image is fused with the visible light image through the thermal-optical matrix image fusion module to obtain a thermal-optical matrix fused crack image; The thermal-optical matrix fused crack image is used as the input of the YOLOv8-AMPN network model, and feature weight adaptive weighted fusion and regional block-level dynamic fusion are performed through the network model to obtain an enhanced thermal-optical matrix fused crack image; Crack identification is performed based on the enhanced thermal-optical matrix fusion crack image.

2. The road crack detection method of thermal-optical matrix fusion according to claim 1 is characterized in that: The visible light-temperature matrix acquisition device comprises: a visible light camera, a coaxial alignment adjustment rod, a push handle, a synchronous trigger, an infrared thermal imaging camera, a level, a laser indicator, and a movable triangular stabilization bracket.

3. The road crack detection method of thermal-optical matrix fusion according to claim 2 is characterized in that: The optical axes of the visible light camera and the infrared thermal imaging camera are adjusted to be parallel by the coaxial alignment rod and the level; the visible light camera and the infrared thermal imaging camera are adjusted to keep the field of view consistent at different focal lengths by the laser pointer.

4. A road crack detection method using thermal-optical matrix fusion according to any one of claims 1 to 3, characterized in that: The method of acquiring a temperature matrix and a visible light image based on a visible light-temperature matrix acquisition device, extracting crack edge features through temperature gradient analysis, and performing binarization processing to obtain a temperature matrix binarization image includes: Obtaining a visible light image, adjusting the visible light image to a preset resolution size, and unifying the image size; Acquire temperature matrix data based on visible light-temperature matrix acquisition device, generate grayscale image and calculate temperature matrix gradient for temperature matrix data to segment crack edge, further generate temperature matrix binary image to correspond to visible light image; Data enhancement is performed on the temperature matrix binary image and the visible light image.

5. The road crack detection method of thermal-optical matrix fusion according to claim 4 is characterized in that: The temperature matrix binary image is fused with the visible light image by the thermal-optical matrix image fusion module to obtain the thermal-optical matrix fused crack image, including: (1) Preliminary feature alignment The temperature matrix features and visible light image features are input into the thermal-optical matrix image fusion module respectively, and the shared convolutional alignment layer is used for feature exchange and feature alignment; (2) Feature complementarity enhancement The similarity score is calculated by the similarity calculation layer, and the enhancement between the temperature matrix features and the visible light image features is adjusted by the learnable parameters; (3) Multi-layer refinement fusion The enhanced features of the visible light image and the temperature matrix are fused layer by layer using the channel-level and space-level attention mechanisms. (4) Thermal-optical enhancement module Through the preset convolution layer, the fused features are refined and extracted; The contribution of each modality to the final feature is calculated using a complementary weight layer, and a fused feature is obtained based on the contribution; The fused features are input into the saliency prediction layer to obtain a crack saliency map.

6. A road crack detection method using thermal-optical matrix fusion according to claim 1 or 5, characterized in that: The method uses the thermal-optical matrix fused crack image as the input of the YOLOv8-AMPN network model, performs feature weight adaptive weighted fusion and regional block-level dynamic fusion through the network model, and obtains an enhanced thermal-optical matrix fused crack image, including: S41: Multi-scale convolution kernel fusion; S42: Dynamic resolution adjustment; S43: Global feature enhancement.

7. The road crack detection method of thermal-optical matrix fusion according to claim 6 is characterized in that: The multi-scale convolution kernel fusion includes: Introduce convolution kernels of different sizes into the convolution module of the YOLOv8-AMPN network model; In the YOLOv8-AMPN network model, the feature maps extracted by different convolution kernels are spliced ​​in different modes to achieve multi-scale fusion of visible light features and temperature matrix features; In the spliced ​​multi-scale features, according to the differences between the visible light features and the temperature matrix features, the feature contributions of the two modalities are adjusted through feature weight adaptive weighted fusion to achieve multi-modal feature expression; The feature weight adaptive weighted fusion includes: Feature segmentation: segment the visible light image and temperature matrix features into regional blocks; The adaptive weight matrix is ​​generated by calculating the average brightness and temperature difference of the area block: For each area block, the visible light image and temperature matrix features are weighted fused through an adaptive weight matrix; By fusing feature blocks of all regional blocks, a fused overall feature map is obtained.

8. The road crack detection method of thermal-optical matrix fusion according to claim 7 is characterized in that: The dynamic resolution adjustment includes: For each area block, a directional filter is used to perform edge detection and generate a direction matrix to determine the main direction of the cracks in the area block; According to the direction matrix, an adaptive convolution kernel is designed for each region block, which is applied to the visible light image and temperature matrix feature extraction respectively; in the design of the adaptive convolution kernel, the shape and size of the convolution kernel are dynamically adjusted according to the direction and width characteristics of the crack; In each area block, the adaptive fusion weight is calculated according to the temperature difference and edge definition; Based on the adaptive fusion weight, the visible light and temperature matrix features in each area block are fused to obtain a fused feature block: A dynamically fused overall feature map is obtained based on all region blocks.

9. The road crack detection method of thermal-optical matrix fusion according to claim 8 is characterized in that: The global feature enhancement includes: The overall feature map of the region block adaptive weighted fusion is spliced ​​with the overall feature map of the region block level dynamic fusion to form a multi-scale feature map; A global pooling operation is performed on the multi-scale feature map to generate a globally enhanced multi-scale feature map.

Images