Road crack detection system and method fusing infrared and visible light images
Through the detection system that fuses infrared and visible light images, the convolution module and feature enhancement module extract and enhance features, and the loss function module is optimized, which solves the problem of low recognition accuracy of road crack detection in low light environments, achieving high accuracy Road crack detection.
Patent Information
- Application Number
- CN202410966233.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-19
AI Technical Summary
The existing road crack detection technology has lower recognition accuracy in low visibility or night scenes, making it impossible to effectively detect road cracks.
Using a high-precision detection system that combines infrared and visible light images, the thermal imaging information and visible light imaging information are obtained through the acquisition module, and the convolution module performs feature extraction and fusion, the feature enhancement module enhances feature map, and the loss function module optimizes the shape and position of the feature map , realize high-precision detection of road cracks.
In low visibility and night scenes, the identification accuracy of road cracks is significantly improved, the feature expression ability is enhanced, the accuracy of crack shape and position is improved, and the detection under complex environmental conditions is adapted to the detection.
Smart Images

Figure CN118982465A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a high-precision road crack detection system and method that integrates infrared and visible light images, and belongs to the technical field of road safety monitoring. Background Art
[0002] With the rapid development of economy and technology, the transportation industry has developed rapidly, and traffic safety is closely related to people's lives. Road cracks pose serious safety hazards. Existing road crack detection technology mainly relies on visible light images. However, the recognition effect of this solution is not ideal in complex environments. For example, in low visibility conditions, the recognition accuracy of visible light images will decrease, and it is completely unusable in night scenes. Therefore, there is an urgent need for a high-precision road crack detection system that can improve recognition accuracy in low light conditions. Summary of the invention
[0003] In order to solve the problem that the recognition accuracy of visible light images decreases under low visibility conditions and the visible light images are completely unusable in night scenes, the present invention further proposes a high-precision road crack detection system and method that integrates infrared and visible light images.
[0004] The technical solution adopted by the present invention to solve the above-mentioned problem is: the present invention includes: an acquisition module, a convolution module, a feature enhancement module and a loss function module; the acquisition module is used to acquire thermal imaging information and visible light imaging information; the convolution module is used to perform feature extraction and feature fusion on the thermal imaging information and the visible light image to obtain a visible light feature map and an infrared feature map, and perform feature fusion on the visible light feature map and the infrared feature map to obtain a first feature map; the feature enhancement module is used to enhance the first feature map to obtain a first enhanced feature map; the loss function module is used to respectively optimize the shape and position of the first enhanced feature map.
[0005] Furthermore, the convolution module includes a first bottleneck unit and a second bottleneck unit; the first bottleneck unit and the second bottleneck unit are connected in series; the length and width of the convolution kernel of the first bottleneck unit are both 3; the length and width of the convolution kernel of the second bottleneck unit are both 3.
[0006] Furthermore, the convolution module includes a residual connection unit, which is used to add the visible light feature map and the infrared feature map to the thermal imaging information and the visible light imaging information to obtain a first feature map.
[0007] Furthermore, the feature enhancement module includes: an attention feature enhancement unit, an attention mechanism unit, an identity mapping unit and a fusion unit; the attention feature enhancement unit is used to screen the most critical channels and spatial areas in the visible light and infrared feature maps by sequentially applying channel attention and spatial attention to obtain a screened feature map; the attention mechanism unit is used to capture the global spatial context features in the feature map by segmenting the information in the horizontal and vertical directions, further enhance the expression of the features, and obtain a global feature map; the identity mapping unit is used to directly transfer the acquired visible light feature map and infrared feature map, keep the input features unchanged, and obtain an identity mapping feature map; the fusion unit is used to perform dimensionality reduction processing on the merged feature map, and add the screened feature map and the global feature map to the identity mapping feature map at a depth level through a linear transformation to obtain a first enhanced feature map.
[0008] Furthermore, the loss function module includes: a detection unit and a penalty unit; the detection unit is used to detect the ratio of the intersection and union between the predicted bounding box and the real bounding box, and the distance between the center points of the predicted box and the real box; the penalty unit is used to measure the consistency of the aspect ratio of the predicted box and the real box.
[0009] Another object of the present invention is to provide a high-precision road crack detection method that fuses infrared and visible light images, specifically comprising: step 100: acquiring thermal imaging information and visible light imaging information through an acquisition module; step 200: using the convolution module to perform feature extraction and feature fusion on the thermal imaging information and the visible light image; step 300: enhancing the first feature map with the help of a feature enhancement module; step 400: optimizing the shape and position of the first enhanced feature map respectively through the loss function module.
[0010] Furthermore, the convolution module is used to perform feature extraction and feature fusion on the thermal imaging information and the visible light image, and the specific steps include: step 201: the first bottleneck unit and the second bottleneck unit are processed in series, the length and width of the convolution kernel of the first bottleneck unit are both 3; the length and width of the convolution kernel of the second bottleneck unit are both 3, and a visible light feature map and an infrared feature map are obtained; step 202: the visible light feature map and the infrared feature map are added to the thermal imaging information and the visible light imaging information through a residual connection unit to obtain a first feature map.
[0011] Further, step 301: through the attention feature enhancement unit, the order of channel attention and spatial attention is applied to filter key channels and spatial areas to obtain a filtered feature map; step 302: through the attention mechanism unit, the horizontal and vertical direction information is segmented to capture the global spatial context features to obtain a global feature map; step 303: through the identity mapping unit, the visible light feature map and the infrared feature map are directly transferred to obtain an identity mapping feature map; step 304: through the fusion unit, the merged feature map is reduced in dimension and linearly transformed to add and fuse the filtered feature map, the global feature map and the identity mapping feature map to obtain a first enhanced feature map.
[0012] Furthermore, the first feature map is enhanced with the help of a feature enhancement module, and the specific steps include: detecting the ratio of the intersection and union between the predicted bounding box and the true bounding box and the distance between the center points of the predicted box and the true box by the detection unit, and measuring the consistency of the aspect ratio of the predicted box and the true box by the penalty item unit.
[0013] The beneficial effects of the present invention are as follows: compared with the prior art, the present invention can detect road cracks in scenes with low visibility and at night by fusing infrared imaging information with visible light image information, and can accurately extract crack features through convolution modules and feature enhancement modules. The loss function module further optimizes the model's capture of crack features and regression of bounding boxes, which can effectively improve the recognition accuracy of road cracks in scenes with low visibility and at night. The present invention can better detect road cracks under complex environmental conditions such as different lighting and weather by combining a variety of image information. By means of attention mechanisms and other means, key feature areas can be effectively screened and feature expression can be enhanced, which can effectively improve the accuracy of crack recognition. Finally, the loss function module can be used to optimize the shape and position of the feature map in a targeted manner, further improving the accuracy of crack shape and position judgment. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 It is a structural diagram of a high-precision road crack detection system that integrates infrared and visible light images according to the present invention;
[0015] Figure 2 It is a structural schematic diagram of a feature enhancement module of a high-precision road crack detection system that integrates infrared and visible light images according to the present invention;
[0016] Figure 3 It is a functional structure diagram of a loss function module of a high-precision road crack detection system that integrates infrared and visible light images according to the present invention;
[0017] Figure 4It is a schematic diagram of the overall network structure of a high-precision road crack detection system that integrates infrared and visible light images according to the present invention;
[0018] Figure 5 It is a structural schematic diagram of a rolling machine module of a high-precision road crack detection system integrating infrared and visible light images according to the present invention;
[0019] Figure 6 It is a flow chart of a high-precision road crack detection method integrating infrared and visible light images according to the present invention;
[0020] Figure 7 It is a flow chart of using the convolution module to extract and fuse features of thermal imaging information and visible light images in a high-precision road crack detection method that fuses infrared and visible light images described in the present invention;
[0021] Figure 8 It is a flow chart of enhancing the first feature map by means of a feature enhancement module in a high-precision road crack detection method that fuses infrared and visible light images described in the present invention. DETAILED DESCRIPTION
[0022] Specific implementation method 1: Figures 1 to 5 As shown, a high-precision road crack detection system that integrates infrared and visible light images includes: an acquisition module, a convolution module, a feature enhancement module and a loss function module; the acquisition module is used to acquire thermal imaging information and visible light imaging information; the convolution module is used to perform feature extraction and feature fusion on the thermal imaging information and the visible light image to obtain a visible light feature map and an infrared feature map, and perform feature fusion on the visible light feature map and the infrared feature map to obtain a first feature map; the feature enhancement module is used to enhance the first feature map to obtain a first enhanced feature map; the loss function module is used to optimize the shape and position of the first enhanced feature map respectively.
[0023] Among them, Figure 4As shown, thermal imaging information and visible light imaging information are obtained through an acquisition module, and corresponding sensors or devices are used to obtain these two types of images. The thermal imaging information and visible light images are extracted through a convolution module. In this embodiment, a new convolution module C2fv1k9 is used to extract feature data, wherein v1 represents its version number, and k9 represents its kernel (real-time operating system), which theoretically covers a 9x9 receptive field. The initial input of C2fv1k9 passes through a layer (CBS, 1x1, s1) that combines convolution, batch normalization, and activation functions to extract features and transform dimensions to obtain feature data; then the feature data is shunted to four different depth-separable convolution layers (DWConv) for deep feature extraction, which capture features of different scales through different kernel sizes (3x3 or 5x5) and dilation rates (d1-d4). The outputs of these branches are spliced together to integrate information at each scale, and then processed by another CBS layer; the extracted features are fused to obtain a visible light feature map and an infrared feature map, and the visible light feature map and the infrared feature map are fused to obtain the first feature map. The first feature map is enhanced by the feature enhancement module to improve the expressiveness of the features; finally, the loss function module is used to optimize the shape and position of the first bounding box of the visible light feature map and the shape and position of the second bounding box of the infrared feature map.
[0024] Compared with the existing technology, the convolution module and feature enhancement module provide the model with powerful feature extraction capabilities, and the loss function module further optimizes the model's capture of crack features and bounding box regression. Through these improvements, the model has made significant progress in accuracy and reliability, especially in complex multimodal image data, achieving a leap in performance. Experimental results show that the new model has achieved significant progress in key performance indicators such as precision, recall rate (measurement of the proportion of positive samples correctly identified by the model) and mAP@50-95 (accuracy between thresholds 50-95). Compared with the previous model, the precision increased from 0.885 to 0.924, the recall rate increased from 0.876 to 0.916, and the mAP@50-95 also jumped from 0.711 to 0.728.
[0025] Optionally, the convolution module includes a first bottleneck unit and a second bottleneck unit; the first bottleneck unit and the second bottleneck unit are connected in series; the length and width of the convolution kernel of the first bottleneck unit are both 3; the length and width of the convolution kernel of the second bottleneck unit are both 3.
[0026] like Figure 5As shown, the convolution module extracts feature data through a bottleneck unit. In this embodiment, the bottleneck unit adopts a C2fv1k9 unit. The C2fv1k9 unit refers to two 3x3 convolutions with residual connections defined as a first bottleneck unit and a second bottleneck unit. The first bottleneck unit and the second bottleneck unit are connected in series, so that C2fv1k9 can capture more feature information, thereby improving the feature representation ability of the model; at the same time, it can better adapt to the structure of the ResNet block, thereby using residual connections to improve the performance of the deep network.
[0027] Compared with the existing technology, the existing bottleneck unit consists of two layers of ordinary 3×3 convolution-BN-SiLU, which has a large number of parameters and a limited receptive field; the improved bottleneck unit is codenamed C2fv1k9, where v1 represents its version number and k9 represents that its kernel theoretically covers a 9×9 receptive field. The C2fv1k9 bottleneck unit uses deep convolutions with different expansion rates to provide the network with a variety of receptive fields, which is beneficial for the network to extract multi-scale visible light and infrared features, and has the advantage of a small number of parameters, which improves the accuracy and robustness of the model while maintaining computational efficiency.
[0028] Optionally, the convolution module includes a residual connection unit, which is used to add the visible light feature map and the infrared feature map to the thermal imaging information and the visible light imaging information to obtain a first feature map.
[0029] The convolution module includes a residual connection unit, which is used to add the visible light feature map and the infrared feature map to the thermal imaging information and the visible light imaging information to obtain a first feature map.
[0030] Compared with the prior art, the present invention can complement the information in the infrared features and the visible light image, thereby improving the accuracy of crack detection.
[0031] Optional, such as Figure 2 As shown, the feature enhancement module includes: an attention feature enhancement unit, an attention mechanism unit, an identity mapping unit and a fusion unit; the attention feature enhancement unit is used to screen the most critical channels and spatial areas in the visible light and infrared feature maps by sequentially applying channel attention and spatial attention to obtain a screened feature map; the attention mechanism unit is used to capture the global spatial context features in the feature map by segmenting the information in the horizontal and vertical directions, further strengthen the expression of the features, and obtain the global feature map; the identity mapping unit is used to directly transfer the obtained visible light feature map and infrared feature map, keep the input features unchanged, and obtain the identity mapping feature map; the fusion unit is used to perform dimensionality reduction processing on the merged feature map, and add the screened feature map and the global feature map with the identity mapping feature map at the depth level through linear transformation to obtain the first enhanced feature map.
[0032] Among them, the attention feature enhancement unit learns the importance weight of each channel through the channel attention mechanism and selects the most critical channel in the feature map; and selects the most critical spatial area in the feature map by learning the importance weight of each spatial position; and focuses on the horizontal and vertical directions of the attention mechanism unit: by splitting the horizontal and vertical information, the global spatial context in the feature map is captured to further enhance the expression of the features; the feature maps processed by CBAM (attention feature enhancement unit) and Coordinate Attention (attention mechanism unit) are added and fused with the output of the identity mapping branch at a deep level, combining the advantages of each attention mechanism and retaining the information of the original features; providing an enhanced feature set for downstream tasks, improving the representativeness of the features, and thus improving the accuracy of detection.
[0033] Among them, the CorAF2n1 architecture is used in the feature enhancement unit to optimize the channel and spatial attention respectively. The structural expression of the separate attention optimization is:
[0034]
[0035] In formula (1), F represents the linear or nonlinear transformation function after concatenation; Concat represents the concatenation operation, which connects the features; represents the attention enhancement feature of the i-th input feature; Represents the infrared enhanced feature of the i-th input feature.
[0036] The enhanced visible light and infrared feature sets are merged through the feature splicing strategy in the depth direction; then, the convolution block (CBS: 1x1 convolution, batch normalization and SiLU activation function) is used to reduce the dimension of the merged feature map, and the fusion of different features is achieved through linear transformation.
[0037] Optional, such as Figure 3 As shown, the loss function module includes: a detection unit and a penalty unit; the detection unit is used to detect the ratio of the intersection and union between the predicted bounding box and the real bounding box, and the distance between the center points of the predicted box and the real box; the penalty unit is used to measure the consistency of the aspect ratio of the predicted box and the real box.
[0038] Among them, the loss function detection unit is configured to detect the ratio of the intersection and union between the predicted bounding box and the real bounding box, and the distance between the center points of the predicted box and the real box; the penalty item unit is configured to measure the consistency of the aspect ratio of the predicted box and the real box to optimize the prediction of the bounding box in the target detection task, wherein the loss function adopted in this embodiment is the CDIoU5 loss function.
[0039] In CDIoU5, the calculation of the penalty term is redefined by introducing the squared distance between the upper left and lower right vertices as part of the penalty term; this consideration of diagonal distance allows the loss function to capture more geometric information, which can theoretically provide more accurate bounding box adjustment guidance.
[0040] CDIOU5, the expression is:
[0041]
[0042] In formula (2), IoU stands for intersection over union, which is the ratio of the intersection and union of the predicted box and the true box; ρ 2 (b p ,b gt ) represents the square of the Euclidean distance between the center point of the predicted box and the center point of the real box; Represents the square of the width and height of the minimum enclosing rectangle that can contain both the predicted box and the true box; αv represents the weight used to adjust the penalty term, where α is a coefficient for balance and v is a term that reflects the consistency of the aspect ratio.
[0043]
[0044] In formula (3), v represents a penalty term used to evaluate the similarity of aspect ratio;
[0045] and Represents the square of the width difference and height difference between the predicted box and the true box;
[0046]
[0047] In formula (4), α represents the ratio that determines the weight of the aspect ratio consistency penalty in the total loss to ensure that the aspect ratio is only considered when the IoU value is high;
[0048]
[0049] |A∩B| represents the intersection area of the predicted box and the true box;
[0050] |A∪B| represents the union area of the predicted box and the true box.
[0051]
[0052] In formula (6) and formula (7), and These are the squared horizontal and vertical distances between the center points of the predicted box and the true box and are used to calculate the deviation of the position.
[0053] like Figure 6 As shown, another object of the present invention is to provide a high-precision road crack detection method that fuses infrared and visible light images, specifically comprising: step 100: acquiring thermal imaging information and visible light imaging information through an acquisition module; step 200: using the convolution module to perform feature extraction and feature fusion on the thermal imaging information and the visible light image; step 300: enhancing the first feature map with the help of a feature enhancement module; step 400: optimizing the shape and position of the first enhanced feature map respectively through the loss function module.
[0054] Among them, the acquisition module accurately and in real time collects thermal imaging information and visible light imaging information; the convolution module processes thermal imaging and visible light images through the convolutional neural network algorithm, extracts key features and fuses them; the feature enhancement module highlights important features; finally, the loss function module measures the difference between the predicted result and the true value by defining a specific loss function to achieve optimization of shape and position.
[0055] Compared with existing technologies, by acquiring two different types of imaging information, a more comprehensive scene understanding can be provided. Thermal imaging can be used to detect temperature differences, and visible light imaging provides conventional visual information, complementing each other's advantages. Feature extraction and fusion help to comprehensively utilize the features of the two images and improve the model's ability to express complex scenes, thereby improving the accuracy and precision of model detection.
[0056] Optionally, the convolution module is used to perform feature extraction and feature fusion on the thermal imaging information and the visible light image, and the specific steps include: Step 201: Processing the first bottleneck unit and the second bottleneck unit in series, the length and width of the convolution kernel of the first bottleneck unit are both 3; the length and width of the convolution kernel of the second bottleneck unit are both 3, and a visible light feature map and an infrared feature map are obtained; Step 202: Adding the visible light feature map and the infrared feature map to the thermal imaging information and the visible light imaging information through a residual connection unit to obtain a first feature map.
[0057] Among them, the first bottleneck unit and the second bottleneck unit of the convolution kernel (3x3) are used to process the thermal imaging information and the visible light image respectively, and their respective features are extracted to obtain the visible light feature map and the infrared feature map; the series processing is helpful to gradually extract and refine the features; the residual connection unit is used to add the extracted features including: visible light feature map and infrared feature map; to the original thermal imaging information and visible light imaging information, which can better retain the original information and avoid the loss of important information in the process of feature extraction and fusion. At the same time, it also helps the back propagation of gradients, making the model training more stable.
[0058] Compared with the existing technology, the use of convolution kernels of specific sizes can effectively capture features of different scales. The 3x3 convolution kernel is more commonly used and can extract more local and detailed features. The residual connection enables the model to better learn small changes in features, thereby improving the model's expressiveness and generalization capabilities. By adding features to the original information, it helps to retain key information in the original image, so that it can be analyzed and identified more accurately in subsequent processing, realizing the detection of complex scenes.
[0059] Optionally, the first feature map is enhanced with the help of a feature enhancement module, and the specific steps include: step 301: through the attention feature enhancement unit, the order of channel attention and spatial attention is applied to filter key channels and spatial areas to obtain a filtered feature map; step 302: through the attention mechanism unit, the horizontal and vertical information is split to capture the global spatial context features to obtain a global feature map; step 303: through the identity mapping unit, the visible light feature map and the infrared feature map are directly transferred to obtain an identity mapping feature map; step 304: through the fusion unit, the merged feature map is reduced in dimension and linearly transformed to add and fuse the filtered feature map, the global feature map and the identity mapping feature map to obtain a first enhanced feature map.
[0060] Among them, the attention feature enhancement unit is used to screen key channels and spatial areas through the processing order of channel attention and spatial attention, so as to obtain a screened feature map, which helps to highlight important feature information. The attention mechanism unit splits the horizontal and vertical information to capture the global spatial context features and obtain the global feature map, so as to obtain a more comprehensive feature representation; the identity mapping unit directly transmits the visible light feature map and the infrared feature map to retain the original feature information; the fusion unit performs dimensionality reduction and linear transformation on the merged feature map, and then adds and fuses the screened feature map, the global feature map and the identity mapping feature map to obtain the first enhanced feature map by integrating multiple features.
[0061] Compared with existing technologies, it can more effectively extract and integrate features, highlight key information, and improve the expressiveness of features; and through identity mapping feature maps, it can effectively prevent the loss of important original information and thus improve the performance and accuracy of the entire system.
[0062] Optionally, the first feature map is enhanced with the help of a feature enhancement module, and the specific steps include: detecting the ratio of the intersection and union between the predicted bounding box and the true bounding box and the distance between the center points of the predicted box and the true box by the detection unit, and measuring the consistency of the aspect ratio of the predicted box and the true box by the penalty item unit.
[0063] Among them, the detection unit calculates the ratio of the intersection and union of the predicted bounding box and the true bounding box (usually called the intersection over union, IoU) to evaluate the accuracy of the prediction; at the same time, it calculates the distance between the center points of the predicted box and the true box; the penalty unit focuses on analyzing and measuring the consistency of the aspect ratio of the predicted box and the true box, and the calculation method is used to quantify the degree of this consistency.
[0064] Compared with the prior art, the present invention can more comprehensively and accurately evaluate the degree of fit between the prediction results and the actual situation; improve the accuracy of detection and prediction, so that the feature enhancement module can play a better role, thereby improving the performance and effect of related applications as a whole.
[0065] Among them, first, thermal imaging information and visible light imaging information are collected through the acquisition module; then the first bottleneck unit and the second bottleneck unit in the convolution module are used to extract features and preliminarily fuse these information, and then further fuse them through the residual connection unit to obtain the first feature map; the attention feature enhancement unit, the attention mechanism unit, the identity mapping unit, etc. in the feature enhancement module play their roles respectively to screen key channels and spatial areas, capture global features, maintain original features, etc., and finally fuse them to obtain the enhanced first enhanced feature map; the loss function module optimizes and constrains the shape and position of the first enhanced feature map through the detection unit and the penalty item unit respectively to ensure the accuracy of the detection results in terms of shape and position, thereby achieving high-precision detection of road cracks, and comprehensively utilizing the respective advantages of infrared and visible light images to improve the overall detection performance.
[0066] Compared with the prior art, the present invention can obtain road crack information more comprehensively by fusing infrared and visible light images, thereby improving the accuracy of detection; and through specially designed convolution modules and feature enhancement modules, it can better realize the extraction and enhancement of crack-related features; the present invention combines a variety of image information to better adapt to detection under complex environmental conditions such as different lighting and weather; through means such as the attention mechanism, it can effectively screen key feature areas and strengthen feature expression, which can effectively improve the accuracy of crack identification; finally, through the loss function module, it can optimize the shape and position of the feature map in a targeted manner, further improving the accuracy of crack shape and position judgment.
[0067] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.
Claims
1. A road crack detection system integrating infrared and visible light images, characterized in that: Includes acquisition module, convolution module, feature enhancement module and loss function module; The acquisition module is used to acquire thermal imaging information and visible light imaging information; The convolution module is used to perform feature extraction and feature fusion on the thermal imaging information and the visible light image to obtain a visible light feature map and an infrared feature map, and perform feature fusion on the visible light feature map and the infrared feature map to obtain a first feature map; The feature enhancement module is used to enhance the first feature map to obtain a first enhanced feature map; The loss function module is used to optimize the shape and position of the first enhanced feature map respectively.
2. A road crack detection system integrating infrared and visible light images according to claim 1, characterized in that: The convolution module includes a first bottleneck unit and a second bottleneck unit; The first bottleneck unit and the second bottleneck unit are connected in series; The length and width of the convolution kernel of the first bottleneck unit are both 3; the length and width of the convolution kernel of the second bottleneck unit are both 3.
3. A road crack detection system integrating infrared and visible light images according to claim 2, characterized in that: The convolution module includes a residual connection unit, which is used to add the visible light feature map and the infrared feature map to the thermal imaging information and the visible light imaging information to obtain a first feature map.
4. The road crack detection system integrating infrared and visible light images according to claim 1, characterized in that: The feature enhancement module includes an attention feature enhancement unit, an attention mechanism unit, an identity mapping unit and a fusion unit; The attention feature enhancement unit is used to screen the most critical channels and spatial regions in the visible light and infrared feature maps by sequentially applying channel attention and spatial attention to obtain a screened feature map; The attention mechanism unit is used to capture the global spatial context features in the feature map by segmenting the information in the horizontal and vertical directions, further strengthen the expression of the features, and obtain the global feature map; The identity mapping unit is used to directly transfer the obtained visible light feature map and the infrared feature map, keep the input features unchanged, and obtain an identity mapping feature map; The fusion unit is used to perform dimensionality reduction processing on the merged feature map, and add and fuse the screening feature map, the global feature map and the identity mapping feature map at a depth level through linear transformation to obtain a first enhanced feature map.
5. The road crack detection system integrating infrared and visible light images according to claim 1, characterized in that: The loss function module includes: a detection unit and a penalty item unit; The detection unit is used to detect the ratio of the intersection and union between the predicted bounding box and the real bounding box, and the distance between the center points of the predicted box and the real box; The penalty unit is used to measure the consistency of the aspect ratio of the predicted box and the real box.
6. A road crack detection method integrating infrared and visible light images, characterized in that: The specific steps include: Step 100: Acquire thermal imaging information and visible light imaging information through an acquisition module; Step 200: using the convolution module to perform feature extraction and feature fusion on the thermal imaging information and the visible light image; Step 300: enhancing the first feature map by means of a feature enhancement module; Step 400: Optimizing the shape and position of the first enhanced feature map respectively through the loss function module.
7. The road crack detection method of fusing infrared and visible light images according to claim 6, characterized in that: The step of using the convolution module to extract and fuse features of the thermal imaging information and the visible light image specifically includes: Step 201: Connect the first bottleneck unit and the second bottleneck unit in series, wherein the length and width of the convolution kernel of the first bottleneck unit are both 3; the length and width of the convolution kernel of the second bottleneck unit are both 3, and obtain a visible light feature map and an infrared feature map; Step 202: Add the visible light feature map and the infrared feature map to the thermal imaging information and the visible light imaging information through a residual connection unit to obtain a first feature map.
8. The road crack detection method of fusing infrared and visible light images according to claim 6, characterized in that: The step of enhancing the first feature map by means of a feature enhancement module specifically comprises: Step 301: by using the attention feature enhancement unit to select the sequence of channel attention and spatial attention, a screening feature map is obtained by applying screening key channels and spatial regions; Step 302: Capture global spatial context features by segmenting horizontal and vertical information through an attention mechanism unit to obtain a global feature map; Step 303: directly transferring the visible light feature map and the infrared feature map through an identity mapping unit to obtain an identity mapping feature map; Step 304: Perform dimension reduction processing on the merged feature map through a fusion unit and perform linear transformation to add and fuse the screening feature map, the global feature map and the identity mapping feature map to obtain a first enhanced feature map.
9. The road crack detection method of fusing infrared and visible light images according to claim 6, characterized in that: The step of enhancing the first feature map by means of a feature enhancement module specifically comprises: The detection unit detects the ratio of the intersection and union between the predicted bounding box and the real bounding box and the distance between the center points of the predicted box and the real box, and the penalty unit measures the consistency of the aspect ratio of the predicted box and the real box.
Citation Information
Patent Citations
Defect detection method combining attention mechanism and adaptive memory fusion network
CN114372968A
Visible light and infrared image fusion enhancement method and system in low-light environment
CN115063329A
Tea disease detection method and related device
CN115294467A
Target detection method based on attention dual-mode feature fusion
CN118155035A
Cited By
Lithium battery electrode raw material feeding device and lithium battery production system and method
CN119793308A
Lithium battery electrode raw material feeding device, lithium battery production system and method thereof
CN119793308B
Road crack detection method based on thermo-optical matrix fusion
CN119919416A
Pavement crack detection equipment based on double spectrums
CN121095246A
