Illumination perception lightweight defect detection method, system and equipment in low-light environment

Abstract

The invention provides an illumination perception lightweight defect detection method, system and equipment in a low-light environment, and belongs to the field of intelligent inspection and automatic defect identification. According to the method, an illumination perception lightweight YOLO framework is constructed, and the following operations are executed in sequence: obtaining a low-light environment image; low-light enhancement and lightweight feature extraction are realized on a feature level by using an illumination adaptive enhancement feature extraction network; performing multi-scale feature fusion through a hybrid path aggregation network; and outputting a detection result based on a dynamic space attention mechanism by adopting a dynamic feature fusion detection head. By adopting the illumination perception lightweight defect detection method, system and equipment in the low-light environment, the problems of poor image quality, large model calculation amount, small target leak detection and strong background interference in the low-light environment are solved, and the method, system and equipment have the advantages of high detection precision, few model parameters and high reasoning speed; the system is suitable for intelligent inspection of infrastructures such as sewers and pipe galleries.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection and automatic defect identification, and in particular to a lightweight defect detection method, system and device based on illumination sensing in low-light environments. Background Technology

[0002] Regular inspections of enclosed or semi-enclosed infrastructure such as sewers, underground utility tunnels, and other similar structures are crucial for ensuring the safe operation of cities. Traditional manual inspection methods suffer from drawbacks such as low efficiency, high risk, and strong subjectivity. With the development of computer vision and mobile robotics technologies, vision-based automatic defect detection methods, especially real-time detection systems mounted on mobile platforms such as remotely controlled unmanned vehicles, have become important alternatives.

[0003] However, in the aforementioned practical application scenarios, defect detection tasks face a series of unique and severe technical challenges: Low light and uneven lighting issues: The target environment is usually dark inside, and there may be strong contrasts between light and dark caused by local light sources (such as fill lights). Low light will lead to a decrease in image signal-to-noise ratio and loss of details; uneven lighting makes it difficult for traditional image processing algorithms to reliably extract effective features, resulting in missed or false detections of defects.

[0004] High real-time processing requirements: The unmanned inspection vehicle needs to analyze and respond to the video streams captured by the camera in real time so that operators can obtain results in a timely manner or the system can autonomously avoid obstacles. Many existing high-performance detection models (such as large models based on Transformer) have high computational complexity and a large number of parameters, making it difficult to meet the high frame rate real-time inference requirements under the limited computing resources of airborne embedded devices.

[0005] Multi-scale and small-target characteristics of defects: The size of defects to be detected (such as cracks, corrosion, and leaks) varies greatly, and most of them are small targets. When traditional feature pyramid networks fuse multi-scale features, the high-resolution features in the shallow layers (containing details of small targets) are easily diluted during the transmission to deeper layers, resulting in poor small target detection performance.

[0006] Complex background interference: The inner wall of the pipe may have complex textures such as stains, watermarks, and moss, which are highly similar to the defect features, forming strong background interference and easily causing the detection model to generate false alarms.

[0007] To address these challenges, existing technologies have explored various approaches: In terms of models, the YOLO series is widely used in the field of real-time detection due to its excellent balance between speed and accuracy, but its native design is not robust enough to low-light conditions; traditional image enhancement algorithms such as Retinex can improve visual quality, but they are usually used as independent preprocessing steps and are not jointly optimized end-to-end with deep learning detection models.

[0008] In terms of attention mechanisms and feature fusion, spatial attention or channel attention modules have been introduced to improve feature selectivity, but the mechanism for dynamically adapting to different lighting and backgrounds is still imperfect; the feature pyramid structure is constantly evolving to improve multi-scale fusion capabilities, but its computational efficiency still has room for optimization.

[0009] In terms of system deployment, many existing solutions still send the acquired images back to the cloud server for processing, resulting in detection latency of up to hundreds of milliseconds, which cannot meet the needs of real-time remote control or instant response. Although there have been attempts to deploy models on edge devices, significant compromises are often required between accuracy, speed and model size.

[0010] Therefore, there is an urgent need for a target detection method and system that is specifically optimized for low-light environments, balances high accuracy and high real-time performance, and can be lightweightly deployed on mobile edge platforms to reliably and efficiently complete the intelligent inspection tasks of infrastructure. Summary of the Invention

[0011] The purpose of this invention is to provide a lightweight defect detection method, system and device for illumination sensing in low-light environments, in order to solve the problems of existing detection methods, such as difficulty in feature extraction under low light and uneven illumination conditions, large model computation, difficulty in real-time operation on mobile edge devices, and insufficient robustness to small targets and complex background interference.

[0012] To achieve the above objectives, the present invention provides a lightweight defect detection method based on illumination sensing in low-light environments, comprising the following steps: Step S1: Acquire image data collected in low-light environments; Step S2: Input the image data into the light-aware lightweight YOLO framework for object detection; The light-aware lightweight YOLO framework includes: An illumination-adaptive enhancement feature extraction network is used to achieve low-light enhancement and lightweight feature extraction at the feature level. Hybrid path aggregation network for multi-scale feature fusion; A dynamic feature fusion detection head is used to output detection results based on a dynamic spatial attention mechanism; Step S3: Output the detection results, including target category, location, and confidence level.

[0013] Preferably, the illumination adaptive enhancement feature extraction network includes an illumination analysis and enhancement module and an efficient convolutional block connected in sequence; The illumination analysis and enhancement module is used to learn the illumination distribution feature map from the input image and adaptively weight the input features; The efficient convolutional block uses a combination of partial convolution and pointwise convolution for lightweight feature extraction.

[0014] Preferably, the illumination analysis and enhancement module includes: The global average pooling layer is used to perform global average pooling on the input image to obtain a preliminary illumination estimation map. The stitching layer is used to stitch the preliminary illumination estimation map with the input image along the channel dimension; Light quantum networks are used to process the spliced ​​feature maps to generate illumination feature maps, which include, in sequence, […]. Convolutional layers, GELU activation function, layer normalization layers, grouped convolutional layers, and channel shuffling layers; The weighting module is used to activate the illumination feature map with the Sigmoid function, and then multiply it element-wise with the input feature map to be enhanced, outputting the enhanced feature map. Efficient convolutional blocks perform the following operations for feature extraction: For the previous input feature map Each channel executes Partial convolution, where This represents the total number of input channels. The output feature map of the partial convolution is concatenated with the remaining channels of the input feature map that have not undergone convolution. Perform on the stitched feature map Pointwise convolution is performed to obtain the output feature map.

[0015] Preferably, the hybrid path aggregation network includes hybrid convolutional modules and cross-layer shortcut connections; The hybrid convolution module adopts a combination structure of partial convolution and pointwise convolution, and a channel shuffling operation is followed by pointwise convolution; partial convolution supports convolution operations on the first 1 / 4, 1 / 2 or 3 / 4 channels of the input feature map; Cross-layer shortcut connections are used to upsample the high-resolution feature maps output by shallow networks and add them to the low-resolution feature maps output by deep networks to achieve feature fusion.

[0016] Preferably, the dynamic feature fusion detection head includes: The dynamic spatial attention module is used to perform global pooling along the horizontal and vertical directions respectively to generate a direction-aware attention weight map; The feature weighted fusion module is used to reweight and fuse multi-scale features based on the attention weight map.

[0017] Preferred low-light environments include sewers, underground utility tunnels, and tunnels.

[0018] Preferably, the light-aware lightweight YOLO framework is trained in the following way: The optimization is performed using a total loss function consisting of classification loss (including Varifocal Loss), regression loss (including Wise-IoU Loss), and target confidence loss. During the training phase, data augmentation is performed on the input images, including geometric augmentation, photometric enhancement, noise addition, region occlusion, and Mosaic stitching. During the inference phase, nonmaximum suppression and invalid box filtering are performed on the output detection boxes.

[0019] Preferably, the total loss function is as follows: ; ; ; in, Represents the total loss function; Indicates classification loss; Indicates regression loss; Indicates the target confidence loss; , , Indicates non-negative weighting coefficients; This represents the total number of samples in the training batch; Indicates the sample index; Indicates the first IoU-aware classification labels for each sample; Indicates the first The predicted class probability of each sample; Indicates the index of the prediction box; Indicates the first The intersection-over-union ratio between each predicted bounding box and its corresponding ground truth bounding box; The baseline value representing IoU; This represents the temperature coefficient.

[0020] This invention also provides a lightweight defect detection system for light sensing in low-light environments, comprising: Unmanned vehicle platform; An image acquisition module mounted on an autonomous vehicle platform is used to acquire image data in low-light environments; A lightweight YOLO detection module for illumination sensing, mounted on an autonomous vehicle platform, is connected to an image acquisition module for target detection from image data. The light-sensing lightweight YOLO detection module includes: An illumination-adaptive enhancement feature extraction network is used to achieve low-light enhancement and lightweight feature extraction at the feature level. A hybrid path aggregation network, connected to an illumination-adaptive enhanced feature extraction network, is used for multi-scale feature fusion; A dynamic feature fusion detection head, connected to a hybrid path aggregation network, is used to output detection results based on a dynamic spatial attention mechanism; And a result output module, which connects to the light-sensing lightweight YOLO detection module, to output detection results including target category, location and confidence level; The communication module is used to transmit the detection results to the remote control station in real time.

[0021] The present invention also provides a computer device, including: a memory and a processor; the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described lightweight defect detection method for light perception in low-light environments.

[0022] Therefore, the present invention employs the above-mentioned lightweight defect detection method, system, and equipment for light sensing in low-light environments, and the beneficial technical effects are as follows: (1) To address the core problem of low illumination leading to difficulty in image feature extraction in existing technologies, this invention utilizes the illumination analysis and enhancement module in RCFNet to achieve adaptive illumination correction and enhancement at the feature level of the neural network. This technology can directly improve the feature representation capabilities of dark and overexposed areas without relying on a separate and time-consuming image preprocessing process, thereby enhancing the robustness of the model in feature extraction under dim and uneven illumination environments and fundamentally reducing the false positive and false negative rates.

[0023] (2) To address the challenge of complex models in existing technologies failing to run in real time on edge devices such as unmanned vehicles, this invention comprehensively optimizes the network using lightweight convolutional structures such as CFBlock and MConv. This significantly reduces model parameters and computational complexity while maintaining powerful feature representation capabilities. This enables the proposed lightweight model to achieve high frame rate real-time inference on embedded AI computing platforms such as Jetson, successfully resolving the key contradiction between detection accuracy and real-time performance under limited computing resources, and providing a reliable technical foundation for online intelligent inspection on mobile platforms.

[0024] (3) To address the issues of easy loss of small target features and false alarms caused by complex background interference in existing scene inspection technologies, this invention enhances the preservation and utilization of detailed features through the cross-layer feature fusion mechanism of MPANet, and utilizes the dynamic spatial attention mechanism in DFF-Head to enable the model to autonomously focus on potential defect areas in the image. This design not only improves the perception sensitivity and localization accuracy of small-scale defects such as cracks and corrosion points, but also enhances the model's ability to discriminate against complex background interference such as stains and watermarks, enabling it to exhibit good generalization and stability in different low-light inspection scenarios. Attached Figure Description

[0025] Figure 1This is a flowchart illustrating the overall workflow of the lightweight defect detection method for illumination sensing in low-light environments according to the present invention. Figure 2 This is a comparison diagram of the C2f-DarkNet and RCFNet network architectures. Figure 2 (a) in the text is C2f-DarkNet. Figure 2 (b) in the text is RCFNet. Figure 2 (c) in the text refers to CFBlock. Figure 2 (d) in the text refers to PConv; Figure 3 This is a schematic diagram of the RXNet architecture; Figure 4 This is a schematic diagram of the Hybrid Path Aggregation Network (MPANet) and its Hybrid Convolutional Module (MConv), where... Figure 4 (a) in the text is MPANet. Figure 4 (b) in the text is MConv; Figure 5 This is a structural diagram of the Dynamic Feature Fusion Detection Head (DFF-Head); Figure 6 A comparison of RXNet's low-light enhancement effects; Figure 7 Bar chart comparing the performance of different feature extraction networks; Figure 8 This is a schematic diagram comparing the feature activation saliency maps of PANet and MPANet. Figure 9 A schematic diagram comparing saliency maps with and without the Dynamic Spatial Attention (DSA) module; Figure 10 A bar chart comparing the mAP@50 performance of each model on the RDD2022 public dataset; Figure 11 A visualization of the detection results for eight types of defects in multiple scenarios. Detailed Implementation

[0026] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0027] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0028] Example 1 This embodiment details the complete implementation scheme of the lightweight defect detection method based on illumination sensing in low-light environments proposed in this invention. For example... Figure 1 As shown, its overall workflow mainly includes three core stages: data acquisition and preprocessing, model forward inference, and result output and system integration.

[0029] I. System Overall Structure and Working Principle

[0030] The detection system of this invention uses a remotely controlled unmanned vehicle as its carrier, and its core is an onboard processing unit that integrates the RX-YOLO (lightweight YOLO framework for illumination sensing) algorithm. System working principle (corresponding to...) Figure 1 The process is as follows: The autonomous vehicle is equipped with a high-definition camera (supporting automatic fill light) and an IMU to collect real-time video streams and pose data in low-light environments.

[0031] The video stream is input to the onboard RX-YOLO detection module for real-time inference.

[0032] The core algorithm of this module processes images sequentially through RCFNet (Illumination Adaptive Enhancement Feature Extraction Network), MPANet (Hybrid Path Aggregation Network), and DFF-Head (Dynamic Feature Fusion Detection Head), and outputs the defect category, bounding box location, and confidence level.

[0033] The test results are displayed locally on one hand, and transmitted back to the ground control station in real time via a wireless communication module on the other hand, so that operators can monitor them and support automatic alarms and report generation.

[0034] II. Specific implementation of the RX-YOLO model architecture.

[0035] 1. Illumination Adaptive Enhancement Feature Extraction Network (RCFNet).

[0036] like Figure 2 In (b), RCFNet replaces the traditional C2f-DarkNet. Figure 2 (a) is used as the backbone network. Its core advantage lies in the deep integration of low-light enhancement and lightweight feature extraction at the feature level, without the need for additional independent preprocessing steps. Figure 2 (a) in the diagram represents the standard structure of C2f-DarkNet, which is mainly composed of multiple stacked C2f modules and uses residual connections to achieve feature transfer. The RCFNet architecture integrates the illumination analysis and enhancement module (RXNet) and the CFBlock high-efficiency convolutional block on the basis of C2f-DarkNet, which realizes feature enhancement and computational efficiency improvement in low-light environments. Figure 2 (c) in the diagram represents the specific structure of CFBlock, which uses a combination of partial convolution and pointwise convolution. Figure 2 In the diagram, (d) represents the core structure of partial convolution (PConv), which performs convolution operations only on a portion of the input channel to reduce computational cost.

[0037] RCFNet consists of an illumination analysis and enhancement module (RXNet) and an efficient convolutional block (CFBlock), used to achieve low-light enhancement and lightweight feature extraction at the feature level.

[0038] For details on the structure of the RXNet module, please refer to [link / reference]. Figure 3 Its core is to learn the illumination distribution features of an image through a lightweight network, thereby achieving adaptive feature enhancement. Figure 3 The multi-stage processing flow of RXNet is demonstrated: the input image first passes through a global average pooling layer to obtain a preliminary illumination estimate, and then is concatenated with the original image and fed into a lightweight quantum network that includes operations such as grouped convolution and channel shuffling. Illumination feature maps are generated through multi-layer feature transformation, and finally weighted fusion with the input features to achieve enhancement.

[0039] Input image First, a preliminary illumination estimation map is obtained through global average pooling. : ; in, Indicates the height of the input image (in pixels); Indicates the width of the input image (in pixels); This indicates the number of channels in the input image, which is 3 here (corresponding to an RGB color image). This represents the channel index, with a value range of 100. This is used to iterate through each channel of the image; Indicates the input image In coordinates First The pixel value of the channel; Represents the horizontal coordinate (range of values) ); Represents the vertical coordinate (range of values) ).

[0040] Preliminary illumination estimation map With input image splicing: ; in, This indicates a channel-level stitching operation, which stitches the original image... ( ) and illumination estimation map ( ) spliced ​​together as dimension Feature map .

[0041] Feature map after stitching Passing through in sequence Convolution (dimensionality reduction and fusion of channel information), GELU activation function (introducing non-linear feature transformation), layer normalization (LN, stabilizing the training process and accelerating convergence), grouped convolution (groups=4, reducing computational complexity and enhancing channel independence), and channel shuffling (CS, breaking down channel isolation caused by grouped convolution and promoting cross-group information interaction) ultimately output an illumination feature map. .

[0042] Illumination feature map After activation by the Sigmoid function, the input feature map is multiplied element-wise: ; in, The enhanced feature map shows clearer details in dark areas and more uniform illumination distribution, laying the foundation for subsequent feature extraction. This represents the input feature map to be enhanced, which is the feature output from the preceding network layer. Represents the Sigmoid function; This indicates element-wise multiplication.

[0043] It achieves adaptive brightening of dark areas and suppression of overexposed areas at the feature level, improving data quality at its source. See the comparison of its enhancement effects. Figure 6 .

[0044] CFBlock (High-Efficiency Convolutional Block): Employs a structure of "partial convolution (PConv) + pointwise convolution (PWConv)".

[0045] Let the input features be... PConv only applies to the former Each channel Convolution, with the remaining channels remaining unchanged: ; Subsequently via PWConv ( Convolutional processing is used for channel fusion. ; in, Represents the input feature map The former One channel; This indicates a partial convolution operation, performed only on the first few bits of the input. Each channel executes Convolution extracts local spatial features; This represents the output feature map of a partial convolution, with dimensions equal to... Consistency, that is ; This indicates pointwise convolution operation ( Convolutional kernels are used to fuse the channel information of the concatenated feature maps, adjust the channel dimensions, and retain key features. This represents the final output feature map of CFBlock, with dimensions of [dimension number missing]. .

[0046] This design maintains feature representation capabilities while requiring only 25% of the parameters of a standard convolution.

[0047] PConv performs spatial convolution only on a portion of the input channels, significantly reducing computation, while PWConv performs channel fusion. This design achieves remarkable lightweighting while maintaining expressive power.

[0048] Figure 2 In the structural diagrams of (c) CFBlock and (d) PConv, C_in represents the number of channels in the input feature map of the module, and C_out represents the number of channels in the output feature map. In the architecture proposed in this invention, CFBlock maintains a consistent number of input and output channels, i.e. PConv, as a submodule, usually maintains the same number of channels.

[0049] 2. Hybrid Path Aggregation Network (MPANet).

[0050] like Figure 4 As shown in (a), MPANet is an enhancement of the traditional PANet, serving as the model's neck.

[0051] MConv module (hybrid convolution): such as Figure 4 As shown in (b), the standard convolution is replaced with MConv. MConv also adopts the efficient structure of PConv+PWConv and can embed attention mechanisms. It supports multiple partial convolution ratios such as 1 / 4, 1 / 2, and 3 / 4, and achieves lightweight and efficient feature transformation in the feature upsampling and downsampling paths.

[0052] Taking a 1 / 4 scale as an example: ; ; Cross-layer shortcut connections: Introducing additional residual connections in the feature pyramid: ; in, This represents the input feature map of the MConv module, with dimension 1. ; The proportion is Partial convolution operations, only on the first part of the input feature map Each channel executes Convolution, the rest The number of channels remains unchanged, and the output dimension remains the same. ; This represents the intermediate feature map after partial convolution, with dimension 1. It preserves the spatial structure and channel information of the input features while reducing the computational load; This represents a channel shuffling operation, which breaks the correlation between channels after pointwise convolution, enhances channel independence, and improves the diversity of feature representation; This represents the final output feature map of the MConv module, with dimensions equal to or greater than the input. Consistency, that is This achieves lightweight feature transformation while maintaining feature resolution; This represents a high-resolution feature map output by a shallow network, with dimensions of [dimensionality missing]. ( , Larger (Smaller), containing fine details about small targets; This represents a low-resolution feature map output by a deep network, with dimension 1. ( , Smaller (Larger), containing global semantic information; This indicates an upsampling operation, which uses bilinear interpolation to transform the shallow feature map. The resolution was adjusted to match the deep feature map. Consistency (i.e.) ), and at the same time through Convolution adjustment channel number to ; This represents the fused feature map, with dimension 1. It contains both deep semantic information and shallow detail information, effectively improving the quality of multi-scale feature fusion, especially improving the detection performance of small targets.

[0053] High-resolution features from shallow layers are directly transferred to deeper layers, ensuring that fine features of small objects are not lost during the fusion process. A comparison of the feature activation performance between MPANet and the standard PANet can be found in [link to documentation]. Figure 8 It is evident that MPANet provides more concentrated activation and clearer boundaries to the target region.

[0054] 3. Dynamic Feature Fusion Detection Head (DFF-Head).

[0055] like Figure 1As shown, the Dynamic Feature Fusion Detection Head (DFF-Head) receives multi-scale feature maps P2, P3, and P4 from the MPANet output, corresponding to high-resolution (shallow), medium-resolution (mid-layer), and low-resolution (deep) features, respectively. Meanwhile, N2, N3, and N4 represent enhanced feature maps output after cross-layer fusion and upsampling / downsampling processing in MPANet, used for multi-scale prediction by the subsequent detection head. Specifically, P2 / N2 represents the high-resolution feature map from the shallow network, containing rich details of small targets; P3 / N3 represents the medium-resolution feature map from the mid-layer network, balancing details and semantic information; and P4 / N4 represents the low-resolution feature map from the deep network, possessing strong semantic expressive power but lower resolution.

[0056] like Figure 5 As shown, this detection head incorporates a core Dynamic Spatial Attention (DSA) mechanism: The DSA module performs global pooling on the input feature map along both the horizontal and vertical directions to generate a pair of direction-aware feature vectors.

[0057] ; ; in, The input feature map of DFF-Head has dimensions of . ( For the number of channels, For height, (width); This represents the feature vector after horizontal global pooling, with dimension . It reflects the global feature distribution of each channel in the horizontal direction and is suitable for capturing defects that extend in the vertical direction (such as vertical cracks). This represents the feature vector after global pooling in the vertical direction, with dimension . It reflects the global feature distribution of each channel in the vertical direction and is suitable for capturing defects that extend in the horizontal direction (such as horizontal cracks). This represents the horizontal coordinate index, with a range of values. ; This represents the vertical coordinate index, with a range of values. ; Indicates the feature map Summing along the horizontal direction (width dimension) yields a dimension of... Feature map; This indicates a mean-squared operation, which makes the pooling result unaffected by the image width, ensuring the scale consistency of features; similarly, This is used to ensure the scale consistency of pooling results in the vertical direction.

[0058] Then, a corresponding spatial attention weight map is generated through convolution.

[0059] ; ; in, express The convolution operation is used to adjust the number of channels of the orientation-sensitive feature vector to 1, generating a single-channel feature map; This represents a horizontal attention weight map with dimension 1. The weight value of each position reflects the correlation between that position and the defect feature in the horizontal direction. The higher the weight, the more likely that position is a defect area. This represents the vertical attention weight map, with dimension [missing information]. The weight value of each position reflects the correlation between that position and the defect feature in the vertical direction.

[0060] These weight maps reweight the original features, enabling the model to dynamically focus on rows and columns in the image where linear (such as cracks) or point-like (such as corrosion) defects may appear, effectively suppressing interference from complex backgrounds.

[0061] The weighted features are: ; in, This represents the weighted feature map, with the dimension still being [dimension number missing]. The feature signals of the defective region are significantly enhanced, and the noise in the background region is effectively suppressed, providing more accurate feature support for subsequent classification and regression tasks.

[0062] Figure 9 The demonstration showed that after introducing the DSA module, the model's attention shifted from scattered background misactivation to precise target focusing.

[0063] III. Model Training, Optimization, and Deployment.

[0064] 1. Data preparation and training.

[0065] Dataset: A self-built sewer defect dataset was used, including: DS (debris and sludge), HC (horizontal cracks), VC (vertical cracks), JF (joint failures), JO (joint openings), LP (lateral bulges), PB (pipe breaks), and SD (surface damage), split in an 8:2 ratio (3506 images) and a test set (877 images).

[0066] Data augmentation: 1) Geometric enhancement: random horizontal / vertical flip (probability 0.5), random rotation (±15°), random cropping and scaling (scaling ratio 0.8~1.2).

[0067] 2) Enhanced luminance: Randomly adjust brightness (±20%), contrast (±20%), and saturation (±20%) to simulate different lighting conditions.

[0068] 3) Noise and Occlusion: Add Gaussian noise (σ=0.01~0.05) and random area occlusion (Cutout, maximum occlusion area 10%).

[0069] 4) Mosaic enhancement: Randomly select 4 images and stitch them together into one to improve the model's adaptability to small targets and complex backgrounds.

[0070] Loss function and training: A total loss function, including Varifocal Loss (classification), Wise-IoU Loss (Equation 14), and confidence loss, was used for optimization. The AdamW optimizer was employed, along with cosine annealing learning rate scheduling, and training was performed for 150 epochs.

[0071] ; ; ; in, This represents the total loss function, which measures the overall difference between the model's predictions and the true labels. It is the optimization objective during the model training process. This represents the classification loss, used to optimize the prediction accuracy of defect categories, enabling the model to accurately distinguish different types of defects (such as cracks, corrosion, and leaks). This represents the regression loss, used to optimize the prediction accuracy of the defect bounding box, enabling the model to accurately locate the position and size of the defect; This represents the target confidence loss, used to optimize the confidence of defects in the prediction box, distinguish between real defects and background interference, and reduce the false detection rate. , , Denotes non-negative weight coefficients that satisfy... (Or adjust the absolute values ​​according to actual needs; the core is to balance the contributions of the three.) Recommended initial settings: , γ=0.2 (can be fine-tuned based on training results); This represents the total number of samples in the training batch (including positive and negative samples). This represents the sample index, used to iterate through each sample in the batch; Indicates the first The IoU-sensory classification label for each sample, with values ​​ranging from 1 to 2. When the sample is a positive sample (a real defect), This represents the IoU value between the predicted bounding box and the ground truth bounding box for this sample. A higher IoU value indicates a higher weight. When the sample is a negative sample (background), ; Indicates the first The predicted class probability of each sample, with values ​​ranging from... This indicates the probability that the model predicts the sample belongs to a certain defect category; This represents the loss term for positive samples. The larger the value (the higher the IoU), the greater its contribution to the loss, prompting the model to pay more attention to positive samples with high confidence. This represents the index of the prediction box, used to traverse each prediction box; Indicates the first The intersection-over-union ratio (IoU) between each predicted bounding box and its corresponding ground truth bounding box ranges from 1. A higher IoU value indicates more accurate positioning. This represents the baseline value for IoU, which is 0.5 and is used to adjust the sensitivity of the weighting factor. This represents the temperature coefficient, with a value of 3.0, and is used to control the variation of the weighting factor.

[0072] 2. Lightweight model and edge deployment.

[0073] Compression and acceleration: The trained model is converted to INT8 format through post-training quantization (PTQ), and the MPANet part is pruned to further compress the model while the accuracy loss is less than 1%.

[0074] Embedded Deployment: The optimized model is converted via ONNX, and an inference engine is built on the NVIDIA Jetson AGX Orin / Xavier NX platform using TensorRT. By enabling FP16 precision, layer fusion, and other techniques, the inference engine can be deployed with an input size of [missing information]. At the same time, it achieves a real-time inference speed of over 40 FPS with an average power consumption of less than 15W, fully meeting the endurance and real-time requirements of the autonomous vehicle platform.

[0075] IV. Verification of Implementation Results

[0076] To verify the overall performance of this invention, a systematic comparative experiment and ablation analysis were conducted on a self-built sewer defect dataset. The results are as follows: 1. Overall testing performance is significantly improved.

[0077] As shown in Table 1, compared with current mainstream advanced models, the RX-YOLO framework proposed in this invention achieves the best detection accuracy (AP@50) in 5 out of 8 defect types (DS, VC, JO, LP, and SD). Its comprehensive evaluation index mAP@50 reaches 88.0%, surpassing the YOLO series models including YOLOv8 and YOLOv12, as well as Transformer base models such as DEIM and RT-DETRv3. At the same time, RX-YOLO achieves an inference speed of 40.5 FPS with an extremely low parameter count of 2.64M, achieving an optimal balance between detection accuracy, inference efficiency, and model lightweightness. Figure 7 The figure further illustrates the performance comparison of different feature extraction networks in terms of detection accuracy (mAP@50) and model parameter count. As can be seen from the figure, RX-YOLO (based on RCFNet) achieves the highest detection accuracy while maintaining an extremely low parameter count, significantly outperforming mainstream lightweight backbone networks such as traditional C2F-CSPDarkNet and FasterNet, thus verifying the effectiveness of this invention in feature extraction and lightweight design.

[0078] Table 1. Performance comparison of RX-YOLO with existing methods on the sewer defect dataset.

[0079] 2. The lightweighting effect of the backbone network is significant.

[0080] Through backbone network design optimization, this invention achieves efficient feature extraction and computational compression. As shown in Table 2, the lightweight version of RX-YOLO-N (based on RCFNet(n)) has only 2.69M parameters and a computational cost (FLOPs) of 7.0G. Compared with the baseline model C2f-CSPDarkNet(n), the number of parameters and computational cost are reduced by 12.7% and 16.7%, respectively, and the inference time per image is shortened to 30 ms. These results demonstrate that this invention significantly reduces model complexity while maintaining strong feature representation capabilities, laying the foundation for real-time detection on resource-constrained embedded platforms or autonomous vehicles.

[0081] 3. The feature fusion and attention modules make significant contributions.

[0082] Ablation experiments validated the effectiveness of MPANet and the DSA-Head module. As shown in Table 3, under the same backbone network (RCFNet), the full MPANet (including MConv and Shortcut) improved the F1 score from 0.820 to 0.869 compared to the standard PANet, while further reducing the number of model parameters to 2.32M. This demonstrates that MPANet can achieve better multi-scale feature fusion with a more streamlined structure. Furthermore, the introduction of the Dynamic Spatial Attention (DSA) module improved the model accuracy to 86.9% and achieved 91.1% mAP@50, significantly outperforming other attention mechanisms or baseline models, demonstrating the effectiveness of this module in focusing on target regions in complex backgrounds.

[0083] Table 2. Efficiency and performance comparison of different backbone networks (using the same Neck and Head).

[0084] Table 3 Performance comparison of pyramid structures (Neck) with different features (based on RCFNet backbone)

[0085] 4. The generalization ability has been effectively verified.

[0086] To examine the model's adaptability in different low-light scenarios, generalization tests were conducted on the publicly available multinational road damage dataset RDD2022. This dataset covers a large number of low-light, small-object scenes, presenting similar visual challenges to the sewer detection task. Experimental results show that ( Figure 10 (See Table 4). RX-YOLO achieved an mAP@50 value of 59.80 on the RDD2022 dataset, which is better than advanced general detection models such as YOLOv12 (59.20) and DEIM (58.50). This proves that its core modules (such as illumination adaptive enhancement and dynamic attention mechanism) have good cross-scene generalization ability and are suitable for various low-light and small target detection tasks.

[0087] Figure 11 The visualization further showcases the detection performance of RX-YOLO for eight types of defects (DS, HC, VC, JF, JO, LP, PB, SD) in various real-world low-light scenarios (such as sewers, pipe corridors, and tunnels). As shown in the figure, the model can stably output high-precision detection boxes under different lighting conditions, complex backgrounds, and multi-scale targets, verifying its practicality and robustness in real-world inspection tasks.

[0088] Table 4. Performance comparison of each model at mAP@50 on the RDD2022 dataset.

[0089] It is worth noting that all contents not described in detail in this invention are existing technologies and are well known to those skilled in the art.

[0090] Therefore, this invention employs the aforementioned lightweight defect detection method, system, and device based on illumination perception in low-light environments. By integrating the RX-YOLO framework, which combines illumination adaptive enhancement, lightweight convolution, and dynamic spatial attention mechanisms, it effectively solves the problems of poor image quality, loss of small target features, and strong background interference in low-light environments without requiring complex preprocessing or high computing power, thereby improving detection accuracy and robustness. Simultaneously, its enhanced multi-scale small target perception capability and complete unmanned vehicle integration solution form a high-performance, easily deployable, and highly practical intelligent inspection system, significantly improving the inspection efficiency and safety of concealed infrastructure.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A lightweight defect detection method based on illumination sensing in low-light environments, characterized in that, Includes the following steps: Step S1: Acquire image data collected in low-light environments; Step S2: Input the image data into the light-aware lightweight YOLO framework for object detection; The light-aware lightweight YOLO framework includes: An illumination-adaptive enhancement feature extraction network is used to achieve low-light enhancement and lightweight feature extraction at the feature level. Hybrid path aggregation network for multi-scale feature fusion; A dynamic feature fusion detection head is used to output detection results based on a dynamic spatial attention mechanism; Step S3: Output the detection results, including target category, location, and confidence level.

2. The lightweight defect detection method for illumination sensing in low-light environments according to claim 1, characterized in that, The illumination adaptive enhancement feature extraction network consists of an illumination analysis and enhancement module and an efficient convolutional block connected in sequence. The illumination analysis and enhancement module is used to learn the illumination distribution feature map from the input image and adaptively weight the input features; The efficient convolutional block uses a combination of partial convolution and pointwise convolution for lightweight feature extraction.

3. The lightweight defect detection method for illumination sensing in low-light environments according to claim 2, characterized in that, The illumination analysis and enhancement module includes: The global average pooling layer is used to perform global average pooling on the input image to obtain a preliminary illumination estimation map. The stitching layer is used to stitch the preliminary illumination estimation map with the input image along the channel dimension; Light quantum networks are used to process the spliced ​​feature maps to generate illumination feature maps, which include, in sequence, […]. Convolutional layers, GELU activation function, layer normalization layers, grouped convolutional layers, and channel shuffling layers; The weighting module is used to activate the illumination feature map with the Sigmoid function, and then multiply it element-wise with the input feature map to be enhanced, outputting the enhanced feature map. Efficient convolutional blocks perform the following operations for feature extraction: For the previous input feature map Each channel executes Partial convolution, where This represents the total number of input channels. The output feature map of the partial convolution is concatenated with the remaining channels of the input feature map that have not undergone convolution. Perform on the stitched feature map Pointwise convolution is performed to obtain the output feature map.

4. The lightweight defect detection method for illumination sensing in low-light environments according to claim 1, characterized in that, Hybrid path aggregation networks include hybrid convolutional modules and cross-layer shortcut connections; The hybrid convolution module adopts a combination structure of partial convolution and pointwise convolution, and a channel shuffling operation is followed by pointwise convolution; partial convolution supports convolution operations on the first 1 / 4, 1 / 2 or 3 / 4 channels of the input feature map; Cross-layer shortcut connections are used to upsample the high-resolution feature maps output by shallow networks and add them to the low-resolution feature maps output by deep networks to achieve feature fusion.

5. The lightweight defect detection method for illumination sensing in low-light environments according to claim 1, characterized in that, The dynamic feature fusion detection head includes: The dynamic spatial attention module is used to perform global pooling along the horizontal and vertical directions respectively to generate a direction-aware attention weight map; The feature weighted fusion module is used to reweight and fuse multi-scale features based on the attention weight map.

6. The lightweight defect detection method for illumination sensing in low-light environments according to claim 1, characterized in that, Low-light environments include sewers, underground utility tunnels, and tunnels.

7. The lightweight defect detection method for illumination sensing in low-light environments according to claim 1, characterized in that, The light-aware lightweight YOLO framework is trained in the following way: The optimization is performed using a total loss function consisting of classification loss (including Varifocal Loss), regression loss (including Wise-IoU Loss), and target confidence loss. During the training phase, data augmentation is performed on the input images, including geometric augmentation, photometric enhancement, noise addition, region occlusion, and Mosaic stitching. During the inference phase, nonmaximum suppression and invalid box filtering are performed on the output detection boxes.

8. The lightweight defect detection method for illumination sensing in low-light environments according to claim 7, characterized in that, The total loss function is as follows: ； ； ； in, Represents the total loss function; Indicates classification loss; Indicates regression loss; Indicates the target confidence loss; , , Indicates non-negative weighting coefficients; This represents the total number of samples in the training batch; Indicates the sample index; Indicates the first IoU-aware classification labels for each sample; Indicates the first The predicted class probability of each sample; Indicates the index of the prediction box; Indicates the first The intersection-over-union ratio between each predicted bounding box and its corresponding ground truth bounding box; This represents the baseline value for IoU; This represents the temperature coefficient.

9. A lightweight defect detection system for illumination sensing in low-light environments, characterized in that, include: Unmanned vehicle platform; An image acquisition module mounted on an autonomous vehicle platform is used to acquire image data in low-light environments; A lightweight YOLO detection module for illumination sensing, mounted on an autonomous vehicle platform, is connected to an image acquisition module for target detection from image data. The light-sensing lightweight YOLO detection module includes: An illumination-adaptive enhancement feature extraction network is used to achieve low-light enhancement and lightweight feature extraction at the feature level. A hybrid path aggregation network, connected to an illumination-adaptive enhanced feature extraction network, is used for multi-scale feature fusion; A dynamic feature fusion detection head, connected to a hybrid path aggregation network, is used to output detection results based on a dynamic spatial attention mechanism; And a result output module, which connects to the light-sensing lightweight YOLO detection module, to output detection results including target category, location and confidence level; The communication module is used to transmit the detection results to the remote control station in real time.

10. A computer device, comprising: Memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, it implements the steps of the light-sensing lightweight defect detection method under low-light conditions as described in any one of claims 1-8.

Images