A deep learning-based method for detecting road damage in low-light substations

Abstract

This invention discloses a method for detecting substation road damage in low-light conditions based on deep learning, belonging to the field of substation road technology. The method includes the following steps: S1: Constructing an original substation road damage image dataset D1; S2: Preprocessing D1 to obtain a processed substation road damage image dataset D2, and performing low-light degradation synthesis on each substation road damage image p in D2 to obtain a low-light version image p', forming a substation road damage image tuple.<p,p'> This invention improves the detection accuracy of substation road damage in low-light environments by introducing reflectivity feature extraction and reflectivity optimization learning. It effectively reduces interference from equipment shading and changes in lighting, and avoids the problem of existing methods relying too heavily on image enhancement techniques and easily introducing noise under low-light conditions.

Description

Technical Field

[0001] This invention relates to the field of substation road technology, specifically a method for detecting road damage in low-light substations based on deep learning. Background Art

[0002] Substation roads are crucial infrastructure for ensuring the smooth transport and maintenance of equipment, and are typically narrow. Numerous substation equipment, such as transformers and disconnectors, are distributed along these roads. The roads between these devices serve not only for maintenance personnel but also for the passage of intelligent inspection equipment such as drones and unmanned vehicles. With the increasing volume of substation maintenance work and the higher frequency of equipment transport, the load on substation roads is constantly increasing, leading to increasingly severe damage. However, due to factors such as shadows cast by equipment and insufficient lighting at night, damage to substation roads in low-light conditions is difficult to detect in a timely manner, increasing the difficulty of substation road maintenance. Therefore, how to efficiently and accurately detect substation road damage under low-light conditions has become an urgent technical challenge.

[0003] Currently, substation road damage detection mainly relies on manual inspection and deep learning methods. While manual inspection can detect obvious substation road damage, it is inefficient, costly, and limited by time and effort, making comprehensive coverage difficult. Under normal lighting conditions, deep learning methods can efficiently and accurately identify road damage. However, substation roads are typically narrow, and equipment on both sides can easily create shadows, resulting in some roads being in low-light conditions. Furthermore, insufficient lighting at night or in inclement weather further exacerbates this problem. Therefore, deep learning models trained under normal lighting conditions show a significant decrease in recognition performance in low-light environments. Existing deep learning methods for substation road damage detection under low-light conditions are mainly divided into two types: detection-by-enhancement and enhancement-for-detection. Detection-by-enhancement improves image quality by enhancing low-light images (e.g., increasing brightness and contrast), thereby improving target detection accuracy. However, under low-light conditions, enhancement-by-enhancement may introduce noise or artifacts, negatively impacting detection performance. Detection augmentation combines image enhancement with the object detection process, optimizing training to better match the enhancement effect with the detection task. While detection augmentation improves detection accuracy, it still relies on the enhancement effect, and its effectiveness is limited in low-light environments, leading to insufficient feature representation and failing to adequately address road detection in low light conditions. Although both methods improve the accuracy of road damage detection in low light under certain conditions, they generally depend on image enhancement, and their effectiveness is limited in low-light environments.

[0004] Based on the above, the current method has the following drawbacks:

[0005] 1. The shadows cast by substation equipment often place substation roads in low-light environments; existing image enhancement techniques have limited effectiveness under low-light conditions and are prone to introducing noise; therefore, existing methods are difficult to effectively meet the needs of substation road detection in low-light environments.

[0006] 2. Existing methods are insufficient in detecting road damage in substations under low light conditions, and are prone to missed or false detections, making it difficult to meet the actual needs of safe and efficient operation and maintenance of substations.

[0007] Therefore, a deep learning-based method for detecting road damage in low-light substations is invented. This method enhances the model's ability to capture target features in low-light environments by optimizing the extraction of reflectivity features, while reducing interference from changes in lighting and equipment shading. This further improves the model's detection accuracy and robustness, thus providing strong technical support for the intelligent maintenance of substation roads. Summary of the Invention

[0008] To address the aforementioned technical problems, according to one aspect of the present invention, the present invention provides the following technical solution:

[0009] A deep learning-based method for detecting road damage in low-light substations includes the following specific steps:

[0010] S1: Construct the original substation road damage image dataset D1;

[0011] S2: Preprocess D1 to obtain the processed substation road damage image dataset D2, and perform low-light degradation synthesis on each substation road damage image p in D2 to obtain a low-light version image p', forming a substation road damage image tuple.<p,p'> A paired substation road damage image dataset D3 is constructed from all the substation road damage image pairs.

[0012] S3: Construct a deep learning-based model for detecting road damage in substations under low light conditions. Input the paired substation road damage image dataset D3 into the model to identify the types of road damage in the substation road damage images and automatically generate bounding boxes to determine the specific location and extent of the damage. The types of road damage include cracks, potholes, warping, and settlement.

[0013] S4: Before training the low-light substation road damage detection model, it is necessary to first set the hyperparameters. After completing the initial configuration, the paired substation road damage image datasets are divided into training set and validation set to ensure the independence and representativeness of each dataset.

[0014] S5: After the model training is completed, the trained low-light substation road damage detection model is used to identify and analyze the current substation road damage data. The final output includes the specific substation road damage type and location identification results, providing substation inspection personnel with accurate early warning information and subsequent handling suggestions.

[0015] As a preferred embodiment of the deep learning-based method for detecting road damage in low-light substations according to the present invention, the specific steps of S1 are as follows:

[0016] S11: Collect images of substation road damage under normal lighting conditions to form the original substation road damage image dataset D1;

[0017] S12: Manually label D1 to provide labels for subsequent model training. During the labeling process, use the labeling tool to select the damaged road areas and add corresponding labels to each damaged area to mark the type of road damage in that area.

[0018] As a preferred embodiment of the deep learning-based method for detecting road damage in low-light substations according to the present invention, the specific steps of S2 are as follows:

[0019] S21: Crop all substation road damage images in D1 to a uniform size;

[0020] S22: Expand D1 using data augmentation operations such as rotation, flipping, and noise addition to generate a substation road damage image dataset D2 containing more substation road damage images;

[0021] S23: Process each substation road damage image p in D2 using brightness adjustment, contrast adjustment, color saturation reduction, blurring and color shifting operations to generate the corresponding low-light version image p'.

[0022] S24: Combine p and p' to form a binary image of road damage at a substation.<p,p'> A paired substation road damage image dataset D3 is constructed from all the above substation road damage image pairs.

[0023] As a preferred embodiment of the deep learning-based method for detecting road damage in low-light substations according to the present invention, the specific steps of S3 are as follows:

[0024] S31: For any substation road damage image binary<p,p'> In the first branch, p is input into the first Conv module to generate the first normal illumination road damage feature map X1, and p' is input into the first Conv module to generate the first low illumination road damage feature map X'1. In the second branch, p is input into the decomposed reflectance and illumination decomposition module DRD to generate the first normal illumination road damage reflectance feature map R1, and p' is input into the DRD module to generate the first low illumination road damage reflectance feature map R'1.

[0025] S32: Will<X1,X'1> The input is fed into the second Conv module to generate the second road damage feature. Figure 2 tuple<X2,X'2> X1 is input into the second Conv module to obtain the second normal illumination road damage feature map X2, and X'1 is input into the second Conv module to obtain the second low illumination road damage feature map X′2. Since the structure of the second Conv module is exactly the same as that of the first Conv module, the continuity and consistency of feature extraction can be ensured. Based on this,<X2,X′2> The data is then passed to the CELAN module to generate a higher-level feature map, namely the third-level road damage feature. Figure 2 tuple<X3,X′3> X2 is input into the CELAN module to obtain the third normal illumination road damage feature map X3, and X′2 is input into the CELAN module to obtain the third low illumination road damage feature map X′3.

[0026] S33: Will<X2,X′2> The input is fed into the DLARK module to generate the second road damage reflectivity feature. Figure 2 tuple<R2,R′2> X2 is input into the DLARK module to obtain the second normal illumination road damage reflectivity feature map R2, and X'1 is input into the DLARK module to obtain the second low illumination road damage reflectivity feature map R'2.

[0027] S34: To further distinguish between reflectance and illumination components in an image, and to maintain consistency between illumination and reflectance,<R1,R'1> and<R2,R'2> The data is input into the IRC (Illumination and Reflection Compensation) module, and through a process involving interaction, recombination, and consistency constraints, the third road damage reflectivity characteristics are generated. Figure 2 tuple<R3,R'3> R1 and R2 are input into the IRC module to obtain the third normal illumination road damage reflectance feature map R3, and R'1 and R'2 are input into the IRC module to obtain the third low illumination road damage reflectance feature map R'3.

[0028] S35: Yes<R3,R′3> and<X3,X'3> Perform a concat operation to fuse reflectivity features with high-level semantic features to obtain the fourth road damage feature. Figure 2 tuple<X4,X'4> The concat operation of R3 and X3 yields the fourth normal illumination road damage feature map X4, and the concat operation of R'3 and X'3 yields the fourth low illumination road damage feature map X'4.

[0029] S36: Will<X4,X'4> The input is fed into the target prediction head to generate a tensor containing prediction information. Each row of the tensor corresponds to a detection result, including the predicted bounding box coordinates, damage category label, and confidence score information.

[0030] As a preferred embodiment of the deep learning-based method for detecting road damage in low-light substations according to the present invention, the overall execution flow of the CELAN module in S32 is as follows:

[0031] S321: In the first branch,<T1,T'1> The input is fed into a Conv2d layer with a kernel size of 3×3 for convolution operation to generate the second intermediate road damage feature. Figure 2 tuple<T2,T'2> The process involves inputting T1 into a Conv2d layer with a 3×3 kernel to obtain the second intermediate road damage feature map T2 under normal lighting conditions, and inputting T'1 into a Conv2d layer with a 3×3 kernel to obtain the second intermediate road damage feature map T'2 under low lighting conditions. Then...<T2,T'2> The input is fed into a RepConv layer with a kernel size of 3×3 for convolution operation to obtain the third intermediate road damage feature. Figure 2 tuple<T3,T'3> The process involves inputting T2 into a RepConv layer with a 3×3 kernel to obtain the third intermediate road damage feature map T3 under normal lighting conditions, and inputting T'2 into a RepConv layer with a 3×3 kernel to obtain the third intermediate road damage feature map T'3 under low lighting conditions. Then...<T3,T'3> The input is fed into a FreqConv layer with a kernel size of 5×5 for convolution operation to generate the fourth intermediate road damage feature. Figure 2 tuple<T4,T'4> The process involves inputting T3 into a FreqConv layer with a 5×5 kernel to obtain the fourth intermediate road damage feature map T4 under normal lighting conditions, and inputting T'3 into a FreqConv layer with a 5×5 kernel to obtain the fourth intermediate road damage feature map T'4 under low lighting conditions.<T4,T'4> The input is fed into a FreqConv layer with a kernel size of 7×7 for convolution operation to generate the fifth intermediate road damage feature. Figure 2tuple<T5,T'5> The fifth intermediate road damage feature map T5 under normal lighting is obtained by inputting T4 into a FreqConv layer with a kernel size of 7×7. The fifth intermediate road damage feature map T'5 under low lighting is obtained by inputting T'4 into a FreqConv layer with a kernel size of 7×7. Finally,<T5,T'5> The input is fed into a Conv2d layer with a kernel size of 3×3 for convolution operation to generate the sixth intermediate road damage feature. Figure 2 tuple<T6,T'6> T5 is input into a Conv2d layer with a kernel size of 3×3 to obtain the sixth intermediate normal lighting road damage feature map T6, and T'5 is input into a Conv2d layer with a kernel size of 3×3 to obtain the sixth intermediate low lighting road damage feature map T'6.

[0032] S322: In the second branch,<T1,T'1> The input is fed into a continuous convolutional structure and dilated convolution is performed to obtain the damage features of the seventh intermediate road. Figure 2 tuple<T7,T'7> T1 is input into a continuous convolutional structure to obtain the seventh intermediate road damage feature map T7 under normal lighting, and T'1 is input into a continuous convolutional structure to obtain the seventh intermediate road damage feature map T'7 under low lighting.

[0033] S323: Will<T2,T'2> ,<T3,T'3> ,<T4,T'4> ,<T5,T'5> ,<T6,T'6> and

[0034] <T7,T'7> Perform a Concat operation to generate the eighth intermediate road damage feature. Figure 2 tuple<T8,T'8> The concat operation is performed on T2, T3, T4, T5, T6 and T7 to obtain the eighth intermediate normal illumination road damage feature map T8, and the concat operation is performed on T'2, T'3, T'4, T'5, T'6 and T'7 to obtain the eighth intermediate low illumination road damage feature map T'8. This achieves effective fusion of multi-scale features and information from different receptive fields, further enhancing the comprehensive expression ability of the low illumination substation road damage detection model for detailed and global features in low illumination scenarios.

[0035] S324: Will<T8,T'8> The input is fed into a Conv2d layer with a kernel size of 5×5 for convolution, and then activated using the Sigmoid activation function to generate the ninth intermediate road damage feature. Figure 2tuple<T9,T'9> The T8 layer is input into a Conv2d layer with a kernel size of 5×5 and activated using the Sigmoid activation function to obtain the ninth intermediate road damage feature map T9 under normal lighting. The T′8 layer is input into a Conv2d layer with a kernel size of 5×5 and activated using the Sigmoid activation function to obtain the ninth intermediate road damage feature map T′9 under low lighting.<T9,T′9> As the final output of the CELAN module, it provides crucial input for subsequent feature extraction and prediction tasks.

[0036] As a preferred embodiment of the deep learning-based method for detecting road damage in low-light substations according to the present invention, the overall execution flow of the DLARK module in S33 is as follows:

[0037] S331: In the first branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 9×9 and a dilation rate of 1, and a BN layer for convolution and normalization operations to obtain the first intermediate reflectance feature. Figure 2 tuple <X 2_1 ,X' 2_1 >, where X2 is input into this convolutional structure to obtain the first intermediate normal illumination reflectance feature map X. 2_1 Inputting X'2 into this convolutional structure yields the first intermediate low-light reflectivity feature map X'. 2_1 ;

[0038] S332: In the second branch,<X2,X′2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 5×5 and a dilation rate of 1 and a BN layer for convolution and normalization operations to generate a second intermediate reflectance feature. Figure 2 tuple <X 2_2 ,X′ 2_2 >, where X2 is input into this convolutional structure to obtain the second intermediate normal illumination reflectance feature map X. 2_2 Inputting X′2 into this convolutional structure yields the second intermediate low-illuminance reflectance feature map X′. 2_2 ;

[0039] S333: In the third branch,<X2,X′2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 5×5 and a dilation rate of 2, and a BN layer, to perform convolution and normalization operations, thereby obtaining the third intermediate reflectance feature. Figure 2 tuple <X 2_3 ,X′ 2_3 >, where X2 is input into this convolutional structure to obtain the third intermediate normal illumination reflectance feature map X. 2_3 Inputting X′2 into this convolutional structure yields the third intermediate low-light reflectivity feature map X′. 2_3;

[0040] S334: In the fourth branch,<X2,X′2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 2×2 and a BN layer, and undergoes convolution and normalization operations to generate the fourth intermediate reflectance feature. Figure 2 tuple <X 2_4 ,X′ 2_4 >, where X2 is input into this convolutional structure to obtain the fourth intermediate normal illumination reflectance feature map X. 2_4 Inputting X′2 into this convolutional structure yields the fourth intermediate low-light reflectivity feature map X′. 2_4 ;

[0041] S335: In the fifth branch,<X2,X′2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 2×2 and a dilation rate of 4, and a BN layer, for convolution and normalization operations to obtain the fifth intermediate reflectance feature. Figure 2 tuple <X 2_5 ,X' 2_5 >, where X2 is input into this convolutional structure to obtain the fifth intermediate normal illumination reflectance feature map X. 2_5 Inputting X'2 into this convolutional structure yields the fifth intermediate low-illuminance reflectance feature map X'. 2_5 ;

[0042] S336: Yes <X 2_1 ,X′ 2_1 >、 <X 2_2 ,X' 2_2 >、 <X 2_3 ,X' 2_3 >、 <X 2_4 ,X' 2_4 >and

[0043] <X 2_5 ,X' 2_5 Perform a Concat operation to obtain the sixth intermediate reflectance feature. Figure 2 tuple <X 2_6 ,X' 2_6 >, where X 2_1 X 2_2 X 2_3 X 2_4 and X 2_5 Performing the Concat operation yields the sixth intermediate normal illumination reflectance feature map X. 2_6 , will X' 2_1 、X' 2_2 、X' 2_3 、X' 2_4 and X' 2_5Performing the Concat operation yields the sixth intermediate low-illuminance reflectance feature map X'. 2_6 ,Will <X 2_6 ,X' 2_6 The input is fed into the AvgPool layer for pooling operations to obtain the seventh intermediate reflectance feature. Figure 2 tuple <X 2_7 ,X' 2_7 >, where X 2_6 The input to the AvgPool layer yields the seventh intermediate normal illumination reflectance feature map X. 2_7 , X′ 2_6 The input to the AvgPool layer yields the seventh intermediate low-illuminance reflectance feature map X'. 2_7 ,Will <X 2_7 ,X' 2_7 The input is fed into a continuous convolutional structure consisting of a Conv2d layer with a kernel size of 1×1, a DWConv layer with a kernel size of 2×2, a DWConv layer with a kernel size of 5×5, and a Conv2d layer with a kernel size of 2×2, and a depthwise convolution operation is performed to generate the eighth intermediate reflectance feature. Figure 2 tuple <X 2_8 ,X' 2_8 >, where X 2_7 The input is fed into a continuous convolutional structure to obtain the eighth intermediate normal illumination reflectance feature map X. 2_8 , X′ 2_7 The input is fed into a continuous convolutional structure to obtain the eighth intermediate low-illuminance reflectance feature map X′. 2_8 In the <X 2_6 ,X′ 2_6 While performing pooling operations, <X 2_6 ,X′ 2_6 The input is fed into a BN layer for normalization to generate the ninth intermediate reflectance feature. Figure 2 tuple <X 2_9 ,X′ 2_9 >, where X 2_6 The input to the BN layer yields the ninth intermediate normal illumination reflectance feature map X. 2_9 , X′ 2_6 The input is fed into the BN layer to obtain the ninth intermediate low-illuminance reflectance feature map X′. 2_9 ;

[0044] S337: Will <X 2_9 ,X′ 2_9 The input is fed into the AvgPool layer for pooling to obtain the tenth intermediate reflectance feature. Figure 2 tuple <X 2_10 ,X′ 2_10 >, where X 2_9The input to the AvgPool layer yields the tenth intermediate normal illumination reflectance feature map X. 2_10 , X′ 2_9 The input to the AvgPool layer yields the tenth intermediate low-illuminance reflectance feature map X′. 2_10 Next <X 2_10 ,X′ 2_10 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence for feature transformation to obtain the eleventh intermediate reflectance feature. Figure 2 tuple <X 2_11 ,X′ 2_11 >, where X 2_10 The input is fed into a structure consisting of fully connected layers, ReLU activation functions, and another fully connected layer in sequence to obtain the eleventh intermediate normal illumination reflectance feature map X. 2_11 , X′ 2_10 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence to obtain the eleventh intermediate low-illuminance reflectance feature map X′. 2_11 ,Will <X 2_11 ,X′ 2_11 >Activation was performed using the Sigmoid activation function to obtain the twelfth intermediate reflectance feature. Figure 2 tuple <X 2_12 ,X′ 2_12 >, where X 2_11 The twelfth intermediate normal illumination reflectance feature map X was obtained by activating the Sigmoid function. 2_12 , X′ 2_11 The twelfth intermediate low-illuminance reflectance feature map X′ was obtained by activating the Sigmoid function. 2_12 ,Will <X 2_9 ,X′ 2_9 >and <X 2_12 ,X′ 2_12 Perform element-wise multiplication to obtain the thirteenth intermediate reflectance feature. Figure 2 tuple <X 2_13 ,X′ 2_13 >, where X 2_9 With X 2_12 Element-wise multiplication yields the thirteenth intermediate normal illumination reflectance characteristic map X. 2_13 , X′ 2_9 With X′ 2_12 Element-wise multiplication yields the thirteenth intermediate low-illuminance reflectance feature map X′. 2_13 ,Will <X 2_13 ,X′ 2_13 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence for feature transformation, resulting in the fourteenth intermediate reflectance feature. Figure 2tuple <X 2_14 ,X′ 2_14 >, where X 2_13 The input is fed into a structure consisting of fully connected layers, ReLU activation functions, and another fully connected layer in sequence to obtain the fourteenth intermediate normal illumination reflectance feature map X. 2_14 , X′ 2_13 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence to obtain the fourteenth intermediate low-illuminance reflectance feature map X′. 2_14 ;

[0045] S338: Yes <X 2_14 ,X′ 2_14 After performing the DropPath operation, the fifteenth intermediate reflectance feature is obtained. Figure 2 tuple <X 2_15 ,X′ 2_15 >, where X 2_14 Performing the DropPath operation yields the fifteenth intermediate normal illumination reflectance feature map X. 2_15 , X′ 2_14 Performing the DropPath operation yields the fifteenth intermediate low-illuminance reflectance feature map X′. 2_15 ,Will <X 2_15 ,X′ 2_15 >and <X 2_8 ,X′ 2_8 Perform a Concat operation to obtain the final second-path damage reflectivity characteristics. Figure 2 tuple<R2,R′2> X 2_15 With X 2_8 Performing a concat operation yields the second normal illumination road damage reflectivity characteristic map R2, and X′ 2_15 With X′ 2_8 The concat operation yields the second low-light road damage reflectivity feature map R′2.

[0046] As a preferred embodiment of the deep learning-based method for detecting road damage in low-light substations according to the present invention, the specific steps of S4 are as follows:

[0047] S41: During training, the model extracts data from the training set according to the set batch size, performs forward calculations, generates prediction results, and then performs backpropagation through the overall loss function L2R-LOSS to optimize the parameters of the substation road damage detection model under low light conditions.

[0048] S42: After each round of training, the data from the validation set is used to evaluate the model and determine its generalization ability.

[0049] S43: To avoid overfitting during training, an early stopping mechanism is introduced. If the loss on the validation set does not improve significantly in multiple consecutive training rounds, the training will be terminated early to save computational resources and ensure the stability of the model in practical applications.

[0050] S44: When the training reaches the predetermined number of rounds and the validation set loss is stable or reaches the ideal level, the training process ends, and an optimized low-light substation road damage detection model is finally obtained. The model can accurately detect substation road damage under low light conditions and maintain good performance in different environments.

[0051] S45: After training, the model will have appropriate parameter configuration and strong generalization ability, and can be applied to new substation road damage detection tasks.

[0052] As a preferred embodiment of the deep learning-based method for detecting road damage in low-light substations described in this invention, wherein: the loss function L2R-LOSS in S41 is derived from the reflectance-learned loss function L r Reorganization Consistency Loss Function L rc And the target detection loss function L target The weighted composition, its functional expression is as follows:

[0053] L L2R-LOSS =λ g L r +λ f L target +λ q L rc

[0054] Where, λ g , λ f and λ q To account for the loss weights, the sum of the three is 1;

[0055] Among them, L r The function expression is as follows:

[0056]

[0057] Where R′1 is the first low-light road damage reflectivity feature map; R′2 is the first low-light road damage reflectivity feature map; N is the total number of feature points in the R′1 and R′2 feature maps, R′ 1_i and R′ 2_i These represent the feature response values ​​corresponding to the i-th position in the feature map; and Let R′1 and R′2 represent the average characteristic response values, respectively. and These are the variances of the characteristic response values ​​of R′1 and R′2, respectively. It is the covariance of the characteristic response values ​​between R'1 and R'2, and C1 and C2 are constants to avoid the denominator being zero;

[0058] Among them, L target The function expression is as follows:

[0059]

[0060] Where α and β represent the weights of the location regression loss and classification loss, respectively; x i and x' i p represents the center point of the ground truth bounding box and the center point of the predicted bounding box for the i-th object, respectively; i and p' i Let represent the true class label and predicted class probability of the i-th target, respectively. Smooth(x) represents the error between the predicted and true values, i.e.

[0061]

[0062] L rc The function expression is as follows:

[0063] L rc =||R1-R′2||1+||R′1-R2||1

[0064] Wherein, R1 is the first normal illumination road damage reflectance characteristic map; R'2 is the second low illumination road damage reflectance characteristic map; R'1 is the first low illumination road damage reflectance characteristic map; R2 is the second normal illumination road damage reflectance characteristic map.

[0065] Compared with existing technologies:

[0066] This invention improves the detection accuracy of substation road damage in low-light environments by introducing reflectivity feature extraction and reflectivity optimization learning. It effectively reduces interference from equipment shading and lighting changes, avoiding the problem of existing methods over-reliance on image enhancement techniques and easy introduction of noise under low-light conditions. At the same time, by combining multi-scale feature extraction and attention mechanisms, the model's adaptability and robustness to narrow roads and complex shading scenarios in substations are enhanced, thereby improving the practicality and reliability of the low-light substation road damage detection model. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the process of the present invention;

[0068] Figure 2 This is a structural diagram of the road damage detection model for low-light substations according to the present invention;

[0069] Figure 3 This is a structural diagram of the CELAN module of the present invention;

[0070] Figure 4 This is a structural diagram of the DLARK module of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0072] This invention provides a deep learning-based method for detecting road damage in low-light substations. Please refer to [link / reference]. Figures 1-4 The specific steps are as follows:

[0073] S1: Construct the original substation road damage image dataset D1;

[0074] The specific steps of S1 are as follows:

[0075] S11: Collect images of substation road damage under normal lighting conditions to form the original substation road damage image dataset D1;

[0076] S12: Manually label D1 to provide labels for subsequent model training. During the labeling process, use the labeling tool to select the damaged road areas and add corresponding labels to each damaged area to mark the type of road damage in that area.

[0077] Wherein, S1 includes, but is not limited to, the following embodiments:

[0078] The images were processed based on a self-collected dataset of original substation road damage images. The dataset contained 2000 images, including 800 images of road cracks, 500 images of road potholes, 450 images of road bulging, and 250 images of road settlement. According to the damage type, the images were labeled into four categories: cracks, potholes, bulging, and settlement. The image data were manually labeled using the LabelImg annotation tool to generate corresponding XML label files, which contained the label names of road damage and the corresponding target annotation box information.

[0079] LabelImg is an open-source image annotation tool, primarily used for labeling image datasets for tasks such as object detection and image segmentation.

[0080] S2: Preprocess D1 to obtain the processed substation road damage image dataset D2, and perform low-light degradation synthesis on each substation road damage image p in D2 to obtain a low-light version image p', forming a substation road damage image tuple.<p,p′> A paired substation road damage image dataset D3 is constructed from all the substation road damage image pairs.

[0081] The specific steps of S2 are as follows:

[0082] S21: Crop all substation road damage images in D1 to a uniform size;

[0083] S22: Expand D1 using data augmentation operations such as rotation, flipping, and noise addition to generate a substation road damage image dataset D2 containing more substation road damage images;

[0084] S23: Process each substation road damage image p in D2 using brightness adjustment, contrast adjustment, color saturation reduction, blurring and color shifting operations to generate the corresponding low-light version image p′.

[0085] S24: Combine p and p′ to form a binary image of road damage at a substation.<p,p′> A paired substation road damage image dataset D3 is constructed from all the above substation road damage image pairs.

[0086] S2 includes, but is not limited to, the following embodiments:

[0087] The original substation road damage image dataset D1 contained 2000 images. First, all images were uniformly cropped to 512×512 pixels. Then, various data augmentation methods were used to expand the dataset, including rotation, flipping, and noise addition, all of which can be implemented using Python. Specifically, rotation was implemented using the `cv2.getRotationMatrix2D()` function; image flipping was implemented using the `cv2.flip()` function; and noise addition was implemented using the `cv2.randPepper()` function. After these augmentation processes, D1 was expanded into a substation road damage image dataset D2 containing 12000 images. Based on this, to simulate road damage scenarios under low-light conditions, further data augmentation was performed on each image in D2. The images undergo low-light processing; specific operations include brightness adjustment, contrast adjustment, color saturation reduction, blurring, and color shifting. Brightness adjustment uses the ImageEnhance.Brightness() method; contrast adjustment uses ImageEnhance.Contrast(); color saturation reduction is achieved using ImageEnhance.Color(); blurring is achieved using the ImageFilter.GaussianBlur() method; and color shifting is accomplished by modifying the pixel values ​​of the image's RGB channels. Finally, the low-light version images are paired one-to-one with the original normal-light images to construct the paired substation road damage image dataset D3, totaling 24,000 images.

[0088] The common operations used to augment the original substation road damage image dataset D1 include the following:

[0089] (1) Rotation: Rotation simulates a change in viewpoint by rotating the image by a certain angle. The rotate() method of the PIL library can be used to randomly select an angle range (e.g., -30° to 30°) and rotate the image around the center point. This method can effectively enhance the model's ability to recognize images at different angles.

[0090] (2) Flip: The flip operation is achieved by mirroring the image along the horizontal or vertical axis; horizontal flipping can be achieved using the transpose(Image.FLIP_LEFT_RIGHT) method of the PIL library, while vertical flipping can be achieved using transpose(Image.FLIP_TOP_BOTTOM) or cv2.flip(); this operation can enhance the model's ability to recognize different directions;

[0091] (3) Noise Injection: The noise injection operation simulates the degradation of image quality by adding Gaussian noise or salt-and-pepper noise to the image; you can use NumPy to generate a Gaussian noise matrix of the same size as the image and add it to the image pixel values, or use the cv2.randSalt() and cv2.randPepper() functions to generate salt-and-pepper noise; this operation helps to improve the robustness of the model under low-quality images;

[0092] The low-light degradation synthesis technique was used to process the substation road damage image dataset D2. The common operations used included the following:

[0093] (1) Brightness Adjustment: Brightness adjustment simulates different lighting conditions by changing the brightness of the image; the PIL.ImageEnhance.Brightness() method can be used to adjust the brightness of the image by setting an enhancement factor; when the enhancement factor is greater than 1, the image becomes brighter, and when it is less than 1, the image becomes darker; this operation helps to simulate the visual effect in nighttime or dim environments;

[0094] (2) Contrast Adjustment: Contrast adjustment simulates different ambient lighting by enhancing or reducing the difference in brightness between images. Using the PIL.ImageEnhance.Contrast() method, an enhancement factor can be set to adjust the contrast. When the enhancement factor is greater than 1, the image contrast increases, and when it is less than 1, the contrast decreases. This operation helps to simulate the changes in the visibility of objects under different lighting conditions.

[0095] (3) Color Saturation Adjustment: Color saturation adjustment simulates the visual effect under low light or different lighting conditions by changing the saturation of the image colors; the PIL.ImageEnhance.Color() method can be used to adjust the saturation of the image; increasing the saturation will make the colors more vivid, while decreasing the saturation will make the image look darker or closer to black and white;

[0096] (4) Blurring: Blurring operations apply filters to an image to blur it, thereby simulating the effect of low light or blurry images. Common blurring methods include Gaussian blur, which can be implemented using PIL.ImageFilter.GaussianBlur(). This operation helps to simulate situations such as insufficient light or motion blur, enhancing the model's robustness to these situations.

[0097] (5) Color Shifting: In low light conditions, the colors of an image will often shift. By applying random shifts to the RGB channels of the image, the color changes in low light conditions can be simulated.

[0098] S3: Construct a deep learning-based model for detecting road damage in substations under low light conditions. Input the paired substation road damage image dataset D3 into the model to identify the types of road damage in the substation road damage images and automatically generate bounding boxes to determine the specific location and extent of the damage. The types of road damage include cracks, potholes, warping, and settlement.

[0099] The specific steps of S3 are as follows:

[0100] S31: For any substation road damage image binary<p,p′> In branch D3, p is input into the first Conv module to generate the first normal illumination road damage feature map X1, and p′ is input into the first Conv module to generate the first low illumination road damage feature map X′1. In branch D3, p is input into the Decomposition-based Reflection and Illumination Decomposition module DRD to generate the first normal illumination road damage reflectance feature map R1, and p′ is input into the DRD module to generate the first low illumination road damage reflectance feature map R′1. For simplicity, X1 and X′1 constitute the first road damage feature map. Figure 2 tuple<X1,X′1> R1 and R′1 constitute the first road damage reflectivity characteristics. Figure 2 tuple<R1,R′1> The first Conv module includes the following components: a Conv2d layer with a kernel size of 3×3 and a stride of 2, a BN layer, and a ReLU activation function, which are connected in sequence. The first Conv module can extract the edge and local features of the image, providing better input data for subsequent layers. The DRD module extracts key features by decomposing the reflectance and illumination components of the image, while reducing the influence of ambient light.

[0101] The DRD module is an existing technology, mainly used to decompose an image into reflectance and illumination components to generate high-quality pseudo-labels. By effectively separating reflectance and illumination information, this module can improve the quality of training data, thereby improving the robustness and performance of the model. It is suitable for the scenario in which this invention is applied.

[0102] S32: Will<X1,X′1> The input is fed into the second Conv module to generate the second road damage feature. Figure 2tuple<X2,X′2> X1 is input into the second Conv module to obtain the second normal illumination road damage feature map X2, and X′1 is input into the second Conv module to obtain the second low illumination road damage feature map X′2. Since the structure of the second Conv module is exactly the same as that of the first Conv module, the continuity and consistency of feature extraction can be ensured. Based on this,<X2,X′2> The data is then passed to the CELAN module to generate a higher-level feature map, namely the third-level road damage feature. Figure 2 tuple<X3,X′3> X2 is input into the CELAN module to obtain the third normal illumination road damage feature map X3, and X'2 is input into the CELAN module to obtain the third low illumination road damage feature map X'3. The CELAN module aims to enhance the global semantic information in the feature maps and provide stronger expressive power for the low illumination substation road damage detection model.

[0103] This invention designs a cross-domain enhanced learning and adaptation network module, CELAN (Cross-domain Enhanced Learning and Adaptation Network module). The CELAN module enhances the global feature capture capability of the low-light substation road damage detection model in low-light scenarios by combining multi-path, multi-scale feature extraction and frequency domain convolution, thereby improving the robustness of the low-light substation road damage detection model in low-light environments and effectively increasing its detection accuracy and generalization performance. The overall structure of the CELAN module is as follows: Figure 3 shown

[0104] The overall execution flow of the CELAN module in S32 is as follows:

[0105] S321: In the first branch,<T1,T′1> The input is fed into a Conv2d layer with a kernel size of 3×3 for convolution operation to generate the second intermediate road damage feature. Figure 2 tuple<T2,T′2> The process involves inputting T1 into a 3×3 Conv2d layer to obtain the second intermediate road damage feature map T2 under normal lighting, and inputting T′1 into a 3×3 Conv2d layer to obtain the second intermediate road damage feature map T′2 under low lighting. This convolutional operation can extract the basic texture information of the substation road damage image while improving the richness of feature representation. The stride of the Conv2d layer is 1, and a zero-padding design is used to ensure...<T2,T′2> The resolution remains unchanged, and then...<T2,T′2> The input is fed into a RepConv layer with a kernel size of 3×3 for convolution operation to obtain the third intermediate road damage feature. Figure 2tuple<T3,T′3> The process involves inputting T2 into a RepConv layer with a 3×3 kernel to obtain the third intermediate road damage feature map T3 under normal lighting conditions, and inputting T′2 into a RepConv layer with a 3×3 kernel to obtain the third intermediate road damage feature map T′3 under low lighting conditions. The reparameterized structure of the RepConv layer effectively improves the speed of the convolutional layer during inference and enhances the nonlinear feature extraction capability during training. Then...<T3,T'3> The input is fed into a FreqConv layer with a kernel size of 5×5 for convolution operation to generate the fourth intermediate road damage feature. Figure 2 tuple<T4,T′4> The process involves inputting T3 into a FreqConv layer with a 5×5 kernel to obtain the fourth intermediate road damage feature map T4 under normal lighting conditions, and inputting T′3 into a FreqConv layer with a 5×5 kernel to obtain the fourth intermediate road damage feature map T′4 under low lighting conditions. Within the FreqConv layer, frequency domain feature extraction further emphasizes the structural features of the damaged area, providing more discriminative information for substation road damage detection in low-light scenarios. Then...<T4,T′4> The input is fed into a FreqConv layer with a kernel size of 7×7 for convolution operation to generate the fifth intermediate road damage feature. Figure 2 tuple<T5,T′5> The fifth intermediate road damage feature map, T5, is obtained by inputting T4 into a FreqConv layer with a kernel size of 7×7. Similarly, T′4 is input into a FreqConv layer with a kernel size of 7×7 to obtain the fifth intermediate road damage feature map, T′5, which is obtained by inputting T′4 into a FreqConv layer with a kernel size of 7×7. By further expanding the receptive field, the FreqConv layer can capture a wider range of contextual information, while simultaneously considering both global and local feature representations of the damaged area. Finally, the...<T5,T′5> The input is fed into a Conv2d layer with a kernel size of 3×3 for convolution operation to generate the sixth intermediate road damage feature. Figure 2 tuple<T6,T′6> T5 is input into a Conv2d layer with a kernel size of 3×3 to obtain the sixth intermediate normal lighting road damage feature map T6, and T′5 is input into a Conv2d layer with a kernel size of 3×3 to obtain the sixth intermediate low lighting road damage feature map T′6.

[0106] After the above multiple convolution operations<T6,T′6> The feature diversity and expressive power have been improved, which has enhanced the ability of the low-light substation road damage detection model to represent substation road damage and provided more substantial semantic information for subsequent feature extraction.

[0107] Among them, the reparameterization structure of the RepConv layer is a method to optimize the network inference efficiency. During the training phase, it uses multi-branch structures (such as 3×3 convolutions, 1×1 convolutions, and skip connections) to enhance feature representation capabilities. During the inference phase, it uses structural reparameterization to equivalently merge multiple branches into a standard 3×3 convolution, thereby reducing computation and accelerating inference. This method improves inference efficiency while ensuring the model's expressive power, making it more advantageous in computationally constrained environments.

[0108] S322: In the second branch,<T1,T′1> The input is fed into a continuous convolutional structure and dilated convolution is performed to obtain the damage features of the seventh intermediate road. Figure 2 tuple<T7,T′7> The method involves inputting T1 into a continuous convolutional structure to obtain the seventh intermediate road damage feature map T7 under normal lighting, and inputting T′1 into the continuous convolutional structure to obtain the seventh intermediate road damage feature map T′7 under low lighting. The continuous convolutional structure consists of a Conv2d layer with a kernel size of 1×1, a Conv2d layer with a kernel size of 9×9, and a Conv2d layer with a kernel size of 3×3, connected in series. The dilation of the Conv2d layers with kernel size of 1×1 and 9×9 is 0, while the dilation of the Conv2d layer with kernel size of 3×3 is 9. The dilated convolution operation can effectively expand the receptive field, capturing global features while maintaining the integrity of detailed features in the substation road damage area.

[0109] S323: Will<T2,T′2> ,<T3,T'3> ,<T4,T′4> ,<T5,T′5> ,<T6,T′6> and

[0110] <T7,T′7> Perform a Concat operation to generate the eighth intermediate road damage feature. Figure 2 tuple<T8,T′8> The concat operation is performed on T2, T3, T4, T5, T6 and T7 to obtain the eighth intermediate normal illumination road damage feature map T8, and the concat operation is performed on T′2, T′3, T′4, T′5, T′6 and T′7 to obtain the eighth intermediate low illumination road damage feature map T′8. This achieves effective fusion of multi-scale features and information from different receptive fields, further enhancing the comprehensive expression ability of the low illumination substation road damage detection model for detailed and global features in low illumination scenarios.

[0111] S324: Will<T8,T′8> The input is fed into a Conv2d layer with a kernel size of 5×5 for convolution, and then activated using the Sigmoid activation function to generate the ninth intermediate road damage feature. Figure 2 tuple<T9,T′9> The T8 layer is input into a Conv2d layer with a kernel size of 5×5 and activated using the Sigmoid activation function to obtain the ninth intermediate road damage feature map T9 under normal lighting. The T′8 layer is input into a Conv2d layer with a kernel size of 5×5 and activated using the Sigmoid activation function to obtain the ninth intermediate road damage feature map T′9 under low lighting.<T9,T′9> As the final output of the CELAN module, it provides key input for subsequent feature extraction and prediction tasks;

[0112] Example:

[0113] Will<T1,T′1> Input is routed into two branches; in the first branch, ...<T1,T′1> The input is fed into a Conv2d layer with a kernel size of 3×3 for convolution operation, generating...<T2,T′2> Where T1 and T′1 are both 128×128×128 in size, and T2 and T′2 are both 128×128×256 in size; then...<T2,T′2> The input is fed into a RepConv layer with a kernel size of 3×3 for convolution operation, resulting in...<T3,T'3> The sizes of T3 and T′3 are both 128×128×256; then...<T3,T′3> The input is fed into a FreqConv layer with a kernel size of 5×5 for convolution operation, generating...<T4,T′4> The sizes of T4 and T′4 are both 128×128×256; then...<T4,T′4> The input is fed into a FreqConv layer with a kernel size of 7×7 for convolution operation, generating...<T5,T′5> The sizes of T5 and T′5 are both 128×128×256; finally,<T5,T′5> The input is fed into a Conv2d layer with a kernel size of 3×3 for convolution operation, generating...<T6,T′6> , where the size of T6 and T′6 is 128×128×256;

[0114] In the second branch,<T1,T'1> The input is given to a continuous convolutional structure and dilated convolution is performed to obtain...<T7,T'7> , where the size of T7 and T'7 is 128×128×256;

[0115] Next,<T2,T'2> ,<T3,T'3> ,<T4,T'4> ,<T5,T'5> ,<T6,T'6> and<T7,T'7> Perform a Concat operation to generate<T8,T'8> , where the size of T8 and T'8 is 128×128×1536;

[0116] Finally,<T8,T'8> The input is fed into a Conv2d layer with a kernel size of 5×5 for convolution, and activated using the Sigmoid activation function to generate...<T9,T'9> , where the size of T9 and T'9 is 128×128×512;

[0117] S33: Will<X2,X′2> The input is fed into the DLARK module to generate the second road damage reflectivity feature. Figure 2 tuple<R2,R′2> The X2 is input into the DLARK module to obtain the second normal illumination road damage reflectance feature map R2, and the X'1 is input into the DLARK module to obtain the second low illumination road damage reflectance feature map R'2. The DLARK module performs depth extraction on the reflectance features of the substation road damage image to capture details and features, thereby providing input for the subsequent IRC module.

[0118] This invention designs a Dark Light Attenuation and Reflection Knowledge Network module (DLARK). The DLARK module is used for reflectance feature extraction. By fusing multi-scale features and an attention mechanism, it improves the adaptability of the low-light substation road damage detection model under low-light conditions. The DLARK module utilizes convolutional operations with different dilation rates to capture multi-scale information, enhancing the perception of target areas. Simultaneously, it combines an attention mechanism to focus on key information regions, effectively reducing background noise interference and strengthening the expression of detailed features in complex scenes. By separately optimizing the reflectance representation, the DLARK module helps the low-light substation road damage detection model extract inherent information related to the object itself from the image, avoiding interference from complex lighting conditions on target detection, thereby increasing the contribution of reflectance to the target detection task and reducing the negative impact of lighting information. Furthermore, this module decouples the learning of reflectance features from the target detection task, better controlling and strengthening the learning of reflectance information, ultimately effectively improving the robustness and accuracy of the low-light substation road damage detection model in low-light scenarios. The structure of the DLARK module is as follows: Figure 4 As shown:

[0119] The overall execution flow of the DLARK module in S33 is as follows:

[0120] S331: In the first branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 9×9 and a dilation rate of 1, and a BN layer for convolution and normalization operations to obtain the first intermediate reflectance feature. Figure 2 tuple <X 2_1 ,X' 2_1>, where X2 is input into this convolutional structure to obtain the first intermediate normal illumination reflectance feature map X. 2_1 Inputting X'2 into this convolutional structure yields the first intermediate low-illuminance reflectance feature map X'. 2_1 This convolution operation captures a wide range of contextual information through a large convolution kernel, while ensuring the stability and consistency of feature representation.

[0121] S332: In the second branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 5×5 and a dilation rate of 1 and a BN layer for convolution and normalization operations to generate a second intermediate reflectance feature. Figure 2 tuple <X 2_2 ,X′ 2_2 >, where X2 is input into this convolutional structure to obtain the second intermediate normal illumination reflectance feature map X. 2_2 Inputting X′2 into this convolutional structure yields the second intermediate low-illuminance reflectance feature map X′. 2_2 The smaller convolutional kernels in this operation focus on extracting detailed features, providing important support for subsequent multi-scale fusion.

[0122] S333: In the third branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 5×5 and a dilation rate of 2, and a BN layer, to perform convolution and normalization operations, thereby obtaining the third intermediate reflectance feature. Figure 2 tuple <X 2_3 ,X' 2_3 >, where X2 is input into this convolutional structure to obtain the third intermediate normal illumination reflectance feature map X. 2_3 Inputting X'2 into this convolutional structure yields the third intermediate low-illuminance reflectance feature map X'. 2_3 Increasing the dilation rate in this convolution operation will further expand the receptive field, effectively capturing the correlation information between distant features.

[0123] S334: In the fourth branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 2×2 and a BN layer, and undergoes convolution and normalization operations to generate the fourth intermediate reflectance feature. Figure 2 tuple <X 2_4 ,X' 2_4 >, where X2 is input into this convolutional structure to obtain the fourth intermediate normal illumination reflectance feature map X. 2_4 Inputting X′2 into this convolutional structure yields the fourth intermediate low-light reflectivity feature map X'. 2_4 ;

[0124] S335: In the fifth branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 2×2 and a dilation rate of 4, and a BN layer, for convolution and normalization operations to obtain the fifth intermediate reflectance feature. Figure 2 tuple <X 2_5 ,X' 2_5 >, where X2 is input into this convolutional structure to obtain the fifth intermediate normal illumination reflectance feature map X. 2_5 Inputting X'2 into this convolutional structure yields the fifth intermediate low-illuminance reflectance feature map X'. 2_5 Through its multi-void design, the DLARK module is capable of capturing features at different scales, and is particularly effective in low-light scenes.

[0125] S336: Yes <X 2_1 ,X' 2_1 >、 <X 2_2 ,X′ 2_2 >、 <X 2_3 ,X' 2_3 >、 <X 2_4 ,X' 2_4 >and

[0126] <X 2_5 ,X' 2_5 Perform a Concat operation to obtain the sixth intermediate reflectance feature. Figure 2 tuple <X 2_6 ,X' 2_6 >, where X 2_1 X 2_2 X 2_3 X 2_4 and X 2_5 Performing the Concat operation yields the sixth intermediate normal illumination reflectance feature map X. 2_6 , will X' 2_1 、X' 2_2 、X' 2_3 、X' 2_4 and X' 2_5 Performing the Concat operation yields the sixth intermediate low-illuminance reflectance feature map X'. 2_6 This Concat operation integrates multi-scale information, resulting in a more comprehensive feature representation; <X 2_6 ,X' 2_6 The input is fed into the AvgPool layer for pooling operations to obtain the seventh intermediate reflectance feature. Figure 2 tuple <X 2_7 ,X' 2_7 >, where X 2_6 The input to the AvgPool layer yields the seventh intermediate normal illumination reflectance feature map X. 2_7 , will X'2_6 The input to the AvgPool layer yields the seventh intermediate low-illuminance reflectance feature map X'. 2_7 Pooling operations enhance the globality of features through downsampling, while reducing the impact of redundant features. <X 2_7 ,X' 2_7 The input is fed into a continuous convolutional structure consisting of a Conv2d layer with a kernel size of 1×1, a DWConv layer with a kernel size of 2×2, a DWConv layer with a kernel size of 5×5, and a Conv2d layer with a kernel size of 2×2, and a depthwise convolution operation is performed to generate the eighth intermediate reflectance feature. Figure 2 tuple <X 2_8 ,X' 2_8 >, where X 2_7 The input is fed into a continuous convolutional structure to obtain the eighth intermediate normal illumination reflectance feature map X. 2_8 , X′ 2_7 The input is fed into a continuous convolutional structure to obtain the eighth intermediate low-illuminance reflectance feature map X'. 2_8 The introduction of depthwise convolution greatly reduces the computational cost while preserving key features, in addition to... <X 2_6 ,X′ 2_6 While performing pooling operations, <X 2_6 ,X' 2_6 The input is fed into a BN layer for normalization to generate the ninth intermediate reflectance feature. Figure 2 tuple <X 2_9 ,X' 2_9 >, where X 2_6 The input is fed into the BN layer to obtain the ninth intermediate normal illumination reflectance feature map X. 2_9 , will X' 2_6 The input to the BN layer yields the ninth intermediate low-illuminance reflectance feature map X'. 2_9 Normalization avoids the vanishing or exploding gradient problem, thereby improving training efficiency and playing a regularization role to prevent overfitting.

[0127] S337: Will <X 2_9 ,X' 2_9 The input is fed into the AvgPool layer for pooling to obtain the tenth intermediate reflectance feature. Figure 2 tuple <X 2_10 ,X' 2_10 >, where X 2_9 The input to the AvgPool layer yields the tenth intermediate normal illumination reflectance feature map X. 2_10 , will X' 2_9 The input to the AvgPool layer yields the tenth intermediate low-illuminance reflectance feature map X'. 2_10 Next <X 2_10 ,X'2_10 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence for feature transformation to obtain the eleventh intermediate reflectance feature. Figure 2 tuple <X 2_11 ,X′ 2_11 >, where X 2_10 The input is fed into a structure consisting of fully connected layers, ReLU activation functions, and another fully connected layer in sequence to obtain the eleventh intermediate normal illumination reflectance feature map X. 2_11 , will X' 2_10 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence to obtain the eleventh intermediate low-illuminance reflectance feature map X'. 2_11 The feature transformation operation further integrates the pooled features, enhancing the module's expressive power and providing a more abstract feature representation for subsequent activation and decision-making processes. <X 2_11 ,X' 2_11 >Activation was performed using the Sigmoid activation function to obtain the twelfth intermediate reflectance feature. Figure 2 tuple <X 2_12 ,X' 2_12 >, where X 2_11 The twelfth intermediate normal illumination reflectance feature map X was obtained by activating the Sigmoid function. 2_12 , will X' 2_11 The twelfth intermediate low-illuminance reflectance feature map X′ was obtained by activating the Sigmoid function. 2_12 ,Will <X 2_9 ,X′ 2_9 >and <X 2_12 ,X' 2_12 Perform element-wise multiplication to obtain the thirteenth intermediate reflectance feature. Figure 2 tuple <X 2_13 ,X' 2_13 >, where X 2_9 With X 2_12 Element-wise multiplication yields the thirteenth intermediate normal illumination reflectance characteristic map X. 2_13 , will X' 2_9 With X' 2_12 Element-wise multiplication yields the thirteenth intermediate low-illuminance reflectance feature map X'. 2_13 This element-wise multiplication operation introduces interactive features, making... <X 2_9 ,X′ 2_9 >and <X 2_12 ,X' 2_12 By fusing information, the feature representation capability is enhanced. Element-wise multiplication can strengthen specific features and suppress irrelevant parts, thereby increasing the module's sensitivity to important features and improving the accuracy of the final output. <X 2_13,X' 2_13 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence for feature transformation, resulting in the fourteenth intermediate reflectance feature. Figure 2 tuple <X 2_14 ,X' 2_14 >, where X 2_13 The input is fed into a structure consisting of fully connected layers, ReLU activation functions, and another fully connected layer in sequence to obtain the fourteenth intermediate normal illumination reflectance feature map X. 2_14 , will X' 2_13 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence to obtain the fourteenth intermediate low-illuminance reflectance feature map X′. 2_14 ;

[0128] S338: Yes <X 2_14 ,X' 2_14 After performing the DropPath operation, the fifteenth intermediate reflectance feature is obtained. Figure 2 tuple <X 2_15 ,X' 2_15 >, where X 2_14 Performing the DropPath operation yields the fifteenth intermediate normal illumination reflectance feature map X. 2_15 , will X' 2_14 Performing the DropPath operation yields the fifteenth intermediate low-illuminance reflectance feature map X'. 2_15 The DropPath operation increases the robustness of the module and reduces overfitting by randomly discarding paths. <X 2_15 ,X' 2_15 >and <X 2_8 ,X' 2_8 Perform a Concat operation to obtain the final second-path damage reflectivity characteristics. Figure 2 tuple<R2,R'2> X 2_15 With X 2_8 Performing a concat operation yields the second normal illumination road damage reflectivity characteristic map R2, and X' 2_15 With X' 2_8 Performing a concat operation yields the second low-light road damage reflectance feature map R'2. This feature... Figure 2 The tuples, as the final output of the DLARK module, provide crucial input for subsequent feature extraction and prediction tasks;

[0129] DropPath is a regularization method that randomly discards some paths during neural network training, thereby forcing the network to learn more robust feature representations. In this way, DropPath avoids the model from over-relying on certain specific paths, enhances the network's generalization ability, and improves performance on unseen data. It is particularly suitable for deep networks and can effectively improve the stability and performance of the model during training.

[0130] Example:

[0131] The second road damage characteristics Figure 2 tuple<X2,X'2> The input is distributed into five branches, where X2 and X'2 are both 128×128×128 in size;

[0132] In the first branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 9×9 and a dilation rate of 1, and a BN layer, for convolution and normalization operations to obtain the desired result. <X 2_1 ,X' 2_1 >, where X 2_1 and X′ 2_1 The size is 128×128×256; in the second branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 5×5 and a dilation rate of 1, and a BN layer, to perform convolution and normalization operations, generating... <X 2_2 ,X' 2_2 >, where X 2_2 and X' 2_2 The size of each is 128×128×128; in the third branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 5×5 and a dilation rate of 2, and a BN layer, for convolution and normalization operations to obtain the desired result. <X 2_3 ,X' 2_3 >, where X 2_3 and X' 2_3 The size of each is 128×128×128; in the fourth branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 2×2 and a BN layer, and convolution and normalization operations are performed to generate... <X 2_4 ,X' 2_4 >, where X 2_4 and X' 2_4 The size of each is 128×128×128; in the fifth branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 2×2 and a dilation rate of 4, and a BN layer, for convolution and normalization operations to obtain the result. <X 2_5 ,X' 2_5>, where X 2_5 and X' 2_5 The size of each is 128×128×128;

[0133] Next, for <X 2_1 ,X' 2_1 >、 <X 2_2 ,X' 2_2 >、 <X 2_3 ,X' 2_3 >、 <X 2_4 ,X' 2_4 >and <X 2_5 ,X' 2_5 Perform a Concat operation to obtain... <X 2_6 ,X' 2_6 >, where X 2_6 and X' 2_6 The size of each is 128×128×768; next, <X 2_6 ,X' 2_6 The input is fed into the AvgPool layer for pooling operations, resulting in... <X 2_7 ,X' 2_7 >, where X 2_7 and X' 2_7 The size of each is 128×128×768; then, <X 2_7 ,X' 2_7 The input is fed into a continuous convolutional structure consisting of a 1×1 Conv2d layer, a 3×3 DWConv layer, a 5×5 DWConv layer, and a 2×2 Conv2d layer, and a depthwise convolution operation is performed to generate... <X 2_8 ,X' 2_8 >, where X 2_8 and X' 2_8 The size is 128×128×512; in the case of <X 2_6 ,X' 2_6 While performing pooling operations, <X 2_6 ,X' 2_6 The input is fed into the BN layer for normalization, generating... <X 2_9 ,X' 2_9 >, where X 2_9 and X' 2_9 The size of each is 64×64×256;

[0134] Will <X 2_9 ,X' 2_9 The input is fed into the AvgPool layer for pooling operations, resulting in... <X 2_10 ,X' 2_10 >, where X 2_10 and X'2_10 The size is 64×64×128; then... <X 2_10 ,X' 2_10 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence for feature transformation operations, resulting in... <X 2_11 ,X' 2_11 >, where X 2_11 and X' 2_11 The size of each is 64×64×512; X 2_11 Activation is performed using the Sigmoid activation function to obtain... <X 2_12 ,X' 2_12 >, where X 2_12 and X' 2_12 The size of each is 64×64×512; next, <X 2_9 ,X' 2_9 >and <X 2_12 ,X' 2_12 Perform element-wise multiplication to obtain <X 2_13 ,X' 2_13 >, where X 2_13 and X' 2_13 The size of each is 64×64×256; then, <X 2_13 ,X' 2_13 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence for feature transformation operations, resulting in... <X 2_14 ,X' 2_14 >, where X 2_14 and X' 2_14 The size is 64×64×128; for <X 2_14 ,X' 2_14 After performing the DropPath operation, you get <X 2_15 ,X' 2_15 >, where X 2_15 and X' 2_15 The size of each is 64×64×128; finally, <X 2_15 ,X' 2_15 >and <X 2_8 ,X' 2_8 Perform a Concat operation to obtain...<R2,R'2> The sizes of R2 and R'2 are both 64×64×640;

[0135] S34: To further distinguish between reflectance and illumination components in an image, and to maintain consistency between illumination and reflectance,<R1,R'1> and<R2,R′2> The data is input into the Illumination and Reflection Compensation (IRC) module, where it undergoes a process involving interaction, recombination, and consistency constraints to generate the third road damage reflectivity characteristics. Figure 2 tuple<R3,R'3> R1 and R2 are input into the IRC module to obtain the third normal illumination road damage reflectance feature map R3, and R'1 and R'2 are input into the IRC module to obtain the third low illumination road damage reflectance feature map R′3.

[0136] S35: Yes<R3,R′3> and<X3,X′3> Perform a concat operation to fuse reflectivity features with high-level semantic features to obtain the fourth road damage feature. Figure 2 tuple<X4,X'4> The concat operation of R3 and X3 yields the fourth normal illumination road damage feature map X4, and the concat operation of R′3 and X′3 yields the fourth low illumination road damage feature map X′4.

[0137] S36: Will<X4,X'4> The data is input into the target prediction head to generate tensors containing prediction information. Each row of the tensor corresponds to a detection result, including the predicted bounding box coordinates, damage category label, and confidence score information. The target prediction head achieves accurate detection and classification of substation road damage images through hierarchical extraction, feature fusion, and consistency modeling. At the same time, it improves the adaptability and robustness of the low-light substation road damage detection model in low-light environments.

[0138] In deep learning object detection models, the Head module is responsible for in-depth analysis and processing of feature maps to predict the object's category, location, and confidence level of its presence. The output of the Head module is typically a three-dimensional tensor, with each row containing prediction information at a specific spatial location, including the category probability of each predicted box, the bounding box coordinates, and the confidence level of the object's presence. With this information, the model can accurately locate and classify objects.

[0139] (1) Boundary box prediction

[0140] In object detection tasks, images are typically divided into multiple grids, with each grid cell responsible for predicting several bounding boxes. The task of bounding box prediction is to determine the location of objects in the image, and mainly includes the following parameters:

[0141] Center point coordinates (x, y): The position of the bounding box center in the entire image is determined by offset calculation relative to the center point of the feature map cell;

[0142] Width and height (w, h): The width and height of the bounding box, representing the width and height relative to the entire image, to ensure the accurate scale and position of the predicted box;

[0143] To stabilize training, bounding box coordinates are usually normalized using the Sigmoid function, which limits the predicted values ​​to between 0 and 1. This normalization operation helps improve the accuracy of coordinate prediction and avoids the box size being too large or too small, thereby reducing prediction bias.

[0144] (2) Category and confidence prediction

[0145] In the prediction of each bounding box, in addition to coordinates, it is also necessary to predict the object's class and confidence level:

[0146] Category prediction: The model outputs a probability vector, representing the probability of each category that may exist within the bounding box; using this probability, the model can identify the specific type of object, such as an insulator or a tower.

[0147] Confidence prediction: Confidence represents the model's confidence in the existence of an object in the current bounding box; its value ranges from 0 to 1, and the higher the value, the more confident the model is that the bounding box contains the target object; in the post-processing stage, bounding boxes with high confidence are usually selected according to the set confidence threshold, thereby improving the accuracy of the detection results;

[0148] In practice, the Head module outputs a three-dimensional tensor containing detailed information about all predicted boxes at each spatial location, namely the class probability, bounding box coordinates, and confidence score. For each location in the feature map, this tensor provides the prediction information for the corresponding bounding box. This design enables the model to output classification and localization information simultaneously, thereby improving the accuracy and efficiency of detection.

[0149] Example:

[0150] Let the size of the feature map be H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map;

[0151] The binary pair<p,p'> ∈D3 is input into the first Conv module to generate<X1,X'1> Where p and p' are both 512×512×3, and X1 and X'1 are both 256×256×64; the binary pair<p,p'> ∈D3 is input into the DRD module to generate<R1,R'1> , where the size of R1 and R'1 is 64×64×640;

[0152] Next,<X1,X'1> Input into the second Conv module to generate<X2,X'2> Where both X2 and X'2 are 128×128×128;<X2,X'2> Transferred to the CELAN module to generate<X3,X'3> Where X3 and X'3 are both 128×128×512;<X2,X'2> While transmitting to the CELAN module, it will also<X2,X'2> Input into the DLARK module to generate<R2,R'2> The sizes of R2 and R'2 are both 64×64×640;

[0153] To further distinguish between reflectance and illumination components in the image, and to maintain consistency between illumination and reflectance,<R1,R'1> and<R2,R'2> Input into the IRC module to generate<R3,R'3> , where the size of R3 and R'3 is 128×128×256;

[0154] Next, regarding<R3,R'3> and<X3,X'3> Perform a concat operation to fuse reflectivity features with high-level semantic features, and obtain<X4,X'4> , where the size of both X4 and X'4 is 128×128×768;

[0155] Finally,<X4,X'4> The data is input into the target prediction head to generate a tensor containing prediction information. Each row of the tensor corresponds to a detection result, including the predicted bounding box coordinates, damage category label, and confidence score. The entire process achieves accurate detection and classification of substation road damage images through hierarchical extraction, feature fusion, and consistency modeling, while improving the model's adaptability and robustness in low-light environments.

[0156] S4: Before training the substation road damage detection model under low light conditions, it is necessary to set hyperparameters, such as selecting an optimizer, determining the initial learning rate, batch size, and number of training rounds. After completing the initial configuration, the paired substation road damage image datasets are divided into training and validation sets to ensure the independence and representativeness of each dataset.

[0157] The specific steps of S4 are as follows:

[0158] S41: During training, the model extracts data from the training set according to the set batch size, performs forward calculations, generates prediction results, and then performs backpropagation through the overall loss function L2R-LOSS to optimize the parameters of the substation road damage detection model under low light conditions.

[0159] This invention designs an L2R-LOSS loss function for training a substation road damage detection model under low light conditions, solving the problem of decreased target detection accuracy under low light conditions. The L2R-LOSS loss function enhances the model's target feature extraction capability in low light environments, thereby improving detection stability and accuracy. By fusing reflectivity learning loss, recombination consistency loss, and target detection loss, multiple losses are synergistically optimized, reducing the impact of light changes on detection results, enhancing the model's adaptability under low light conditions, and improving the robustness of substation road damage detection.

[0160] The loss function L2R-LOSS in S41 is derived from the reflectance learning loss function L r Reorganization Consistency Loss Function L rc And the target detection loss function L target The weighted composition, its functional expression is as follows:

[0161] L L2R-LOSS =λ g L r +λ f L target +λ q L rc

[0162] Where, λ g , λ f and λ q To account for the loss weights, the sum of the three is 1;

[0163] Among them, L r The function expression is as follows:

[0164]

[0165] Where R'1 is the first low-light road damage reflectivity feature map; R'2 is the first low-light road damage reflectivity feature map; N is the total number of feature points in the R'1 and R'2 feature maps, R' 1_i and R' 2_i These represent the feature response values ​​corresponding to the i-th position in the feature map; and Let R'1 and R'2 represent the average characteristic response values, respectively. and These are the variances of the characteristic response values ​​of R'1 and R'2, respectively. It is the covariance of the characteristic response values ​​between R'1 and R'2, and C1 and C2 are constants to avoid the denominator being zero;

[0166] Among them, L target The function expression is as follows:

[0167]

[0168] Where α and β represent the weights of the location regression loss and classification loss, respectively; x i and x' i p represents the center point of the ground truth bounding box and the center point of the predicted bounding box for the i-th object, respectively; i and p' i Let represent the true class label and predicted class probability of the i-th target, respectively. Smooth(x) represents the error between the predicted and true values, i.e.

[0169]

[0170] L rc The function expression is as follows:

[0171] L rc =||R1-R′2||1+||R′1-R2||1

[0172] Wherein, R1 is the first normal illumination road damage reflectance characteristic map; R'2 is the second low illumination road damage reflectance characteristic map; R'1 is the first low illumination road damage reflectance characteristic map; R2 is the second normal illumination road damage reflectance characteristic map;

[0173] S42: After each round of training, the data from the validation set is used to evaluate the model and determine its generalization ability.

[0174] S43: To avoid overfitting during training, an early stopping mechanism is introduced. If the loss on the validation set does not improve significantly in multiple consecutive training rounds, the training will be terminated early to save computational resources and ensure the stability of the model in practical applications. Overfitting refers to a model that performs well on the training set but deteriorates on the validation set.

[0175] S44: When the training reaches the predetermined number of rounds and the validation set loss is stable or reaches the ideal level, the training process ends, and an optimized low-light substation road damage detection model is finally obtained. The model can accurately detect substation road damage under low light conditions and maintain good performance in different environments.

[0176] S45: After training, the model will have appropriate parameter configuration and strong generalization ability, and can be applied to new substation road damage detection tasks.

[0177] S5: After the model training is completed, the trained low-light substation road damage detection model is used to identify and analyze the current substation road damage data. The final output includes the specific substation road damage type and location identification results, providing substation inspection personnel with accurate early warning information and subsequent handling suggestions.

[0178] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for detecting road damage in low-light substations based on deep learning, characterized in that, The specific steps are as follows: S1: Construct the original substation road damage image dataset D1; S2: Preprocess D1 to obtain the processed substation road damage image dataset D2, and perform low-light degradation synthesis on each substation road damage image p in D2 to obtain a low-light version image p', forming a substation road damage image tuple.<p,p'> A paired substation road damage image dataset D3 is constructed from all the substation road damage image pairs. S3: Construct a deep learning-based low-light substation road damage detection model. Input the paired substation road damage image dataset D3 into the low-light substation road damage detection model to identify the road damage types in the substation road damage images and automatically generate localization boxes to determine the specific location and extent of the damage. The road damage types include cracks, potholes, warping, and settlement. S4: Before training the low-light substation road damage detection model, it is necessary to first set the hyperparameters. After completing the initial configuration, the paired substation road damage image datasets are divided into training set and validation set to ensure the independence and representativeness of each dataset. S5: After the model training is completed, the trained low-light substation road damage detection model is used to identify and analyze the current substation road damage data. The final output includes the specific substation road damage type and location identification results, providing substation inspection personnel with accurate early warning information and subsequent handling suggestions.

2. The method for detecting road damage in low-light substations based on deep learning according to claim 1, characterized in that, The specific steps of S1 are as follows: S11: Collect images of substation road damage under normal lighting conditions to form the original substation road damage image dataset D1; S12: Manually label D1 to provide labels for subsequent model training. During the labeling process, use the labeling tool to select the damaged road areas and add corresponding labels to each damaged area to mark the type of road damage in that area.

3. The method for detecting road damage in low-light substations based on deep learning according to claim 1, characterized in that, The specific steps of S2 are as follows: S21: Crop all substation road damage images in D1 to a uniform size; S22: Expand D1 using data augmentation operations such as rotation, flipping, and noise addition to generate a substation road damage image dataset D2 containing more substation road damage images; S23: Process each substation road damage image p in D2 using brightness adjustment, contrast adjustment, color saturation reduction, blurring and color shifting operations to generate the corresponding low-light version image p'. S24: Combine p and p' to form a binary image of road damage at a substation.<p,p'> A paired substation road damage image dataset D3 is constructed from all the above substation road damage image pairs.

4. The method for detecting road damage in low-light substations based on deep learning according to claim 1, characterized in that, The specific steps of S3 are as follows: S31: For any substation road damage image binary<p,p'> In the first branch, p is input into the first Conv module to generate the first normal illumination road damage feature map X1, and p' is input into the first Conv module to generate the first low illumination road damage feature map X'1. In the second branch, p is input into the decomposed reflectance and illumination decomposition module DRD to generate the first normal illumination road damage reflectance feature map R1, and p' is input into the DRD module to generate the first low illumination road damage reflectance feature map R'1. S32: Will<X1,X'1> The input is fed into the second Conv module to generate a second road damage feature map tuple.<X2,X'2> X1 is input into the second Conv module to obtain the second normal illumination road damage feature map X2, and X'1 is input into the second Conv module to obtain the second low illumination road damage feature map X'2. Since the structure of the second Conv module is exactly the same as that of the first Conv module, the continuity and consistency of feature extraction can be ensured. Based on this,<X2,X'2> The data is passed to the CELAN module to generate a higher-level feature map, namely the third road damage feature map tuple.<X3,X'3> X2 is input into the CELAN module to obtain the third normal illumination road damage feature map X3, and X'2 is input into the CELAN module to obtain the third low illumination road damage feature map X'3. S33: Will<X2,X'2> The input is fed into the DLARK module to generate a second road damage reflectivity feature map tuple.<R2,R'2> X2 is input into the DLARK module to obtain the second normal illumination road damage reflectivity feature map R2, and X'1 is input into the DLARK module to obtain the second low illumination road damage reflectivity feature map R'2. S34: To further distinguish between reflectance and illumination components in an image, and to maintain consistency between illumination and reflectance,<R1,R'1> and<R2,R'2> The data is input into the IRC (Illumination and Reflection Compensation) module. Through a process involving interaction, recombination, and consistency constraints, a tuple of the third road damage reflectivity feature map is generated.<R3,R'3> R1 and R2 are input into the IRC module to obtain the third normal illumination road damage reflectance feature map R3, and R'1 and R'2 are input into the IRC module to obtain the third low illumination road damage reflectance feature map R'3. S35: Yes<R3,R'3> and<X3,X'3> Perform a concat operation to fuse reflectivity features and high-level semantic features to obtain a tuple of the fourth road damage feature map.<X4,X'4> The concat operation of R3 and X3 yields the fourth normal illumination road damage feature map X4, and the concat operation of R'3 and X'3 yields the fourth low illumination road damage feature map X'4. S36: Will<X4,X'4> The input is fed into the target prediction head to generate a tensor containing prediction information. Each row of the tensor corresponds to a detection result, including the predicted bounding box coordinates, damage category label, and confidence score information.

5. The method for detecting road damage in low-light substations based on deep learning according to claim 4, characterized in that, The overall execution flow of the CELAN module in S32 is as follows: S321: In the first branch,<T1,T'1> The input is fed into a Conv2d layer with a kernel size of 3×3 for convolution operation, generating a second intermediate road damage feature map tuple.<T2,T'2> The process involves inputting T1 into a Conv2d layer with a 3×3 kernel to obtain the second intermediate road damage feature map T2 under normal lighting conditions, and inputting T'1 into a Conv2d layer with a 3×3 kernel to obtain the second intermediate road damage feature map T'2 under low lighting conditions. Then...<T2,T'2> The input is fed into a RepConv layer with a kernel size of 3×3 for convolution, resulting in a tuple of the third intermediate road damage feature map.<T3,T'3> The process involves inputting T2 into a RepConv layer with a 3×3 kernel to obtain the third intermediate road damage feature map T3 under normal lighting conditions, and inputting T'2 into a RepConv layer with a 3×3 kernel to obtain the third intermediate road damage feature map T'3 under low lighting conditions. Then...<T3,T'3> The input is fed into a FreqConv layer with a kernel size of 5×5 for convolution, generating a tuple of the fourth intermediate road damage feature map.<T4,T'4> The process involves inputting T3 into a FreqConv layer with a 5×5 kernel to obtain the fourth intermediate road damage feature map T4 under normal lighting conditions, and inputting T'3 into a FreqConv layer with a 5×5 kernel to obtain the fourth intermediate road damage feature map T'4 under low lighting conditions.<T4,T'4> The input is fed into a FreqConv layer with a kernel size of 7×7 for convolution, generating a tuple of the fifth intermediate road damage feature map.<T5,T'5> The fifth intermediate road damage feature map T5 under normal lighting is obtained by inputting T4 into a FreqConv layer with a kernel size of 7×7. The fifth intermediate road damage feature map T'5 under low lighting is obtained by inputting T'4 into a FreqConv layer with a kernel size of 7×7. Finally,<T5,T'5> The input is fed into a Conv2d layer with a kernel size of 3×3 for convolution operation, generating a tuple of the sixth intermediate road damage feature map.<T6,T'6> T5 is input into a Conv2d layer with a kernel size of 3×3 to obtain the sixth intermediate normal lighting road damage feature map T6, and T'5 is input into a Conv2d layer with a kernel size of 3×3 to obtain the sixth intermediate low lighting road damage feature map T'6. S322: In the second branch,<T1,T'1> The input is fed into a continuous convolutional structure and dilated convolution is performed to obtain a binary tuple of the feature map of the seventh intermediate road damage.<T7,T'7> T1 is input into a continuous convolutional structure to obtain the seventh intermediate road damage feature map T7 under normal lighting, and T'1 is input into a continuous convolutional structure to obtain the seventh intermediate road damage feature map T'7 under low lighting. S323: Will<T2,T'2> ,<T3,T'3> ,<T4,T'4> ,<T5,T'5> ,<T6,T'6> and <T7,T'7> Perform a Concat operation to generate a binary tuple of the eighth intermediate road damage feature map.<T8,T'8> The concat operation is performed on T2, T3, T4, T5, T6 and T7 to obtain the eighth intermediate normal illumination road damage feature map T8, and the concat operation is performed on T'2, T'3, T'4, T'5, T'6 and T'7 to obtain the eighth intermediate low illumination road damage feature map T'8. This achieves effective fusion of multi-scale features and information from different receptive fields, further enhancing the comprehensive expression ability of the low illumination substation road damage detection model for detailed and global features in low illumination scenarios. S324: Will<T8,T'8> The input is fed into a Conv2d layer with a kernel size of 5×5 for convolution, and then activated using the Sigmoid activation function to generate a tuple of the ninth intermediate road damage feature map.<T9,T'9> The method involves inputting T8 into a Conv2d layer with a kernel size of 5×5 and activating it using the Sigmoid activation function to obtain the ninth intermediate road damage feature map T9 under normal lighting conditions. Similarly, inputting T'8 into a Conv2d layer with a kernel size of 5×5 and activating it using the Sigmoid activation function yields the ninth intermediate road damage feature map T'9 under low lighting conditions.<T9,T'9> As the final output of the CELAN module, it provides crucial input for subsequent feature extraction and prediction tasks.

6. The method for detecting road damage in low-light substations based on deep learning according to claim 4, characterized in that, The overall execution flow of the DLARK module in S33 is as follows: S331: In the first branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 9×9 and a dilation rate of 1, and a BN layer for convolution and normalization operations to obtain the first intermediate reflectance feature map tuple. <X 2_1 ,X' 2_1 >, where X2 is input into this convolutional structure to obtain the first intermediate normal illumination reflectance feature map X. 2_1 Inputting X'2 into this convolutional structure yields the first intermediate low-illuminance reflectance feature map X'. 2_1 ; S332: In the second branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 5×5 and a dilation rate of 1 and a BN layer for convolution and normalization operations, generating a second intermediate reflectance feature map tuple. <X 2_2 ,X' 2_2 >, where X2 is input into this convolutional structure to obtain the second intermediate normal illumination reflectance feature map X. 2_2 Inputting X'2 into this convolutional structure yields the second intermediate low-illuminance reflectance feature map X'. 2_2 ; S333: In the third branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 5×5 and a dilation rate of 2, and a BN layer, to perform convolution and normalization operations, resulting in a third intermediate reflectance feature map tuple. <X 2_3 ,X' 2_3 >, where X2 is input into this convolutional structure to obtain the third intermediate normal illumination reflectance feature map X. 2_3 Inputting X'2 into this convolutional structure yields the third intermediate low-illuminance reflectance feature map X'. 2_3 ; S334: In the fourth branch,<X2,X′2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 2×2 and a BN layer, and undergoes convolution and normalization operations to generate a fourth intermediate reflectance feature map tuple. <X 2_4 ,X' 2_4 >, where X2 is input into this convolutional structure to obtain the fourth intermediate normal illumination reflectance feature map X. 2_4 Inputting X'2 into this convolutional structure yields the fourth intermediate low-light reflectivity feature map X'. 2_4 ; S335: In the fifth branch,<X2,X'2> The input is fed into a convolutional structure consisting of a Conv2d layer with a kernel size of 2×2 and a dilation rate of 4, and a BN layer. Convolution and normalization operations are performed to obtain the fifth intermediate reflectance feature map tuple. <X 2_5 ,X' 2_5 >, where X2 is input into this convolutional structure to obtain the fifth intermediate normal illumination reflectance feature map X. 2_5 Inputting X'2 into this convolutional structure yields the fifth intermediate low-illuminance reflectance feature map X'. 2_5 ; S336: Yes <X 2_1 ,X' 2_1 >、 <X 2_2 ,X' 2_2 >、 <X 2_3 ,X' 2_3 >、 <X 2_4 ,X' 2_4 >and <X 2_5 ,X' 2_5 Perform a concat operation to obtain the sixth intermediate reflectivity feature map tuple. <X 2_6 ,X' 2_6 >, where X 2_1 X 2_2 X 2_3 X 2_4 and X 2_5 Performing the Concat operation yields the sixth intermediate normal illumination reflectance feature map X. 2_6 , will X' 2_1 、X' 2_2 、X' 2_3 、X' 2_4 and X' 2_5 Performing the Concat operation yields the sixth intermediate low-illuminance reflectance feature map X'. 2_6 ,Will <X 2_6 ,X' 2_6 The input is fed into the AvgPool layer for pooling operations, resulting in the seventh intermediate reflectance feature map tuple. <X 2_7 ,X' 2_7 >, where X 2_6 The input to the AvgPool layer yields the seventh intermediate normal illumination reflectance feature map X. 2_7 , will X' 2_6 The input to the AvgPool layer yields the seventh intermediate low-illuminance reflectance feature map X'. 2_7 ,Will <X 2_7 ,X' 2_7 The input is fed into a continuous convolutional structure consisting of a Conv2d layer with a kernel size of 1×1, a DWConv layer with a kernel size of 2×2, a DWConv layer with a kernel size of 5×5, and a Conv2d layer with a kernel size of 2×2, and a depthwise convolution operation is performed to generate an eighth intermediate reflectance feature map tuple. <X 2_8 ,X' 2_8 >, where X 2_7 The input is fed into a continuous convolutional structure to obtain the eighth intermediate normal illumination reflectance feature map X. 2_8 , X′ 2_7 The input is fed into a continuous convolutional structure to obtain the eighth intermediate low-illuminance reflectance feature map X′. 2_8 In the <X 2_6 ,X' 2_6 While performing pooling operations, <X 2_6 ,X' 2_6 The input is fed into a BN layer for normalization, generating the ninth intermediate reflectance feature map tuple. <X 2_9 ,X' 2_9 >, where X 2_6 The input is fed into the BN layer to obtain the ninth intermediate normal illumination reflectance feature map X. 2_9 , will X' 2_6 The input to the BN layer yields the ninth intermediate low-illuminance reflectance feature map X'. 2_9 ; S337: Will <X 2_9 ,X' 2_9 The input is fed into the AvgPool layer for pooling, resulting in the tenth intermediate reflectivity feature map tuple. <X 2_10 ,X' 2_10 >, where X 2_9 The input to the AvgPool layer yields the tenth intermediate normal illumination reflectance feature map X. 2_10 , will X' 2_9 The input to the AvgPool layer yields the tenth intermediate low-illuminance reflectance feature map X'. 2_10 Next <X 2_10 ,X' 2_10 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence for feature transformation, resulting in the eleventh intermediate reflectance feature map tuple. <X 2_11 ,X' 2_11 >, where X 2_10 The input is fed into a structure consisting of fully connected layers, ReLU activation functions, and another fully connected layer in sequence to obtain the eleventh intermediate normal illumination reflectance feature map X. 2_11 , will X' 2_10 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence to obtain the eleventh intermediate low-illuminance reflectance feature map X'. 2_11 ,Will <X 2_11 ,X′ 2_11 Activation is performed using the Sigmoid activation function to obtain the twelfth intermediate reflectance feature map binary. <X 2_12 ,X' 2_12 >, where X 2_11 The twelfth intermediate normal illumination reflectance feature map X was obtained by activating the Sigmoid function. 2_12 , will X' 2_11 The twelfth intermediate low-illuminance reflectance feature map X' is obtained by activating the Sigmoid function. 2_12 ,Will <X 2_9 ,X' 2_9 >and <X 2_12 ,X' 2_12 Perform element-wise multiplication to obtain the thirteenth intermediate reflectivity feature map binary. <X 2_13 ,X' 2_13 >, where X 2_9 With X 2_12 Element-wise multiplication yields the thirteenth intermediate normal illumination reflectance characteristic map X. 2_13 , X′ 2_9 With X' 2_12 Element-wise multiplication yields the thirteenth intermediate low-illuminance reflectance feature map X'. 2_13 ,Will <X 2_13 ,X' 2_13 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence for feature transformation, resulting in the fourteenth intermediate reflectance feature map tuple. <X 2_14 ,X' 2_14 >, where X 2_13 The input is fed into a structure consisting of fully connected layers, ReLU activation functions, and another fully connected layer in sequence to obtain the fourteenth intermediate normal illumination reflectance feature map X. 2_14 , will X' 2_13 The input is fed into a structure consisting of fully connected layers, a ReLU activation function, and another fully connected layer in sequence to obtain the fourteenth intermediate low-illuminance reflectance feature map X'. 2_14 ; S338: Yes <X 2_14 ,X' 2_14 After performing the DropPath operation, we obtain the tuple of the fifteenth intermediate reflectance feature map. <X 2_15 ,X' 2_15 >, where X 2_14 Performing the DropPath operation yields the fifteenth intermediate normal illumination reflectance feature map X. 2_15 , will X' 2_14 Performing the DropPath operation yields the fifteenth intermediate low-illuminance reflectance feature map X'. 2_15 ,Will <X 2_15 ,X' 2_15 >and <X 2_8 ,X' 2_8 Perform a Concat operation to obtain the final binary tuple of the second road damage reflectivity feature map.<R2,R'2> X 2_15 With X 2_8 Performing a concat operation yields the second normal illumination road damage reflectivity characteristic map R2, and X' 2_15 With X' 2_8 The concat operation yields the second low-light road damage reflectivity feature map R'2.

7. The method for detecting road damage in low-light substations based on deep learning according to claim 6, characterized in that, The specific steps of S4 are as follows: S41: During training, the model extracts data from the training set according to the set batch size, performs forward calculations, generates prediction results, and then performs backpropagation through the overall loss function L2R-LOSS to optimize the parameters of the substation road damage detection model under low light conditions. S42: After each round of training, the data from the validation set is used to evaluate the model and determine its generalization ability. S43: To avoid overfitting during training, an early stopping mechanism is introduced. If the loss on the validation set does not improve significantly in multiple consecutive training rounds, the training will be terminated early to save computational resources and ensure the stability of the model in practical applications. S44: When the training reaches the predetermined number of rounds and the validation set loss is stable or reaches the ideal level, the training process ends, and an optimized low-light substation road damage detection model is finally obtained. The model can accurately detect substation road damage under low light conditions and maintain good performance in different environments. S45: After training, the model will have appropriate parameter configuration and strong generalization ability, and can be applied to new substation road damage detection tasks.

8. The method for detecting road damage in low-light substations based on deep learning according to claim 7, characterized in that, The loss function L2R-LOSS in S41 is derived from the reflectance learning loss function L r Reorganization Consistency Loss Function L rc And the target detection loss function L target The weighted composition, its functional expression is as follows: L L2R-LOSS =λ g L r +λ f L target +λ q L rc Where, λ g , λ f and λ q To account for the loss weights, the sum of the three is 1; Among them, L r The function expression is as follows: Where R'1 is the first low-light road damage reflectivity feature map; R'2 is the first low-light road damage reflectivity feature map; N is the total number of feature points in the R'1 and R'2 feature maps, R' 1_i and R' 2_i These represent the feature response values ​​corresponding to the i-th position in the feature map; and Let R'1 and R'2 represent the average characteristic response values, respectively. and These are the variances of the characteristic response values ​​of R'1 and R'2, respectively. It is the covariance of the characteristic response values ​​between R'1 and R'2, and C1 and C2 are constants to avoid the denominator being zero; Among them, L target The function expression is as follows: Where α and β represent the weights of the location regression loss and classification loss, respectively; x i and x' i p represents the center points of the ground truth bounding box and the predicted bounding box of the i-th object, respectively; i and p' i Let represent the true class label and predicted class probability of the i-th target, respectively. Smooth(x) represents the error between the predicted and true values, i.e. L rc The function expression is as follows: L rc =||R1-R2||1+||R1-R2||1 Wherein, R1 is the first normal illumination road damage reflectance characteristic map; R'2 is the second low illumination road damage reflectance characteristic map; R'1 is the first low illumination road damage reflectance characteristic map; R2 is the second normal illumination road damage reflectance characteristic map.

Images