A road lamp inspection and distribution anomaly detection method based on a UAV and a night vision enhancement model

Abstract

The application relates to the technical field of intelligent inspection and computer vision, in particular to a road lamp inspection and distribution anomaly detection method based on a UAV and a night vision enhancement model, wherein the UAV flies along a preset track at a low altitude, synchronously collects night road images and self-positioning attitude data, carries out multi-channel vision enhancement processing on the collected night road images, inputs an improved YOLO model to complete lamp target detection, outputs lamp candidate frames and confidence, constructs an input tensor through multi-channel feature fusion, and embeds CBAM in a C2f module of a Backbone part of YOLOv8. The road lamp inspection and distribution anomaly detection method based on the UAV and the night vision enhancement model has high night detection precision, through the synergistic improvement of multi-channel vision enhancement, a CBAM attention mechanism and a brightness weighted loss function, the lamp detection accuracy of the model in a strong interference night scene reaches more than 93%, the false detection rate is reduced to less than 5%, and the false detection problem caused by strong light interference at night is effectively solved.

Description

Technical Field

[0001] This invention relates to the fields of intelligent inspection and computer vision technology, specifically a method for road street light inspection and distribution anomaly detection based on UAVs and night vision enhancement models. Background Technology

[0002] As a core guarantee for nighttime traffic safety, the stability of road streetlights and the rationality of their distribution directly affect the life and property safety of traffic participants. The current road streetlight inspection mainly relies on manual inspection or vehicle-mounted inspection. Inspection personnel need to travel along the road section by section and judge the lighting status of the streetlights by visual observation. This has inherent defects such as low efficiency, high safety risks of nighttime operation, high manpower and time costs, and high rate of missed inspections and false inspections.

[0003] With the development of drone and computer vision technologies, some solutions have attempted to use drones equipped with image acquisition devices, combined with general target detection models, to inspect streetlights and improve inspection efficiency. However, in complex urban scenarios at night, such solutions still have many key technical shortcomings and are difficult to meet actual operation and maintenance needs. For example, the invention patent CN114547863A, "A Streetlight Inspection System and Method Based on Drones," discloses the use of drones to acquire streetlight images and use image recognition technology to determine faults, but it uses a general target detection model and has not optimized for strong light interference at night. The invention patent CN115272879A, "Streetlight Inspection Method and Device Based on Computer Vision," uses traditional image processing algorithms to extract streetlight areas, which cannot effectively handle occlusion problems and does not address the detection of abnormal streetlight distribution.

[0004] Specifically, the core shortcomings of existing technologies can be summarized into the following four points:

[0005] (1) High false detection rate due to strong light interference at night: In urban scenes at night, there are various non-target strong light interferences such as vehicle high beams, shop advertising signs, and road reflective signs. General target detection models cannot effectively distinguish between streetlights and interference light sources, resulting in a high false detection rate.

[0006] (2) Occlusion problem leads to missed detection: Tree branches, utility poles, bridge structures and other structures on both sides of the road can partially or completely obscure streetlights. Existing models have difficulty penetrating the obstruction to extract effective features, resulting in a large number of missed detections.

[0007] (3) Failure to utilize prior information on spatial structure: Streetlights exhibit significant linear and equidistant distribution characteristics along the road direction. Existing solutions mostly adopt a single-point independent detection mode, failing to integrate the prior information of this structure into the detection process, which further reduces the reliability of the detection.

[0008] (4) Unable to determine engineering-level distribution anomalies: Existing technologies can only determine the on / off status of a single street light, and cannot effectively identify engineering-level distribution anomalies that affect the continuity of road lighting, such as missing lights, abnormal spacing, and partial light outages on elevated roads, making it difficult to support refined operation and maintenance decisions.

[0009] In summary, existing street light detection solutions based on target detection models such as YOLO focus primarily on target recognition itself. They lack systematic technical design for adapting to complex nighttime environments, target area constraints, spatial distribution modeling, and anomaly statistical analysis. This results in poor applicability in actual nighttime inspection scenarios and makes it difficult to meet the dual requirements of road maintenance departments for inspection accuracy and efficiency.

[0010] To address the aforementioned issues, we propose an improvement method for road light inspection and distribution anomaly detection based on UAVs and a nighttime visual enhancement model. Summary of the Invention

[0011] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0012] This invention provides a method for road street light inspection and distribution anomaly detection based on UAVs and night vision enhancement models, comprising the following steps:

[0013] S1. The drone flies at low altitude along a preset road trajectory, simultaneously collecting nighttime road images and its own positioning and attitude data;

[0014] S2. Perform multi-channel visual enhancement processing on the collected nighttime road images, input the improved YOLO model to complete street light target detection, output street light candidate boxes and confidence scores, construct the input tensor through multi-channel feature fusion, and enhance the contrast between luminous targets and dark backgrounds;

[0015] In the C2f module of the Backbone part of YOLOv8, CBAM, or Channel-Spatial Joint Attention Mechanism, is embedded to enhance effective feature extraction through the dynamic allocation of channel weights and spatial weights.

[0016] In YOLOv8's original loss functions, such as classification loss Bounding box loss Confidence loss Based on this, a brightness-weighted loss term is introduced. This makes the model more sensitive to the response of real streetlights;

[0017] S3. Based on the transformation of UAV positioning attitude data and image coordinate system, the center points of the street light candidate boxes are spatially grouped, and the street light distribution curves on the left and right sides of the road are fitted respectively to construct the target detection area;

[0018] First, extract the center point coordinates of all street light candidate boxes detected by the improved YOLO model to form a center point set. , where n is the number of candidate boxes. Given the center pixel coordinates of the i-th candidate box, and combining the mapping relationship between the UAV flight trajectory and the image coordinate system, determine the pixel coordinates of the road centerline in the image. Candidate points are grouped into left and right groups based on their position relative to the road centerline.

[0019] RANSAC (Random Sampling Consensus) algorithm is used to fit quadratic polynomial curves to the point sets on both sides to achieve robust fitting and outlier removal.

[0020] Using the curves on both sides obtained by fitting the RANSAC algorithm as the center, a strip-shaped target region is constructed. Subsequent halo extraction and anomaly analysis are performed only within this region. Defined as: ;

[0021] S4. Within the target area obtained by fitting, extract the central feature points of the street light halo by dividing the brightness threshold and calculating the centroid;

[0022] Target area to be constructed Within the system, brightness threshold segmentation is performed for each street light candidate box to extract the actual street light halo area and calculate the halo centroid as the precise center feature point of the street light.

[0023] S5. Construct a street light distribution spacing model, and use statistical feature analysis to determine and locate street light distribution anomalies;

[0024] By combining drone positioning data, the precise center feature points of the streetlights on both the left and right sides can be determined. Transform from image coordinate system to world coordinate system to obtain real geographic coordinates , where lon is longitude and lat is latitude;

[0025] The geographical coordinates of streetlights on the same side are sorted according to the road extension direction, i.e. the inspection direction, to obtain an ordered set of points. The Euclidean distance between two adjacent streetlights is calculated to form a spacing sequence. Based on the statistical characteristics of the spacing sequence, a dual-threshold anomaly judgment rule is constructed to accurately identify distribution anomalies.

[0026] As a preferred embodiment of the present invention, the night vision enhancement input modeling steps are as follows:

[0027] (1) Color space conversion: Let the original input image be... ,in Using pixel coordinates, convert them to the HSV color space to separate the luminance component. ;

[0028] (2) Luminance gradient calculation: The gradient map of the luminance component is calculated based on the Sobel operator to characterize the degree of drastic change in luminance. The formula is:

[0029] ;

[0030] in, These are the partial derivatives of the luminance component in the x and y directions, respectively;

[0031] (3) Multi-channel input construction: The feature tensor of the final input model is defined as a 5-channel tensor: ;

[0032] Where R, G, and B are the red, green, and blue channels of the original image, and V is the HSV spatial luminance component. As a brightness gradient map, this multi-channel input can simultaneously represent color information, brightness information, and brightness change information, enabling the model to accurately capture the core features of the street light emitting area.

[0033] As a preferred embodiment of the present invention, the network structure embedded with CBAM improves the calculation method as follows:

[0034] (1) Channel attention weight calculation: For the input feature map Global average pooling (AvgPool) and global max pooling (MaxPool) are performed separately to obtain two 1×1×C feature vectors, where C is the number of channels and H and W are the feature map sizes. After processing by a shared MLP network, these vectors are summed, and the channel attention weights are obtained by applying a Sigmoid activation function. :

[0035] ;

[0036] (2) Spatial attention weight calculation: The feature maps after channel attention weighting are subjected to global average pooling and global max pooling again to obtain two 1×H×W feature maps. After concatenation, they are convolved with a 7×7 convolution kernel and the spatial attention weights are obtained by using the Sigmoid activation function. : ;

[0037] (3) Attention-weighted fusion: The final output feature map is the product of channel attention weighting and spatial attention weighting. ;

[0038] in, This is element-wise multiplication.

[0039] As a preferred embodiment of the present invention, the overall loss function of the nighttime brightness-weighted loss function is defined as follows: ;

[0040] in, This is the brightness loss weighting coefficient, with a value ranging from 0.3 to 0.5, which can be adaptively adjusted according to the actual scene;

[0041] Brightness loss item Defined as the weighted absolute error between the predicted brightness and the actual brightness within the detection box: ;

[0042] In the formula, This is the predicted brightness value for the i-th pixel within the bounding box. This represents the actual brightness value of the corresponding pixel. The brightness weight is determined by the average brightness within the detection frame. With the image's maximum average brightness The ratio is determined as follows: .

[0043] As a preferred embodiment of the present invention, the streetlight space is divided into left and right groups:

[0044] Left side street light point set: ;

[0045] Right side street light point set: .

[0046] As a preferred embodiment of the present invention, the specific steps of robust curve fitting are as follows:

[0047] (1) Model assumptions: It is assumed that the distribution of streetlights conforms to a quadratic polynomial curve:

[0048] , where a, b, and c are fitting coefficients;

[0049] (2) Random sampling: Randomly select 3 sample points from the point set and solve for the initial values ​​of the coefficients of the quadratic polynomial;

[0050] (3) Residual calculation: Calculate the residuals from all candidate points to the fitted curve, i.e., the Euclidean distances:

[0051] ;

[0052] (4) Interior point determination: Set residual threshold The value is determined based on the street light installation spacing and image resolution, ranging from 2 to 5 pixels. If the point is an internal point (i.e., a real streetlight point), it is determined to be an external point (i.e., an interference point).

[0053] (5) Iterative optimization: Repeat the sampling, fitting, and residual calculation steps, and record the fitting model with the most inliers as the optimal curve;

[0054] (6) Interference removal: Remove all outliers to obtain the precise point set of the streetlights on the left and right sides. and .

[0055] As a preferred technical solution of the present invention, the target area In The fitted curve is shown on the left. The right side is d is the half width of the strip area, which is determined based on the ratio between the installation height of the street light and the flight height of the drone, and the value ranges from 10 to 20 pixels.

[0056] As a preferred embodiment of the present invention, the specific steps for halo center extraction are as follows:

[0057] (1) Adaptive threshold segmentation: Determine the brightness segmentation threshold based on the Otsu method (OTSU). For the brightness components within the candidate box Perform binarization:

[0058] ;

[0059] in, This indicates that the pixel belongs to the streetlight halo area;

[0060] (2) Calculation of the centroid of the halo: Using the binarized image as weight, calculate the centroid coordinates of the halo region. As the precise center feature point of the streetlight:

[0061] .

[0062] As a preferred embodiment of the present invention, an ordered point set is:

[0063] m represents the precise number of streetlights on the same side;

[0064] Spacing sequence :

[0065]

[0066] in, , These are the longitude circle radius and latitude circle radius at the corresponding latitudes, ensuring the geographical accuracy of distance calculations.

[0067] As a preferred technical solution of the present invention, the specific steps of the anomaly determination model are as follows:

[0068] (1) Statistical characteristic calculation: Calculate the mean of the interval sequence. with standard deviation Characterizing the spacing and dispersion of streetlights under normal conditions: , ;

[0069] (2) Anomaly detection rules: Set statistical threshold k and maximum allowable interval. The value of k ranges from 2 to 3, corresponding to a confidence interval of 95.45% to 99.73%. According to road design specifications, the spacing is 1.5 times the standard spacing. If any element in the spacing sequence meets any of the following conditions, the corresponding road segment is considered to have an abnormal distribution:

[0070] Condition 1: If the spacing of a single street light deviates significantly from the normal range, it is determined that a single street light is missing or offset.

[0071] Condition 2: If the occurrence of this phenomenon occurs continuously for m times, with m ≥ 2, and the spacing between multiple consecutive sections exceeds the standard, it is determined that there is a partial street light outage or a large area of ​​missing street lights on the elevated road.

[0072] (3) Abnormal area location: For the spacing that is determined to be abnormal, the starting and ending geographic coordinates of the abnormal road segment are determined by combining the UAV positioning data, and the abnormal type is output, such as single light missing, single light offset, partial light failure, and large area missing, and the length of the abnormal area is output simultaneously.

[0073] The beneficial effects of this invention are: the road street light inspection and distribution anomaly detection method based on UAV and night vision enhancement model has high night detection accuracy. Through the synergistic improvement of multi-channel vision enhancement, CBAM attention mechanism and brightness weighted loss function, the model achieves a street light detection accuracy of over 93% in night scenes with strong interference, and reduces the false detection rate to below 5%, effectively solving the problem of false detection caused by strong light interference at night.

[0074] Strong resistance to occlusion: Robust curve fitting based on RANSAC algorithm can effectively eliminate interference points formed by occlusion, reducing the false detection rate to below 8%, thus solving the false detection defect caused by occlusion.

[0075] Integrating spatial prior information: By making full use of the structural priors of "linear distribution" and "equal spacing distribution" of streetlights, and through regional fitting and statistical modeling, an upgrade from "single point detection" to "overall distribution analysis" has been achieved;

[0076] It can identify engineering-level distribution anomalies: For the first time, it can accurately determine distribution anomalies such as missing streetlights, offset streetlights, and partial streetlight failures, filling the gap in existing technologies that cannot support refined operation and maintenance;

[0077] High inspection efficiency: The drone's autonomous inspection speed is 4~6m / s, which is more than 10 times more efficient than manual inspection, greatly reducing inspection costs and safety risks. Attached Figure Description

[0078] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0079] Figure 1 This invention provides a lamppost area location map for a road street light inspection and distribution anomaly detection method based on UAVs and night vision enhancement models.

[0080] Figure 2 This invention relates to a lamppost area localization mask image for a road street light inspection and distribution anomaly detection method based on UAVs and night vision enhancement models.

[0081] Figure 3 This is a region fitting map of a road street light inspection and distribution anomaly detection method based on UAV and night vision enhancement model according to the present invention;

[0082] Figure 4 This invention relates to a region fitting mask diagram for a method of road street light inspection and distribution anomaly detection based on UAVs and night vision enhancement models.

[0083] Figure 5 This is a halo extraction image of a road street light inspection and distribution anomaly detection method based on UAV and night vision enhancement model according to the present invention;

[0084] Figure 6 This is a halo filtering diagram of a road street light inspection and distribution anomaly detection method based on UAV and night vision enhancement model according to the present invention;

[0085] Figure 7 This invention relates to a method for inspecting and detecting distribution anomalies of road streetlights based on unmanned aerial vehicles and a night vision enhancement model, which includes halo filtering and area display diagrams.

[0086] Figure 8 This invention provides an anomaly detection and filtering diagram for a road street light inspection and distribution anomaly detection method based on UAV and night vision enhancement model.

[0087] Figure 9 This invention relates to an anomaly judgment, filtering, and area display map of a road street light inspection and distribution anomaly detection method based on UAVs and night vision enhancement models;

[0088] Figure 10 This is the final anomaly determination diagram of the road street light inspection and distribution anomaly detection method based on UAV and night vision enhancement model of the present invention; Detailed Implementation

[0089] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0090] A method for road street light inspection and distribution anomaly detection based on UAVs and night vision enhancement models includes the following steps:

[0091] The drone flies at low altitude along a preset trajectory on the road, simultaneously collecting nighttime road images and its own positioning and attitude data;

[0092] Multi-channel visual enhancement processing is performed on the collected nighttime road images. The improved YOLO model is input to complete the street light target detection, and the street light candidate boxes and confidence scores are output. The input tensor is constructed by multi-channel feature fusion to enhance the contrast between the luminous target and the dark background.

[0093] In the C2f module of the Backbone part of YOLOv8, CBAM, or Channel-Spatial Joint Attention Mechanism, is embedded to enhance effective feature extraction through the dynamic allocation of channel weights and spatial weights.

[0094] In YOLOv8's original loss functions, such as classification loss Bounding box loss Confidence loss Based on this, a brightness-weighted loss term is introduced. This makes the model more sensitive to the response of real streetlights;

[0095] Based on the transformation between UAV positioning attitude data and image coordinate system, the center points of street light candidate boxes are spatially grouped, and the street light distribution curves on the left and right sides of the road are fitted respectively to construct the target detection area.

[0096] First, extract the center point coordinates of all street light candidate boxes detected by the improved YOLO model to form a center point set. , where n is the number of candidate boxes. Given the center pixel coordinates of the i-th candidate box, and combining the mapping relationship between the UAV flight trajectory and the image coordinate system, determine the pixel coordinates of the road centerline in the image. Candidate points are grouped into left and right groups based on their position relative to the road centerline.

[0097] RANSAC (Random Sampling Consensus) algorithm is used to fit quadratic polynomial curves to the point sets on both sides to achieve robust fitting and outlier removal.

[0098] Using the curves on both sides obtained by fitting the RANSAC algorithm as the center, a strip-shaped target region is constructed. Subsequent halo extraction and anomaly analysis are performed only within this region. Defined as: ;

[0099] Within the target area obtained by fitting, the central feature points of the street light halo are extracted by brightness threshold segmentation and centroid calculation;

[0100] Target area to be constructed Within the system, brightness threshold segmentation is performed for each street light candidate box to extract the actual street light halo area and calculate the halo centroid as the precise center feature point of the street light.

[0101] A street light distribution spacing model is constructed, and statistical feature analysis is used to determine and locate street light distribution anomalies.

[0102] By combining drone positioning data, the precise center feature points of the streetlights on both the left and right sides can be determined. Transform from image coordinate system to world coordinate system to obtain real geographic coordinates , where lon is longitude and lat is latitude;

[0103] The geographical coordinates of streetlights on the same side are sorted according to the road extension direction, i.e. the inspection direction, to obtain an ordered set of points. The Euclidean distance between two adjacent streetlights is calculated to form a spacing sequence. Based on the statistical characteristics of the spacing sequence, a dual-threshold anomaly judgment rule is constructed to accurately identify distribution anomalies.

[0104] The steps for nighttime visual enhancement input modeling are as follows:

[0105] (1) Color space conversion: Let the original input image be... ,in Using pixel coordinates, convert them to the HSV color space to separate the luminance component. ;

[0106] (2) Luminance gradient calculation: The gradient map of the luminance component is calculated based on the Sobel operator to characterize the degree of drastic change in luminance. The formula is:

[0107] ;

[0108] in, These are the partial derivatives of the luminance component in the x and y directions, respectively;

[0109] (3) Multi-channel input construction: The feature tensor of the final input model is defined as a 5-channel tensor: ;

[0110] Where R, G, and B are the red, green, and blue channels of the original image, and V is the HSV spatial luminance component. As a brightness gradient map, this multi-channel input can simultaneously represent color information, brightness information, and brightness change information, enabling the model to accurately capture the core features of the street light emitting area.

[0111] The improved computation method for the network structure embedded with CBAM is as follows:

[0112] (1) Channel attention weight calculation: For the input feature map Global average pooling (AvgPool) and global max pooling (MaxPool) are performed separately to obtain two 1×1×C feature vectors, where C is the number of channels and H and W are the feature map sizes. After processing by a shared MLP network, these vectors are summed, and the channel attention weights are obtained by applying a Sigmoid activation function. :

[0113] ;

[0114] (2) Spatial attention weight calculation: The feature maps after channel attention weighting are subjected to global average pooling and global max pooling again to obtain two 1×H×W feature maps. After concatenation, they are convolved with a 7×7 convolution kernel and the spatial attention weights are obtained by using the Sigmoid activation function. : ;

[0115] (3) Attention-weighted fusion: The final output feature map is the product of channel attention weighting and spatial attention weighting. ;

[0116] in, This is element-wise multiplication.

[0117] The overall loss function of the nighttime brightness-weighted loss function is defined as follows: ;

[0118] in, This is the brightness loss weighting coefficient, with a value ranging from 0.3 to 0.5, which can be adaptively adjusted according to the actual scene;

[0119] Brightness loss item Defined as the weighted absolute error between the predicted brightness and the actual brightness within the detection box: ;

[0120] In the formula, This is the predicted brightness value for the i-th pixel within the bounding box. This represents the actual brightness value of the corresponding pixel. The brightness weight is determined by the average brightness within the detection frame. With the image's maximum average brightness The ratio is determined as follows: .

[0121] Streetlight space is grouped left and right:

[0122] Left side street light point set: ;

[0123] Right side street light point set: .

[0124] The specific steps for robust curve fitting are as follows:

[0125] (1) Model assumptions: It is assumed that the distribution of streetlights conforms to a quadratic polynomial curve:

[0126] , where a, b, and c are fitting coefficients;

[0127] (2) Random sampling: Randomly select 3 sample points from the point set and solve for the initial values ​​of the coefficients of the quadratic polynomial;

[0128] (3) Residual calculation: Calculate the residuals from all candidate points to the fitted curve, i.e., the Euclidean distances:

[0129] ;

[0130] (4) Interior point determination: Set residual threshold The value is determined based on the street light installation spacing and image resolution, ranging from 2 to 5 pixels. If the point is an internal point (i.e., a real streetlight point), it is determined to be an external point (i.e., an interference point).

[0131] (5) Iterative optimization: Repeat the sampling, fitting, and residual calculation steps, and record the fitting model with the most inliers as the optimal curve;

[0132] (6) Interference removal: Remove all outliers to obtain the precise point set of the streetlights on the left and right sides. and .

[0133] target area In The fitted curve is shown on the left. The right side is d is the half width of the strip area, which is determined based on the ratio between the installation height of the street light and the flight height of the drone, and the value ranges from 10 to 20 pixels.

[0134] Specific steps for halo center extraction:

[0135] (1) Adaptive threshold segmentation: Determine the brightness segmentation threshold based on the Otsu method (OTSU). For the brightness components within the candidate box Perform binarization:

[0136] ;

[0137] in, This indicates that the pixel belongs to the streetlight halo area;

[0138] (2) Calculation of the centroid of the halo: Using the binarized image as weight, calculate the centroid coordinates of the halo region. As the precise center feature point of the streetlight:

[0139] .

[0140] Ordered point set:

[0141] m represents the precise number of streetlights on the same side;

[0142] Spacing sequence :

[0143]

[0144] in, , These are the longitude circle radius and latitude circle radius at the corresponding latitudes, ensuring the geographical accuracy of distance calculations.

[0145] The specific steps of the anomaly detection model are as follows:

[0146] (1) Statistical characteristic calculation: Calculate the mean of the interval sequence. with standard deviation Characterizing the spacing and dispersion of streetlights under normal conditions: , ;

[0147] (2) Anomaly detection rules: Set statistical threshold k and maximum allowable interval. The value of k ranges from 2 to 3, corresponding to a confidence interval of 95.45% to 99.73%. According to road design specifications, the spacing is 1.5 times the standard spacing. If any element in the spacing sequence meets any of the following conditions, the corresponding road segment is considered to have an abnormal distribution:

[0148] Condition 1: If the spacing of a single street light deviates significantly from the normal range, it is determined that a single street light is missing or offset.

[0149] Condition 2: If the occurrence of this phenomenon occurs continuously for m times, with m ≥ 2, and the spacing between multiple consecutive sections exceeds the standard, it is determined that there is a partial street light outage or a large area of ​​missing street lights on the elevated road.

[0150] (3) Abnormal area location: For the spacing that is determined to be abnormal, the starting and ending geographic coordinates of the abnormal road segment are determined by combining the UAV positioning data, and the abnormal type is output, such as single light missing, single light offset, partial light failure, and large area missing, and the length of the abnormal area is output simultaneously.

[0151] Example: Figures 1-6 As shown,

[0152] 1. Overview of Implementation Examples

[0153] This embodiment uses the nighttime street light inspection of urban elevated roads as an application scenario to explain in detail the specific implementation process, equipment selection, parameter settings and implementation effects of the present invention, and to verify the feasibility and superiority of the present invention.

[0154] 2. Equipment and Material Selection

[0155] 2.1 Unmanned Aerial Vehicle Platform

[0156] It adopts the DJI IDock3 fully automated take-off and landing airport, paired with the Matrice 4TD flight platform; the 4TD has a maximum level flight speed of 10m / s, an empty flight endurance of 42 minutes, and an IP55 protection rating, and can operate continuously in environments ranging from -20℃ to 50℃; the airport has a built-in omnidirectional millimeter-wave radar and visual obstacle avoidance system, and supports infrared obstacle avoidance at night to ensure the safety of low-altitude flight over elevated roads.

[0157] 2.2 Image Acquisition Module

[0158] 4TD natively integrated three-sensor gimbal camera:

[0159] Visible light: 4K 1 / 1.3″ CMOS, 48 million effective pixels, minimum illumination of 0.0007 lux, supports starlight-level night shooting;

[0160] Infrared: 640×512 vanadium oxide uncooled detector, frame rate 30Hz, temperature measurement accuracy ±2℃;

[0161] Laser ranging: range 0.5~1200m, accuracy ±0.5m;

[0162] The three-axis mechanical gimbal has a stabilization accuracy of ±0.01°, which can accurately capture the halo effect of streetlights and the absence of light strips at night.

[0163] 2.3 Positioning Module: Airport 3 has a built-in RTK base station. The 4TD supports GPS (L1+L5), Beidou (B1+B2), and Galileo (E1+E5) systems with ten frequency points. The RTK horizontal accuracy is 1cm+1ppm, the vertical accuracy is 1.5cm+1ppm, and the IMU outputs at 200Hz, meeting the stable positioning requirements in weak GNSS environments under urban overpasses.

[0164] 2.4 Edge Computing Module

[0165] NVIDIA Jetson OrinNX core board (8GB LPDDR5, 100 TOPS INT8 computing power, 15W power consumption); directly connected to Airport 3 via network cable, the original 4K image stream is transcoded and then input into Jetson, halo recognition + light strip segmentation + template matching end-to-end inference latency ≤100ms, realizing "on-the-fly judgment" real-time missing detection.

[0166] 3. Implementation steps

[0167] 3.1 Inspection Preparation Phase

[0168] (1) Obtaining information on the inspection section: On-site surveys determined that the standard installation spacing of streetlights is 30m, and the maximum allowable spacing is... The patrol section is 10km long;

[0169] (2) Plan the flight trajectory: Set the UAV to fly along the center line of the elevated road through the ground control module, with the flight altitude consistent with the height of the light pole, the flight speed of 10m / s, the image acquisition frame rate of 30fps, and the shooting interval of 0.1s;

[0170] (3) Algorithm parameter settings: Improve the brightness loss weight of the YOLO model CBAM attention mechanism has 256 channels; residual threshold for RANSAC fitting. The statistical threshold for determining pixel distribution anomalies is k=2.5, and the number of consecutive anomalies is m=2.

[0171] 3.2 Unmanned Aerial Vehicle (UAV) Inspection Phase

[0172] (1) The UAV takes off according to the preset trajectory. The laser radar obstacle avoidance module detects obstacles (bridge pillars, tree branches, etc.) in front in real time. When the distance of the detected obstacle is less than 2m, the flight altitude (±0.5m) or heading is automatically adjusted to avoid the obstacle.

[0173] (2) The image acquisition module enables HDR mode and low light enhancement function, and simultaneously acquires visible light and infrared images, which are transmitted to the edge computing platform in real time; the positioning module outputs the longitude, latitude, altitude and flight attitude data of the UAV in real time, and synchronizes them to the data processing module.

[0174] 3.3 Data Processing Stage

[0175] (1) Nighttime street light detection: The edge computing platform performs multi-channel enhancement on the acquired RGB images and generates a 5-channel input tensor. Input an improved YOLOv8 model, output street light candidate boxes and their confidence scores (with a confidence threshold of 0.7 to filter low-confidence candidate boxes), such as... Figure 1 , Figure 2 As shown;

[0176] (2) Region fitting and interference removal: Extract the coordinates of the center point of the candidate box, and based on the road centerline Divided into left and right groups; the RANSAC algorithm was used to fit quadratic polynomial curves to each group, and interference points with residuals greater than 3 pixels (such as shop advertising lights and vehicle lights) were removed to obtain accurate street light point sets for the left and right sides; a strip-shaped target region with a half-width d=15 pixels was constructed, such as... Figure 3 , Figure 4 As shown;

[0177] (3) Halo center extraction: Within the target area, the brightness threshold is determined using the OTSU algorithm. (0~255 grayscale range), perform binarization segmentation, calculate the centroid of the halo for each street light, convert it to world coordinates, and sort it according to the inspection direction, such as... Figure 5 As shown;

[0178] (4) Distribution anomaly determination: Calculate the spacing sequence of adjacent streetlights and obtain the mean value. Standard deviation Three anomalies were detected: ① Spacing between the 12th and 13th streetlights ,satisfy ① It was determined to be a single missing lamp; ② The spacing between lamps 56-57 and 57-58 was 48m and 52m respectively, both greater than... And if it occurs twice in a row, it is determined to be a partial lamp failure; ③ The spacing between the 102nd and 103rd streetlights It is determined to be a single lamp misalignment, such as Figures 6-9 As shown.

[0179] 3.4 Result Output Stage

[0180] like Figure 10 As shown, the data processing module transmits the geographic coordinates, anomaly type, and length of the abnormal area of ​​the abnormal road segment to the ground control module in real time, and generates an inspection report, which includes image screenshots of the abnormal location, location information, and maintenance suggestions (such as single light replacement, patch installation, and position correction).

[0181] 4. Implementation Results

[0182] The inspection task of 10km elevated road in this embodiment took approximately 42 minutes, which is 11.4 times more efficient than manual inspection (approximately 8 hours). On-site verification showed that the results of the three anomaly detections were accurate, with no misjudgments or omissions. The accuracy rate for street light detection was 93.6%, and the accuracy rate for anomaly section identification was 91.2%, significantly better than existing technologies (traditional YOLOv8 has an accuracy rate of 78.3% and cannot identify distribution anomalies). The implementation results demonstrate that this invention can effectively meet the actual needs of nighttime street light inspection on elevated roads, providing reliable technical support for refined operation and maintenance.

[0183] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for road street light inspection and distribution anomaly detection based on UAVs and night vision enhancement models, characterized in that, Includes the following steps: S1. The drone flies at low altitude along a preset road trajectory, simultaneously collecting nighttime road images and its own positioning and attitude data; S2. Perform multi-channel visual enhancement processing on the collected nighttime road images, input the improved YOLO model to complete street light target detection, output street light candidate boxes and confidence scores, construct the input tensor through multi-channel feature fusion, and enhance the contrast between luminous targets and dark backgrounds; In the C2f module of the Backbone part of YOLOv8, CBAM, or Channel-Spatial Joint Attention Mechanism, is embedded to enhance effective feature extraction through the dynamic allocation of channel weights and spatial weights. In YOLOv8's original loss functions, such as classification loss Bounding box loss Confidence loss Based on this, a brightness-weighted loss term is introduced. This makes the model more sensitive to the response of real streetlights; S3. Based on the transformation of UAV positioning attitude data and image coordinate system, the center points of the street light candidate boxes are spatially grouped, and the street light distribution curves on the left and right sides of the road are fitted respectively to construct the target detection area; First, extract the coordinates of the center points of all candidate street light boxes detected by the improved YOLO model to form a center point set. , where n is the number of candidate boxes. Given the center pixel coordinates of the i-th candidate box, and combining the mapping relationship between the UAV flight trajectory and the image coordinate system, determine the pixel coordinates of the road centerline in the image. Candidate points are grouped into left and right groups based on their position relative to the road centerline; RANSAC (Random Sampling Consensus) algorithm is used to fit quadratic polynomial curves to the point sets on both sides to achieve robust fitting and outlier removal. Using the curves on both sides obtained by fitting the RANSAC algorithm as the center, a strip-shaped target region is constructed. Subsequent halo extraction and anomaly analysis are performed only within this region. Defined as: ; S4. Within the target area obtained by fitting, extract the central feature points of the street light halo by dividing the brightness threshold and calculating the centroid; Target area to be constructed Within the system, brightness threshold segmentation is performed for each street light candidate box to extract the actual street light halo area and calculate the halo centroid as the precise center feature point of the street light. S5. Construct a street light distribution spacing model, and use statistical feature analysis to determine and locate street light distribution anomalies; By combining drone positioning data, the precise center feature points of the streetlights on both the left and right sides can be determined. Transform from image coordinate system to world coordinate system to obtain real geographic coordinates , where lon is longitude and lat is latitude; The geographical coordinates of streetlights on the same side are sorted according to the road extension direction, i.e. the inspection direction, to obtain an ordered set of points. The Euclidean distance between two adjacent streetlights is calculated to form a spacing sequence. Based on the statistical characteristics of the spacing sequence, a dual-threshold anomaly judgment rule is constructed to accurately identify distribution anomalies.

2. The method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, The steps for nighttime visual enhancement input modeling are as follows: (1) Color space conversion: Let the original input image be... ,in Using pixel coordinates, convert them to the HSV color space to separate the luminance component. ; (2) Luminance gradient calculation: The gradient map of the luminance component is calculated based on the Sobel operator to characterize the degree of drastic change in luminance. The formula is: ； in, These are the partial derivatives of the luminance component in the x and y directions, respectively; (3) Multi-channel input construction: The feature tensor of the final input model is defined as a 5-channel tensor: ; Where R, G, and B are the red, green, and blue channels of the original image, and V is the HSV spatial luminance component. As a brightness gradient map, this multi-channel input can simultaneously represent color information, brightness information, and brightness change information, enabling the model to accurately capture the core features of the street light emitting area.

3. The method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, The improved computation method for the network structure embedded with CBAM is as follows: (1) Channel attention weight calculation: For the input feature map Global average pooling (AvgPool) and global max pooling (MaxPool) are performed separately to obtain two 1×1×C feature vectors, where C is the number of channels and H and W are the feature map sizes. After processing by a shared MLP network, these vectors are summed, and the channel attention weights are obtained by applying a Sigmoid activation function. : ； (2) Spatial attention weight calculation: The feature maps after channel attention weighting are subjected to global average pooling and global max pooling again to obtain two 1×H×W feature maps. After concatenation, they are convolved with a 7×7 convolution kernel and the spatial attention weights are obtained by using the Sigmoid activation function. : ； (3) Attention-weighted fusion: The final output feature map is the product of channel attention weighting and spatial attention weighting. ; in, This is element-wise multiplication.

4. The method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, The overall loss function of the nighttime brightness-weighted loss function is defined as follows: ; in, This is the brightness loss weighting coefficient, with a value ranging from 0.3 to 0.5, which can be adaptively adjusted according to the actual scene; Brightness loss item Defined as the weighted absolute error between the predicted brightness and the actual brightness within the detection box: ; In the formula, This is the predicted brightness value for the i-th pixel within the bounding box. This represents the actual brightness value of the corresponding pixel. The brightness weight is determined by the average brightness within the detection frame. With the image's maximum average brightness The ratio is determined as follows: .

5. The method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, Streetlight space is grouped left and right: Left side street light point set: ; Right side street light point set: .

6. The method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, The specific steps for robust curve fitting are as follows: (1) Model assumptions: It is assumed that the distribution of streetlights conforms to a quadratic polynomial curve: , where a, b, and c are fitting coefficients; (2) Random sampling: Randomly select 3 sample points from the point set and solve for the initial values ​​of the coefficients of the quadratic polynomial; (3) Residual calculation: Calculate the residuals from all candidate points to the fitted curve, i.e., the Euclidean distances: ； (4) Interior point determination: Set residual threshold The value is determined based on the street light installation spacing and image resolution, ranging from 2 to 5 pixels. If the point is an internal point (i.e., a real streetlight point), it is determined to be an external point (i.e., an interference point). (5) Iterative optimization: Repeat the sampling, fitting, and residual calculation steps, and record the fitting model with the most inliers as the optimal curve; (6) Interference removal: Remove all outliers to obtain the precise point set of the streetlights on the left and right sides. and .

7. The method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, target area In The fitted curve is shown on the left. The right side is d is the half-width of the strip area, which is determined based on the ratio between the installation height of the street light and the flight height of the drone, and the value ranges from 10 to 20 pixels.

8. The method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, Specific steps for halo center extraction: (1) Adaptive threshold segmentation: The brightness segmentation threshold is determined based on the Otsu method (OTSU). For the brightness components within the candidate box Perform binarization: ； in, This indicates that the pixel belongs to the streetlight halo area; (2) Calculation of the centroid of the halo: Using the binarized image as weight, calculate the centroid coordinates of the halo region. As the precise center feature point of the streetlight: 。 9. A method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, orderly... Dot set: m represents the precise number of streetlights on the same side; Spacing sequence : in, , These are the longitude circle radius and latitude circle radius at the corresponding latitudes, ensuring the geographical accuracy of distance calculations.

10. The method for road street light inspection and distribution anomaly detection based on UAV and night vision enhancement model according to claim 1, characterized in that, The specific steps of the anomaly detection model are as follows: (1) Statistical characteristic calculation: Calculate the mean of the interval sequence. with standard deviation Characterizing the spacing and dispersion of streetlights under normal conditions: , ; (2) Anomaly detection rules: Set statistical threshold k and maximum allowable interval. The value of k ranges from 2 to 3, corresponding to a confidence interval of 95.45% to 99.73%. According to road design specifications, the spacing is 1.5 times the standard spacing. If any element in the spacing sequence meets any of the following conditions, the corresponding road segment is considered to have an abnormal distribution: Condition 1: If the spacing of a single street light deviates significantly from the normal range, it is determined that a single street light is missing or offset. Condition 2: If the occurrence occurs continuously m times, m≥2, and the spacing of multiple consecutive sections exceeds the standard, it is judged that there is a partial interruption of streetlights or a large area of ​​missing streetlights on the elevated road. (3) Abnormal area location: For the spacing that is determined to be abnormal, the starting and ending geographic coordinates of the abnormal road segment are determined by combining the UAV positioning data, and the abnormal type is output, such as single light missing, single light offset, partial light failure, and large area missing, and the length of the abnormal area is output simultaneously.

Images