Street lamp intelligent inspection management system based on deep learning

Abstract

The invention relates to the technical field of intelligent city management and computer vision, and discloses an intelligent street lamp inspection management system based on deep learning. The system comprises an image recognition module, an algorithm processing module, an environment module, a data analysis module, an authority management module, an intelligent path planning module, a video tracking module, a multi-scale detection module, an image conversion module and a detection storage module. And the image recognition module is used for carrying out image data acquisition on the street lamps and realizing image recognition, extracting features of the street lamps by using a convolutional neural network, and carrying out target detection by using a Yolov4 algorithm model. Through automatic image recognition and detection functions, the system can replace manual visual inspection and monitor the state of the street lamp in real time, the time required for inspection is greatly shortened by using a deep learning target detection algorithm and an efficient intelligent path planning technology, the system can complete inspection of a large number of street lamps in a short time, and the inspection efficiency is improved. And the inspection efficiency is obviously improved.

Description

Technical Field

[0001] The present invention relates to the fields of smart city management and computer vision technology, and specifically to a street lamp intelligent inspection and management system based on deep learning. Background Art

[0002] With the acceleration of urbanization, the number and scale of urban lighting facilities and street lights have increased rapidly. Traditional street light inspection and maintenance mainly rely on manual visual inspection, which has disadvantages such as low efficiency, labor consumption, and inability to detect faults in time. There are many street lights in cities, they are widely distributed, and the working environment is changeable. Manual inspection methods cannot effectively meet the detection needs of sudden faults.

[0003] Traditional manual inspections rely on inspectors to check each road section and each lamp one by one. A complete inspection cycle usually takes several days. This method is inefficient and prone to omissions. Since street lamp failures often occur accidentally, manual visual inspections are difficult to detect problems in a timely manner, resulting in street lamps remaining unrepaired for a long time. This is especially true at night or in bad weather, further reducing the efficiency of manual inspections.

[0004] Manual inspections are time-consuming and labor-intensive, and require the collaboration of multiple staff members, resulting in high overall maintenance costs. Inaccurate records also increase subsequent processing costs.

[0005] Traditional inspection methods make it difficult to effectively collect and analyze inspection data. They lack a holistic understanding of street light usage and areas with frequent failures, and are unable to provide a decision-making basis for subsequent management. The lack of data support makes it impossible to scientifically plan resource allocation, inspection route optimization, and other aspects.

[0006] Therefore, those skilled in the art provide a street lamp intelligent inspection and management system based on deep learning to solve the problems raised in the above background technology. Summary of the Invention

[0007] In view of the shortcomings of the existing technology, the present invention provides a street lamp intelligent inspection and management system based on deep learning to solve the problems raised in the above background technology.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: a street lamp intelligent inspection and management system based on deep learning, including an image recognition module, an algorithm processing module, an environment module, a data analysis module, a permission management module, an intelligent path planning module, a video tracking module, a multi-scale detection module, an image conversion module, and a detection storage module;

[0009] The image recognition module is used to collect image data of street lamps and realize image recognition, extract the features of street lamps using convolutional neural networks, and perform target detection using the Yolov4 algorithm model;

[0010] The algorithm processing module is used to process the detected image data;

[0011] The environment module includes defogging, denoising, deblurring and night enhancement processing units, and uses CutOut and GAN technology to achieve data enhancement under different weather conditions;

[0012] The data analysis module combines historical trajectory and video playback functions to provide route planning, maintenance record management, statistics and analysis;

[0013] The authority management module is used to set the access rights of inspection personnel and supports the addition, deletion, modification and query management of data permissions;

[0014] The intelligent path planning module performs path planning based on work order data and task requirements, and automatically generates inspection and maintenance routes;

[0015] The video tracking module is used to track and identify streetlight targets in video surveillance and optimize video detection performance;

[0016] The multi-scale detection module uses PANet and FPN structures to improve the detection accuracy of street lamps at different scales;

[0017] The image conversion module converts the RGB image into HSV or YUV format;

[0018] The detection storage module records the detection results of the street lamps, the number of faulty lamps and the time points in the system database, and provides a detection report in image and text form.

[0019] Preferably, in the image recognition module, a convolutional neural network combined with a Yolov4 algorithm model is used to detect street lamp targets. The core processing of the algorithm includes feature extraction, detection box regression and classification. The following is an analysis of the algorithm formula in Yolov4:

[0020] The Yolov4 uses convolution operations to extract features from images. The process performs convolution operations on different positions of the image through convolution kernels. The output of each convolution layer is calculated as follows:

[0021] Y i，j，k =σ(∑ m，n X i+m，j+n W m，n，k +b k ), where Y i，j，kis the value of the convolution result at the i-th, j-th position of the k-th channel, X i+m，j+n is the pixel of the input image at the corresponding position, W m，n，k is the weight of the convolution kernel, b k is the bias term, σ is the activation function;

[0022] The Yolov4 predicts multiple detection boxes on the feature map, calculates their boundary positions and confidence levels, and the regression formula for position prediction is as follows: x =σ(t x )+c x , b y =σ(t y )+c y , Among them, (b x , b y ) is the center coordinate of the detection frame, t x and t y is the offset predicted by the network, (c x , c y ) is the coordinate of the upper left corner of the anchor box, (p w , p h ) is the width and height of the anchor box, t w and t h is the scaling factor for network prediction;

[0023] Yolov4 uses the Softmax or Sigmoid activation function to calculate the probability of the target category, and uses the Softmax function to normalize the probability prediction of the target category:

[0024] Among them, P(y=k|x) is the predicted probability of category k, z k is the activation value of category k, where k is the total number of categories;

[0025] Yolov4 uses an improved IoU loss function to measure the overlap between the detection box and the true box. The IoU loss function formula is:

[0026] Among them, IoU represents the IoU value between the detection box and the real box, c is the minimum diagonal length of the closure box surrounding the detection box and the real box, and α and v are weight parameters.

[0027] Preferably, the algorithm processing module includes calculation formulas for detection rate and false alarm rate;

[0028] The detection rate refers to the proportion of street lights that the system correctly identifies, and is used to measure the accuracy of the system. The calculation formula is:

[0029] Where n1 represents the actual number of street lights successfully detected by the system, and M is the total number of street lights;

[0030] The false alarm rate refers to the ratio of non-street lights that the system mistakenly identifies as street lights, reflecting the system's misdetection situation. The calculation formula is:

[0031] Where n2 represents the number of non-street lights that are mistakenly identified as street lights, and N represents the total number of targets detected by the system.

[0032] Preferably, in the algorithm processing module, the unbalanced data is processed by a weighted loss function, and the loss function is calculated as:

[0033] Among them, C is the total number of categories, w i is the weight of the category, y i is the true label, is the predicted value;

[0034] In the algorithm processing module, if accurate detection of the target at different positions is necessary, position accuracy calculation is introduced to measure the deviation of the detection frame:

[0035] in, is the predicted position, p is the actual position, d max is the maximum distance in the image.

[0036] Preferably, in the environment module, data enhancement is achieved through CutOut and generative adversarial network technology. The specific algorithm formula is as follows:

[0037] Assume that the input image is X∈R H×W×C , where H and W are the height and width of the image, C is the number of channels, and the center position of the occlusion is arbitrarily determined (c x ,c y ) and the side length l of the occluded area, then the occluded area is:

[0038]

[0039] Generative adversarial networks are used to enhance data under weather conditions. Generative adversarial networks consist of a generator G and a discriminator D, which are used to generate and discriminate synthetic images. The data enhancement process formula of the generative adversarial network is as follows:

[0040] Generator G: The generator inputs arbitrary noise z into the neural network to generate synthetic images Among them, z~p z , usually from a normal or uniform distribution;

[0041] Discriminator D: The discriminator D receives the input image and the output value represents the authenticity of the image. The loss of the discriminator is:

[0042]

[0043] Among them, p data is the distribution of real data;

[0044] Generator loss: The goal of the generator is to deceive the discriminator so that it cannot distinguish between real images and generated images. The generator loss is:

[0045] Optimization goal: Minimize the loss of the generator and discriminator through training iterations, while generating realistic augmented data:

[0046]

[0047] Preferably, in the data analysis module, data is used for path planning, fault statistics and analysis. The following is the algorithm formula:

[0048] The intelligent path planning is based on Euclidean distance or other path optimization algorithms to determine the best route in inspection or maintenance tasks. The core formula is as follows:

[0049] If the street light point position is (x i -y i ) and (y i -y j ), the distance calculation formula is:

[0050]

[0051] Path optimization: Assume that the inspection path is (P1, P2, ..., P n ), the total path length is:

[0052]

[0053] Combined with historical trajectories and fault records, the failure rate of street lamps is calculated and predicted. Assuming that the total number of monitored failures in the time period [T1, T2] is f, and the total number of monitored street lamps is N, the failure rate is calculated as follows: Where FR is the failure rate percentage;

[0054] Moving average method: Fault records of n days (f1, f2, ..., f n ), predict the failure value on day n+1 for:

[0055] Exponential smoothing method: For the smoothing coefficient α (0<α<1), the forecast value on the t+1th day for:

[0056] in, is the predicted value at the previous moment, f t is the actual value at the moment.

[0057] Preferably, in the intelligent path planning module, the path planning algorithm includes Dijkstra algorithm, A algorithm and dynamic planning;

[0058] The Dijkstra algorithm is used to find the shortest path from a starting point to an end point in a weighted graph. For a graph G = (V, E), where V is a set of vertices, E is a set of edges, and each edge (u, v) has a weight w(u, v);

[0059] The shortest path update formula is: d(v) = min(d(v), d(u) + w(u, v)).

[0060] Among them, d(v) is the shortest distance from the starting point to vertex v, d(u) is the distance from the starting point to vertex u, and w(u,v) is the weight of the edge from u to v;

[0061] The A algorithm uses a heuristic evaluation function h(v) to accelerate the path search process. The total evaluation function is: f(v) = g(v) + h(v).

[0062] Among them, f(v) is the total evaluation value of node v, g(v) is the actual cost from the starting point to node v, and h(v) is the heuristic estimate from node v to the target node.

[0063] Preferably, the dynamic programming formula adopts the dynamic programming method of the traveling salesman problem if the inspection task needs to traverse multiple nodes and return to the starting point. The minimum cost of the trip is:

[0064]

[0065] Where C(S,v) is the minimum path length that traverses the nodes in S and ends at v, and d(u,v) is the distance between node u and node v.

[0066] Preferably, in the multi-scale detection module, a specific neural network structure and a multi-scale training method are used to improve the detection accuracy of targets of different sizes. The following is the algorithm formula:

[0067] The feature pyramid network enhances the network's ability to detect objects of different scales by constructing a bottom-up feature pyramid. The feature graph pyramid is generated using the following formula:

[0068] Pl =W l *C l +upsample(P l+1 ), where P l is the output feature map at scale l, W l is the learnable weight, C l is the original feature map, upsample(P l+1 ) represents the upsampling operation on the feature map of the previous layer;

[0069] The path aggregation network further improves the information transmission between different layers through top-down and bottom-up path aggregation based on the feature pyramid network. The specific calculation formula is:

[0070] P l ′=concat(P l ,downsample(P l-1 ′)), where P l ′ is the improved feature map, downsample(P l-1 ′) is the down sampling of the lower layer feature map, and concat represents the splicing operation of the feature map;

[0071] The multi-scale training method dynamically adjusts the input image size so that the network can learn information of different scales during training. Assume that the input image is X∈R H×W×C , randomly adjust the input size (H′, W′) with different scales during training:

[0072] (H′, W′) = α × (H, W), where α is the scaling factor;

[0073] Deformable convolution adds an offset term to the standard convolution kernel to adapt to the deformation of the target at different scales. Given the standard convolution kernel position (x i ,y i ), the offset is (Δx i ,Δy i ), then the output calculation formula of deformable convolution is:

[0074] y(p0)=∑ i w(p i )·x(p0+p i +Δp i ), where p0 is the convolution center point, w(p i ) is the convolution kernel weight, x(p0+p i +Δp i ) is the value of the input feature map at the offset position.

[0075] Preferably, in the image conversion module, the influence of the color temperature of street lamps of different materials is avoided, and the original RGB data is converted to HSV or YUV. The algorithm formula is:

[0076] The algorithm formula for converting RGB to HSV is:

[0077]

[0078]

[0079] The algorithm formula for converting RGB to YUV is:

[0080]

[0081] The present invention provides a street lamp intelligent inspection and management system based on deep learning. It has the following beneficial effects:

[0082] 1. Through automated image recognition and detection functions, the present invention's system can replace manual visual inspection and monitor the status of street lamps in real time. It also uses deep learning target detection algorithms and efficient intelligent path planning technology to greatly reduce the time required for inspections, enabling the system to complete inspections of a large number of street lamps in a relatively short period of time, significantly improving inspection efficiency.

[0083] 2. The system of the present invention can automatically identify the lighting status of street lamps and immediately generate alarm information when a fault is found, realizing rapid response. Compared with manual inspection, the intelligent system can work around the clock without being restricted by time and weather, ensuring that street lamp faults can be detected in the shortest time possible and the information is fed back to maintenance personnel in a timely manner to arrange repairs, thereby improving fault response speed and reducing the dwell time of faulty street lamps.

[0084] 3. The present invention effectively reduces the frequency of manual participation and reduces dependence on inspection personnel through an automated inspection system. At the same time, the system's integrated work order management and data analysis functions can automatically generate maintenance tasks and intelligently plan the optimal path, reducing unnecessary manual intervention and excess travel costs, thereby significantly reducing the overall cost of inspection and maintenance and optimizing the allocation of human resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 is a system diagram of the present invention;

[0086] Figure 2 Schematic diagram of HSV or YUV conversion of original RGB data in the present invention. DETAILED DESCRIPTION

[0087] To help those skilled in the art understand the present invention, the following will provide a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0088] The present invention is described in detail below with reference to the accompanying drawings:

[0089] Example:

[0090] Please see the attached Figure 1 -Attached Figure 2 , an embodiment of the present invention provides a street lamp intelligent inspection and management system based on deep learning, including an image recognition module, an algorithm processing module, an environment module, a data analysis module, a permission management module, an intelligent path planning module, a video tracking module, a multi-scale detection module, an image conversion module, and a detection storage module;

[0091] The image recognition module is used to collect image data of street lights and realize image recognition, extract the features of street lights using convolutional neural networks, and use the Yolov4 algorithm model for target detection; the algorithm processing module is used to process the detected image data; the environment module includes defogging, denoising, deblurring and night enhancement processing units, and realizes data enhancement under different weather conditions through CutOut and GAN technology; the data analysis module combines historical trajectory and video playback functions to provide path planning, maintenance record management, statistics and analysis; the authority management module is used to set the access rights of inspection personnel, It supports the management of adding, deleting, modifying, and checking data permissions; the intelligent path planning module performs path planning based on work order data and task requirements, and automatically generates inspection and maintenance routes; the video tracking module is used to track and identify street light targets in video surveillance and optimize video detection performance; the multi-scale detection module uses PANet and FPN structures to improve the detection accuracy of street lights at different scales; the image conversion module converts RGB images into HSV or YUV format; the detection storage module records the detection results of street lights, the number of faulty lights, and the time points in the system database, and provides detection reports in image and text form.

[0092] The benefit of the image recognition module is that it can efficiently identify the status of street lights, so that the system can quickly and accurately detect whether the street lights are working properly, reduce the reliance on manual visual inspections, and improve inspection efficiency and detection accuracy; the benefit of the algorithm processing module is to ensure the accuracy of the detection results and provide reliable algorithm support for the system's fault judgment and feedback; the benefit of the environment module is to improve the adaptability and robustness of the system and to operate stably in various complex environments; the benefit of the data analysis module is to help management departments fully understand the status of street lights, facilitate the identification of high-incidence areas of faults and make trend predictions, optimize inspection and maintenance work, and support scientific decision-making; the benefit of the authority management module is to support data addition, deletion, modification and query authority management, ensuring data security and confidentiality The intelligent path planning module effectively improves inspection efficiency and resource utilization, reduces ineffective time during inspections, optimizes inspection routes, and reduces inspection costs. The video tracking module improves the detection performance of video surveillance and reduces the possibility of missed and false detections. The multi-scale detection module enables the system to maintain a high level of accuracy when identifying near and far streetlights or targets of different sizes. The image conversion module helps maintain stable detection results under various environmental conditions. The detection storage module supports the export of inspection reports in image and text formats, providing complete data support for inspection and maintenance work and facilitating long-term data analysis and record preservation.

[0093] In the image recognition module, a convolutional neural network combined with the Yolov4 algorithm model is used to detect street lamp targets. The core processing of this algorithm includes feature extraction, detection box regression, and classification. The following is an analysis of the algorithm formula in Yolov4:

[0094] Yolov4 uses convolution operations to extract features from images. The process performs convolution operations at different positions of the image through convolution kernels. The output of each convolution layer is calculated as follows:

[0095] Y i，j，k =σ(∑ m，n X i+m，j+n W m，n，k +b k ), where Y i，j，k is the value of the convolution result at the i-th, j-th position of the k-th channel, X i+m，j+n is the pixel of the input image at the corresponding position, W m，n，k is the weight of the convolution kernel, b k is the bias term, σ is the activation function;

[0096] Yolov4 predicts multiple detection boxes on the feature map, calculates their boundary positions and confidence levels, and the regression formula for position prediction is as follows: x =σ(t x )+c x, b y =σ(t y )+c y , Among them, (b x , b y ) is the center coordinate of the detection frame, t x and t y is the offset predicted by the network, (c x , c y ) is the coordinate of the upper left corner of the anchor box, (p w , p h ) is the width and height of the anchor box, t w and t h is the scaling factor for network prediction;

[0097] Yolov4 uses the Softmax or Sigmoid activation function to calculate the probability of the target category, and uses the Softmax function to normalize the probability prediction of the target category:

[0098] Among them, P(y=k|x) is the predicted probability of category k, z k is the activation value of category k, where k is the total number of categories;

[0099] Yolov4 uses an improved IoU loss function to measure the overlap between the detection box and the true box. The IoU loss function formula is:

[0100] Among them, IoU represents the IoU value between the detection box and the real box, c is the minimum diagonal length of the closure box surrounding the detection box and the real box, and α and v are weight parameters.

[0101] The main functions of Yolov4 algorithm:

[0102] Identify key information in the image through convolution operations;

[0103] Determine the location and size of the target through detection frame regression;

[0104] Use the Softmax or Sigmoid function to determine the target category within the detection box;

[0105] Through the CIOU loss function, the position, size and matching degree of the detection box are optimized to improve the accuracy and stability of the model.

[0106] The algorithm processing module includes calculation formulas for detection rate and false alarm rate;

[0107] The detection rate refers to the proportion of street lights that the system correctly identifies. It is used to measure the accuracy of the system. The calculation formula is:

[0108] Where n1 represents the actual number of street lights successfully detected by the system, and M is the total number of street lights;

[0109] The false alarm rate refers to the ratio of non-street lights that the system mistakenly identifies as street lights, reflecting the system's misdetection situation. Its calculation formula is:

[0110] Where n2 represents the number of non-street lights that are mistakenly identified as street lights, and N represents the total number of targets detected by the system.

[0111] The detection rate reflects the accuracy of the system, indicating the proportion of street lights that the system can identify. The higher the detection rate, the fewer missed detections the system has, and the higher the accuracy.

[0112] The false alarm rate reflects the system's error detection rate, indicating the proportion of non-street lamp objects that the system mistakenly identifies as street lamps. The higher the false alarm rate, the more false detections the system makes, reflecting the system's error recognition ability.

[0113] In the algorithm processing module, the unbalanced data is processed by the weighted loss function, and the loss function is calculated as:

[0114]

[0115] Among them, C is the total number of categories, w i is the weight of the category, y i is the true label, is the predicted value;

[0116] In the algorithm processing module, if accurate detection of the target at different positions is necessary, position accuracy calculation is introduced to measure the deviation of the detection frame:

[0117] in, is the predicted position, p is the actual position, d max is the maximum distance in the image.

[0118] The weighted loss function is used to address data imbalance. By assigning different weights to different classes, the loss function more effectively focuses on the minority classes in the imbalanced data, reduces the missed detection rate of minority class samples, and improves the fairness and accuracy of the overall model. When there are fewer samples of certain classes in the dataset, higher weights are assigned to these classes to help the model better learn these minority classes and improve its detection and classification performance.

[0119] Position accuracy calculation is used to quantify the positioning deviation of the detection frame. It measures the positioning accuracy of the detection frame by calculating the distance between the predicted position and the actual position and normalizing it to the maximum distance range of the image.

[0120] In the environment module, data enhancement is achieved through CutOut and generative adversarial network technology. The specific algorithm formula is as follows:

[0121] Assume that the input image is X∈R H×W×C , where H and W are the height and width of the image, C is the number of channels, and the center position of the occlusion is arbitrarily determined (c x ,c y ) and the side length l of the occluded area, then the occluded area is:

[0122]

[0123] Generative adversarial networks are used to enhance data under weather conditions. Generative adversarial networks consist of a generator G and a discriminator D, which are used to generate and discriminate synthetic images. The data enhancement process formula of the generative adversarial network is as follows:

[0124] Generator G: The generator inputs arbitrary noise z into the neural network to generate synthetic images Among them, z~p z , usually from a normal or uniform distribution;

[0125] Discriminator D: The discriminator D receives the input image and the output value represents the authenticity of the image. The loss of the discriminator is:

[0126]

[0127] Among them, p data is the distribution of real data;

[0128] Generator loss: The goal of the generator is to deceive the discriminator so that it cannot distinguish between real images and generated images. The generator loss is:

[0129] Optimization goal: Minimize the loss of the generator and discriminator through training iterations, while generating realistic augmented data:

[0130]

[0131] CutOut technology simulates the situation of partial information loss by blocking areas in the image, forcing the model to rely more on global features rather than local features of specific areas. Occlusion can improve the robustness and generalization ability of the model, especially in the environment of partial occlusion or information loss.

[0132] The main function of the generator is to generate realistic synthetic images from noise, simulating data in a specific environment. The synthetic data is used to expand the training dataset and improve the performance of the model in a specific environment.

[0133] The discriminator's task is to distinguish between real images and generated images. It improves its own discrimination ability by maximizing the loss function, helping the generator to continuously improve during the generation process and making the generated images more realistic.

[0134] The goal of the generator is to deceive the discriminator and make the generated images look more realistic. The generator strives to make the discriminator unable to distinguish between real images and generated images by minimizing its loss, thereby improving the quality of generated data.

[0135] The optimization process of the generative adversarial network is a game process. The generator and the discriminator compete with each other. The generator attempts to generate more realistic images, while the discriminator continuously improves its ability to distinguish between true and false. Ultimately, the generative adversarial network model reaches a Nash equilibrium, and the generated augmented data is more realistic, which helps the model perform well in training under data scarcity or specific conditions.

[0136] In the data analysis module, data is used for path planning, fault statistics, and analysis. The following is the algorithm formula:

[0137] Intelligent path planning is based on Euclidean distance or other path optimization algorithms to determine the best route during inspection or maintenance tasks. The core formula is as follows:

[0138] If the street light point position is (x i -y i ) and (y i -y j ), the distance calculation formula is:

[0139]

[0140] Path optimization: Assume that the inspection path is (P1, P2, ..., P n ), the total path length is:

[0141]

[0142] Combined with historical trajectories and fault records, the failure rate of street lamps is calculated and predicted. Assuming that the total number of monitored failures in the time period [T1, T2] is f, and the total number of monitored street lamps is N, the failure rate is calculated as follows: Where FR is the failure rate percentage;

[0143] Moving average method: Fault records of n days (f1, f2, ..., f n ), predict the failure value on day n+1 for:

[0144] Exponential smoothing method: For the smoothing coefficient α (0<α<1), the forecast value on the t+1th day for:

[0145] in, is the predicted value at the previous moment, f t is the actual value at the moment.

[0146] Euclidean distance is used to calculate the shortest straight-line distance between streetlights and is the basic unit of path planning. By calculating the distance between each streetlight, the system can optimize the route of inspection or maintenance tasks based on distance, thereby achieving efficient path planning.

[0147] Total path length is used to evaluate the travel time of inspection or maintenance tasks. By calculating the total length of the path points, the shortest or optimal path can be determined, helping to save time and resources.

[0148] Failure rate calculation is used to quantify the frequency of streetlight failures. By counting failures over a period of time and calculating the failure rate, the system can assess the overall health of streetlights and help determine the priority and necessity of inspections.

[0149] The moving average method is used to smooth historical fault data and predict future fault conditions. By calculating the average value of the data for the last n days, it eliminates short-term fluctuations in the data and provides a more stable reference for future decision-making.

[0150] Exponential smoothing is used to make predictions based on recent fault records. The smoothing coefficient controls the weight of the impact of historical data on the prediction. The method can quickly adapt to changing trends in the data and make gradual adjustments based on historical data to improve the accuracy of the prediction.

[0151] In the intelligent path planning module, the path planning algorithms include Dijkstra algorithm, A algorithm and dynamic planning;

[0152] Dijkstra's algorithm is used to find the shortest path from a starting point to an end point in a weighted graph. For a graph G = (V, E), where V is a set of vertices, E is a set of edges, and each edge (u, v) has a weight w(u, v).

[0153] The shortest path update formula is: d(v) = min(d(v), d(u) + w(u, v)).

[0154] Among them, d(v) is the shortest distance from the starting point to vertex v, d(u) is the distance from the starting point to vertex u, and w(u,v) is the weight of the edge from u to v;

[0155] Algorithm A uses the heuristic evaluation function h(v) to speed up the path search process. The total evaluation function is: f(v) = g(v) + h(v).

[0156] Where f(v) is the total evaluation value of node v, g(v) is the actual cost from the starting point to node v, and h(v) is the heuristic estimate from node v to the target node;

[0157] Dynamic programming formula: If the inspection task needs to traverse multiple nodes and return to the starting point, the dynamic programming method of the traveling salesman problem is used. The minimum cost of the trip is:

[0158]

[0159] Where C(S,v) is the minimum path length that traverses the nodes in S and ends at v, and d(u,v) is the distance between node u and node v.

[0160] The Dijkstra algorithm is used to find the shortest path from the starting point to the end point in a weighted graph. By continuously updating the shortest path distance of each node, the algorithm determines the minimum total cost path from the starting point to the target node. It optimizes route selection in inspection tasks, completes the task within the shortest path, and reduces inspection costs and time.

[0161] The A algorithm introduces a heuristic evaluation function h(v) and uses the estimated value to accelerate the search for the optimal path during the path search process. This allows the algorithm to converge to the optimal solution more quickly when dealing with large graphs or complex path problems, reducing unnecessary searches. This is particularly useful for inspection tasks, because inspection routes often have many target nodes and are widely distributed. The A algorithm quickly generates a near-optimal path, improving task efficiency.

[0162] In an inspection task, if it is necessary to traverse multiple nodes and eventually return to the starting point, the optimal path can be found by solving the traveling salesman problem. Dynamic programming reduces the amount of computation by storing the solutions to subproblems. It can effectively solve the traveling salesman problem and the path with the minimum total cost, making the planning of the inspection task optimal and efficient.

[0163] In the multi-scale detection module, a specific neural network structure and multi-scale training method are used to improve the detection accuracy of targets of different sizes. The following is the algorithm formula:

[0164] The feature pyramid network enhances the network's ability to detect objects of different scales by constructing a bottom-up feature pyramid. The feature graph pyramid is generated using the following formula:

[0165] P l =W l *C l +upsample(P l+1 ), where P lis the output feature map at scale l, W l is the learnable weight, C l is the original feature map, upsample(P l+1 ) represents the upsampling operation on the feature map of the previous layer;

[0166] The path aggregation network further improves the information transmission between different layers through top-down and bottom-up path aggregation based on the feature pyramid network. The specific calculation formula is:

[0167] P l ′=concat(P l ,downsample(P l-1 ′)), where P l ′ is the improved feature map, downsample(P l-1 ′) is the down sampling of the lower layer feature map, and concat represents the splicing operation of the feature map;

[0168] The multi-scale training method dynamically adjusts the input image size so that the network can learn information of different scales during training. Assume that the input image is X∈R H×W×C , randomly adjust the input size (H′, W′) with different scales during training:

[0169] (H′, W′) = α × (H, W), where α is the scaling factor;

[0170] Deformable convolution adds an offset term to the standard convolution kernel to adapt to the deformation of the target at different scales. Given the standard convolution kernel position (x i ,y i ), the offset is (Δx i ,Δy i ), then the output calculation formula of deformable convolution is:

[0171] y(p0)=∑ i w(p i )·x(p0+p i +Δp i ), where p0 is the convolution center point, w(p i ) is the convolution kernel weight, x(p0+p i +Δp i ) is the value of the input feature map at the offset position.

[0172] The Feature Pyramid Network (FPN) builds a feature pyramid from the bottom up, enabling the network to capture feature information at different scales. Through multi-level feature maps, the network can detect objects of different sizes in the image, thereby improving the ability to detect objects at multiple scales. The FPN makes the model more sensitive to identifying small and large objects, improving the accuracy and robustness of overall detection.

[0173] The path aggregation network improves the information transfer and feature fusion capabilities between feature maps at different levels through top-down and bottom-up path aggregation. It can better utilize the correlation between features of different scales and improve the detection effect of targets of different sizes. Especially in complex scenes, it can enhance the mutual information between small and large targets, further improving detection accuracy.

[0174] The multi-scale training method randomly adjusts the size of the training images so that the network can see images of different scales during the training process. This allows the model to learn to adapt to objects of different scales, thereby enhancing the model's generalization performance and being able to more effectively handle objects of different sizes in actual detection tasks.

[0175] Deformable convolution adds an offset to the standard convolution operation, allowing the convolution kernel to more flexibly adapt to the characteristics of the target at different scales and deformations. This helps capture targets with shape changes or complex geometric forms, and is particularly suitable for detecting targets that are irregular in size and position, significantly improving the detection performance of multi-scale and deformed targets.

[0176] In the image conversion module, to avoid the influence of the color temperature of street lamps of different materials, the original RGB data is converted to HSV or YUV. The algorithm formula is:

[0177] The algorithm formula for converting RGB to HSV is:

[0178]

[0179] The algorithm formula for converting RGB to YUV is:

[0180]

[0181] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A street lamp intelligent inspection and management system based on deep learning, characterized in that: It includes image recognition module, algorithm processing module, environment module, data analysis module, authority management module, intelligent path planning module, video tracking module, multi-scale detection module, image conversion module, and detection storage module; The image recognition module is used to collect image data of street lamps and realize image recognition, extract features of street lamps using convolutional neural networks, and perform target detection using Yolov4 algorithm model; The algorithm processing module is used to process the detected image data; The environment module includes defogging, denoising, deblurring and night enhancement processing units, which realize data enhancement under different weather conditions through CutOut and GAN technology; The data analysis module combines historical trajectory and video playback functions to provide path planning, maintenance record management, statistics and analysis; The authority management module is used to set the access rights of inspection personnel and supports the management of adding, deleting, modifying and checking data permissions; The intelligent path planning module performs path planning based on work order data and task requirements, and automatically generates inspection and maintenance routes; The video tracking module is used to track and identify street lamp targets in video surveillance and optimize video detection performance; The multi-scale detection module adopts PANet and FPN structures to improve the detection accuracy of street lamps at different scales; The image conversion module converts the RGB image into HSV or YUV format; The detection storage module records the detection results of the street lamps, the number of faulty lamps and the time points thereof into the system database, and provides a detection report in the form of images and text.

2. According to the deep learning-based intelligent inspection and management system for street lamps in claim 1, it is characterized in that: In the image recognition module, a convolutional neural network is used in combination with the Yolov4 algorithm model to detect street lamp targets. The core processing of the algorithm includes feature extraction, detection frame regression and classification. The following is an analysis of the algorithm formula in Yolov4: The Yolov4 uses convolution operation to extract features from the image. The process performs convolution operation at different positions of the image through the convolution kernel. The output of each convolution layer is calculated as follows: Y i，j，k =σ9∑ m，n X i+m，j+n ·W m，n，k +b k ), where Y i，j，k is the value of the convolution result at the i-th, j-th position of the k-th channel, X i+m，j+n is the pixel of the input image at the corresponding position, W m，n，k is the weight of the convolution kernel, b k is the bias term, σ is the activation function; The Yolov4 predicts multiple detection boxes on the feature map, calculates their boundary positions and confidence levels, and the regression formula for position prediction is as follows: x =σ(t x )+c x , b y =σ(t y )+c y , Among them, (b x , b y ) is the center coordinate of the detection box, t x and t y is the offset predicted by the network, (c x , c y ) is the coordinate of the upper left corner of the anchor box, (p w , p h ) is the width and height of the anchor box, t w and t h is the scaling factor predicted by the network; The Yolov4 uses the Softmax or Sigmoid activation function to calculate the probability of the target category, and the Softmax function is used to normalize the probability prediction of the target category: Among them, P(y=k|x) is the predicted probability of category k, z k is the activation value of category k, where k is the total number of categories; The Yolov4 uses an improved IoU loss function to measure the overlap between the detection box and the real box. The IoU loss function formula is: Among them, IoU represents the IoU value between the detection box and the real box, c is the minimum diagonal length of the enclosing box surrounding the detection box and the real box, and α and v are weight parameters.

3. According to the deep learning-based intelligent inspection and management system for street lamps in claim 1, it is characterized in that: The algorithm processing module includes calculation formulas for detection rate and false alarm rate; The detection rate refers to the proportion of street lights that the system correctly identifies, which is used to measure the accuracy of the system. The calculation formula is: Among them, n1 represents the actual number of street lights successfully detected by the system, and M is the total number of street lights; The false alarm rate refers to the ratio of non-street lights that are mistakenly identified as street lights by the system, reflecting the system's misdetection situation. The calculation formula is: Among them, n2 represents the number of non-street lights that are mistakenly identified as street lights, and N represents the total number of targets detected by the system.

4. According to the deep learning-based intelligent inspection and management system for street lamps in claim 1, it is characterized in that: In the algorithm processing module, the unbalanced data is processed by a weighted loss function, and the loss function is calculated as: Among them, C is the total number of categories, w i is the weight of the category, y i is the true label, is the predicted value; In the algorithm processing module, if accurate detection of the target at different positions is necessary, position accuracy calculation is introduced to measure the deviation of the detection frame: in, is the predicted position, p is the actual position, d max is the maximum distance in the image.

5. According to the deep learning-based intelligent inspection and management system for street lamps in claim 1, it is characterized in that: In the environment module, data enhancement is achieved through CutOut and generative adversarial network technology. The following is the specific algorithm formula: Assume the input image is X∈R H×W×C , where H and W are the height and width of the image, C is the number of channels, and the center position of the occlusion is arbitrarily determined (c x ,c y ) and the side length l of the occluded area, then the occluded area is: Generative adversarial networks are used for data enhancement under weather conditions. Generative adversarial networks consist of generators G and discriminators D, which are used to generate and discriminate synthetic images. The data enhancement process formula of generative adversarial networks is as follows: Generator G: The generator inputs arbitrary noise z into the neural network to generate synthetic images Among them, z~p z , usually from a normal or uniform distribution; Discriminator D: The discriminator D receives the input image and the output value represents the authenticity of the image. The loss of the discriminator is: Among them, p data is the distribution of real data; Generator loss: The goal of the generator is to deceive the discriminator so that it cannot distinguish between real images and generated images. The generator loss is: Optimization goal: Minimize the loss of the generator and discriminator through training iterations, while generating realistic augmented data:

6. According to the deep learning-based intelligent inspection and management system for street lamps in claim 1, it is characterized in that: In the data analysis module, data is used for path planning, fault statistics and analysis. The following is the algorithm formula: The intelligent path planning is based on Euclidean distance or other path optimization algorithms to determine the best route in inspection or maintenance tasks. The core formula is as follows: If the street light point position is (x i -y i ) and (y i -y j ), then the distance calculation formula is: Path optimization: Assume that the inspection path is (P1, P2, ..., P n ), the total path length is: Combined with historical trajectories and fault records, the failure rate of street lamps is counted and predicted. Assuming that in the time period [T1, T2], the total number of monitored failures is f, and the total number of monitored street lamps is N, the failure rate is calculated as follows: Among them, FR is the failure rate percentage; Moving average method: for n days of fault records (f1, f2, ..., f n ), predict the failure value on day n+1 for: Exponential smoothing method: For the smoothing coefficient α (0<α<1), the forecast value for the t+1th day for: in, is the predicted value at the previous moment, f t is the actual value at the moment.

7. The street lamp intelligent inspection and management system based on deep learning according to claim 1 is characterized in that: In the intelligent path planning module, the path planning algorithms include Dijkstra algorithm, A algorithm and dynamic planning; The Dijkstra algorithm is used to find the shortest path from a starting point to an end point in a weighted graph. For a graph G = (V, E), V is a set of vertices, E is a set of edges, and each edge (u, v) has a weight w(u, v); The shortest path update formula is: d(v)=min(d(v),d(u)+w(u,v)), Among them, d(v) is the shortest distance from the starting point to vertex v, d(u) is the distance from the starting point to vertex u, and w(u,v) is the weight of the edge from u to v; The A algorithm uses a heuristic evaluation function h(v) to speed up the path search process. The total evaluation function is: f(v)=g(v)+h(v). Among them, f(v) is the total evaluation value of node v, g(v) is the actual cost from the starting point to node v, and h(v) is the heuristic estimate from node v to the target node.

8. The street lamp intelligent inspection and management system based on deep learning according to claim 7 is characterized in that: The dynamic programming formula, if the inspection task needs to traverse multiple nodes and return to the starting point, adopts the dynamic programming method of the traveling salesman problem, and the minimum cost of the trip is: Among them, C(S,v) is the minimum path length that traverses the nodes in S and ends at v, and d(u,v) is the distance between node u and node v.

9. The street lamp intelligent inspection and management system based on deep learning according to claim 1, characterized in that: In the multi-scale detection module, a specific neural network structure and a multi-scale training method are used to improve the detection accuracy of targets of different sizes. The following is the algorithm formula: The feature pyramid network builds a bottom-up feature pyramid to enhance the network's ability to detect objects of different scales. The feature graph pyramid is generated by the following formula: P l =W l *C l +upsample(P l+1 ), where P l is the output feature map at scale l, W l is the learnable weight, C l is the original feature map, upsample(P l+1 ) represents the upsampling operation on the feature map of the previous layer; The path aggregation network further improves the information transmission between different layers through top-down and bottom-up path aggregation based on the feature pyramid network. The specific calculation formula is: P l ′=concat(P l ,downsample(P l-1 ′)), where P l ′ is the improved feature map, downsample(P l-1 ′) is the down sampling of the lower layer feature map, and concat represents the concatenation operation of the feature map; The multi-scale training method dynamically adjusts the input image size so that the network can learn information of different scales during training. Assume that the input image is X∈R H×W×C , using different scales of input size (H′, W′) for random adjustment during training: (H′, W′) = α × (H, W), where α is the scaling factor; Deformable convolution adds an offset term to the standard convolution kernel to adapt to the deformation of the target at different scales. Given a standard convolution kernel position (x i ,y i ), the offset is (Δx i ,Δy i ), then the output calculation formula of deformable convolution is: y(p0)=∑ i w(p i )·x(p0+p i +Δp i ), where p0 is the convolution center point, w(p i ) is the convolution kernel weight, x(p0+p i +Δp i ) is the value of the input feature map at the offset position.

10. The street lamp intelligent inspection and management system based on deep learning according to claim 1, characterized in that: In the image conversion module, the influence of the color temperature of street lamps of different materials is avoided, and the original RGB data is converted to HSV or YUV. The algorithm formula is: The algorithm formula for converting RGB to HSV is: The algorithm formula for converting RGB to YUV is:

Images