A neural network-based method for identifying open circuits and broken bridges

Abstract

This invention relates to a neural network-based method for identifying broken roads and bridges, belonging to the fields of intelligent driving and computer vision technology. The method involves creating a dataset by augmenting a small number of real images of broken roads and bridges, along with images obtained through model building, affine transformation, and generative adversarial networks. This dataset is then imported into an improved YOLOv8 neural network with added MSCAM and LWM modules for deep learning training to obtain optimal weight parameters, which are then used to detect broken roads and bridges. Video footage captured by an onboard camera is imported into the trained neural network, and the system determines whether objects in the video meet a threshold; if so, a broken road or bridge is identified. This invention effectively utilizes a small amount of data for training, and the improved YOLOv8 neural network training results in higher recognition accuracy and a longer detection range.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving and computer vision technology, and relates to a method for identifying broken roads and bridges based on neural networks. Background Technology

[0002] Driving safety is a core issue in the development of intelligent driving technology. In recent years, intelligent driving technologies, represented by Advanced Driver Assistance Systems (ADAS), have made significant progress. By integrating multiple sensors such as cameras, radar, and lidar, and combining them with advanced algorithms, vehicles' ability to perceive their surroundings has been significantly improved. Features such as adaptive cruise control, lane keeping assist, and automatic emergency braking have been applied in many mass-produced models, effectively reducing the incidence of traffic accidents.

[0003] However, current intelligent driving systems still have a significant weakness when dealing with a type of "low-frequency, high-risk" extreme road condition: the ability to identify and respond to sudden road closures or bridge collapses ahead. Such extreme dangerous situations often pose a devastating threat to the lives and property of drivers and passengers.

[0004] Currently, no commercially available autonomous vehicles are equipped with dedicated recognition and warning systems for severe road structural damage such as broken roads and bridges. Drivers primarily rely on visual observation of the road surface and emergency braking accordingly. However, human drivers' reaction time, as well as potential distractions and fatigue during long-distance driving, pose significant safety hazards. In recent years, numerous tragic accidents involving vehicles falling and injuries have occurred due to failure to detect broken roads or bridges in time, highlighting the gaps and inadequacies in current technology in this area.

[0005] A thorough analysis of current intelligent driving perception systems reveals three main technical bottlenecks in identifying broken roads / bridges:

[0006] (1) Extreme scarcity of key data: The training of deep learning models cannot be separated from massive and diverse data. However, in publicly available road traffic datasets, image or sensor data samples of scenarios such as road closures and bridge collapses are extremely rare. By searching major global autonomous driving dataset platforms, only a very small amount of relevant and effective data can be obtained. This scarcity of data makes it difficult to effectively train traditional data-driven models, fundamentally limiting the development and iteration of algorithms.

[0007] (2) Bottleneck in long-distance small target recognition: The initial features of road or bridge breaks (such as distant bridge surface cracks or the edge lines of road collapses) occupy only a very small percentage of pixels in the sensor (especially camera) image when the vehicle is traveling at a long distance, appearing as "small targets". Current general target detection algorithms experience a sharp drop in recognition accuracy when dealing with such long-distance, small-angle scenarios. This means that the vehicle may need to travel to a very close distance to trigger a warning, and by then the best opportunity to take effective braking or evasive action has often been missed.

[0008] (3) Insufficient robustness under complex environmental interference: The actual driving environment is complex and changeable. Obstacles on the road (such as vehicles in front, roadside trees), severe weather (such as rain, snow, dense fog), and drastic changes in lighting (such as entering and exiting tunnels, strong light at night) can seriously interfere with the quality of the sensor's raw data. In addition, noise, watermarks, and other factors that may occur during the image sensor acquisition process also pose a serious challenge to the stability and reliability (i.e., robustness) of traditional recognition models, which can easily lead to missed detections or false alarms.

[0009] In summary, current intelligent driving technology has significant technical shortcomings in addressing serious road safety hazards such as road closures and bridge collapses. The scarcity of data samples, insufficient ability to identify small targets at long distances, and low robustness in complex environments collectively contribute to the lack of relevant functions. This technological gap directly poses a potential constraint on the development of the entire intelligent driving industry.

[0010] Therefore, there is an urgent need in the industry to develop a brand-new technical solution that can effectively identify and judge whether there are structural defects such as road breaks or bridge collapses on the road ahead from a long distance with high precision and speed. Summary of the Invention

[0011] In view of this, the purpose of this invention is to provide a method for identifying broken circuits and bridges based on neural networks. By feeding the dataset of broken circuits and bridges into an improved YOLOv8 neural network model for training, the features of the broken circuit and bridge data are extracted and the relevant weight parameters are solved. The weight parameters obtained from the training are used to achieve high-precision long-distance identification of broken circuits and bridges in imported video data.

[0012] To achieve the above objectives, the present invention provides the following technical solution:

[0013] A method for identifying open circuits and broken bridges based on neural networks specifically includes the following steps:

[0014] S1: Collect images of broken roads and bridges, perform data annotation and data augmentation, and create a dataset of broken roads and bridges.

[0015] S2: Improve the YOLOv8 neural network model using the MSCAM (Multi-Scale Convolutional Attention Module) and LWM, and add a small target detection head;

[0016] S3: Utilize the improved YOLOv8 neural network model to perform deep learning on the broken circuit and broken bridge dataset, extract the features of broken circuit and broken bridge, and train to obtain the optimal weight parameters.

[0017] S4: Import the video footage captured by the vehicle-mounted camera into the trained, improved YOLOv8 neural network model. The model autonomously determines whether the objects in the video reach a threshold. If they do, it indicates the presence of a broken road or bridge, and the identification is successful.

[0018] Furthermore, step S1 specifically includes the following steps:

[0019] S11: Collect real photos of broken roads and bridges from the internet; use Generative Adversarial Networks (GANs) to generate reliable images of broken roads and bridges; build 3D models to obtain images of broken roads and bridges;

[0020] S12: Use affine transformation to generate long-distance images of broken roads and bridges from the collected images, and then perform data annotation;

[0021] S13: Use data augmentation algorithms to augment the labeled image data, and realize functions such as flipping, rotating, scaling, cropping, noise reduction, color dithering, random grayscale conversion, and random lighting transformation.

[0022] S14: Integrate the above image files and their data annotation files into a broken road and broken bridge dataset.

[0023] Furthermore, in step S2, the MSCAM is an EMCAD_block structure used to replace some convolutional structures in the YOLOv8 backbone network. It includes multi-scale convolutional blocks (MSCB), channel attention blocks (CAB), and spatial attention blocks (SAB). By fusing multi-scale convolution and multi-dimensional attention mechanisms during feature extraction, joint enhancement of spatial and channel features is achieved.

[0024] The CAB block extracts channel information through global average pooling and max pooling, generates weight coefficients, and then uses the Sigmoid function to weight the original feature map to enhance key channel features. The SAB block generates a spatial attention map through pooling operations to guide the model to focus on key regions in the feature map and improve localization accuracy. The MSCB block uses different convolution kernels such as 1×1, 3×3, and 5×5 for parallel processing and fuses multi-scale features to adapt to objects of different sizes.

[0025] Further, in step S2, the LWM specifically includes: using Discrete Wavelet Transform (DWT) to decompose the input feature map into four sub-bands: Low Frequency Approximation (LL), Horizontal High Frequency (LH), Vertical High Frequency (HL), and Diagonal High Frequency (HH), to achieve natural multi-scale separation of image structure and detail information; subsequently, the SAB sub-module acts independently on each sub-band, learning and applying adaptive weights through a lightweight attention mechanism (such as channel or spatial attention) to enhance key information (such as strengthening high-frequency edges and suppressing low-frequency noise); finally, Inverse Discrete Wavelet Transform (IDWT) accurately reconstructs the weighted sub-bands back to the spatial domain and fuses them to form the enhanced output feature map.

[0026] Furthermore, in step S2, the small target detection head is a small target detection branch added on the basis of the original YOLOv8 detection head. It extracts local feature information from the high-resolution feature layer and performs localization and classification output through deep convolution, which can significantly improve the detection recall rate of small targets without increasing the computational load significantly.

[0027] Furthermore, in step S2, the specific structure of the improved YOLOv8 neural network model is as follows: the backbone network includes 5 convolutional layers, 4 C2f_LWM modules, 1 SPPF module, and 1 MSCAM module. Among them, layers 0, 1, 3, 5, and 7 are convolutional layers, layers 2, 4, 6, and 8 are C2f_LWM modules (Cross-Stage Partial with Lightweight Wavelet Module), layer 9 is SPPF module (Spatial Pyramid Pooling - Fast), and layer 10 is MSCAM module. The C2f_LWM module introduces a lightweight wavelet neural network into the traditional C2f structure. Through Discrete Wavelet Transform (DWT) and Inverse Wavelet Transform (IDWT) mechanisms, it achieves multi-frequency domain decomposition and reconstruction of input features, effectively separating low-frequency background information from high-frequency edge details. This module combines multi-scale convolution with wavelet filters, using trainable wavelet decomposition and reconstruction parameters to achieve adaptive frequency domain feature enhancement. This significantly reduces the number of model parameters and computational complexity while maintaining feature expressive power, improving the network's feature compression efficiency and running speed. The SPPF module is used to spatially aggregate features at different scales to enhance the model's multi-scale feature extraction capabilities. The MSCAM module consists of a multi-scale convolutional block (MSCB), a channel attention block (CAB), and a spatial attention block (SAB). By integrating multi-scale convolution and multi-dimensional attention mechanisms during feature extraction, it achieves joint enhancement of spatial and channel features. Specifically, the MSCB module uses a multi-scale depthwise separable convolutional structure to extract multi-scale feature information, the CAB module adaptively calculates channel weights to highlight key feature channels, and the SAB module generates a spatial attention weight map to strengthen the features of the target region. This effectively improves the accuracy and robustness of the network in identifying broken roads and bridges in complex backgrounds.

[0028] The neck network consists of 3 upsampling layers, 6 feature concatenation layers, 6 C2f layers, and 3 convolutional layers. Layers 11, 14, and 17 are upsampling layers; layers 12, 15, 18, 21, 24, and 27 are feature concatenation layers; layers 13, 16, 19, 22, 25, and 28 are C2f layers; and layers 20, 23, and 26 are convolutional layers.

[0029] The detection section includes four detection heads.

[0030] The image processing workflow of the improved YOLOv8 neural network model includes the following steps:

[0031] S21: The image is input into the backbone network of the model; each convolutional layer uses a 3×3 convolutional kernel to perform convolution operations by sliding on the image to extract local features; the stride of the convolutional layer is 2, and the size of the input feature map is halved for each convolutional layer; the number of channels of the convolutional layers is gradually increased in the order of (64, 128, 256, 512, 1024); the C2f_LWM module first uses 1×1 convolution to adjust the number of channels of the feature map, and then uses 3×3 convolution to extract local spatial information. The wavelet transform in the C2f_LWM module, through multi-scale spatial frequency feature fusion, lightweight compression and noise reduction optimization, enables the improved YOLOv8 to extract multi-scale target features more accurately and efficiently in complex scenes; the SPPF module is a fast spatial pooling pyramid layer with a pooling kernel size of 5; the output image will enter the MSCAM module, which adaptively strengthens key features and suppresses redundant information through multi-scale convolution fusion and channel-spatial attention weighting, effectively improving the model's feature representation ability for multi-scale targets and complex scenes;

[0032] S22: The feature maps extracted from the backbone network are input into the neck network for feature fusion. Through multiple upsampling and feature fusion, the feature maps are finally input to the detection head at layers 19, 22, 25, and 28 for detection. The dynamic detection head predicts the bounding box coordinates of the target: a 3×3 convolution operation is performed on the fused feature map with the ReLU function as the activation function, and then a 1×1 convolution is used to adjust the channel dimension to the regression parameters, which are four values: x, y, w, and h. Where x and y represent the horizontal and vertical coordinates of the center point of the bounding box, respectively; w and h represent the width and height of the bounding box, respectively. Finally, the target category and bounding box are output.

[0033] Furthermore, step S3 specifically includes the following steps:

[0034] S31: Adjust the core settings of the deep learning algorithm, such as Epoch, Batch Size, and optimizer, and select the improved YOLOv8 neural network model.

[0035] S32: Write the dataset of broken circuits and bridges into the training algorithm, use the improved YOLOv8 neural network model for deep learning, and extract the features of broken circuits and bridges.

[0036] S33: After training, the optimal weight parameters are obtained and written back into the model for detection.

[0037] Furthermore, step S4 specifically includes the following steps:

[0038] S41: Load the optimal weight parameter file into the improved YOLOv8 neural network model;

[0039] S42: Adjust precision and recall by setting confidence thresholds and IoU thresholds appropriately;

[0040] S43: Convert the video footage captured by the vehicle-mounted camera into a different format and import it into the detection algorithm;

[0041] S44: Run the detection algorithm. The model automatically determines whether the objects in the video reach the threshold. If they do, it means that there is a broken circuit or bridge, and the recognition is successful.

[0042] The beneficial effects of this invention are as follows:

[0043] (1) The present invention employs multiple methods to simulate and augment the original data when constructing the dataset, ensuring the quality of the training data, making it comprehensive, realistic, and abundant, and compensating for the lack of data on long-distance road and bridge breaks and the problem of numerous road environment interference factors.

[0044] (2) In improving the YOLOv8 neural network model, the MSCAM module (multi-scale convolutional attention enhancement module) added in this invention can effectively improve the model's sensitivity to targets of different scales, the LWM module (wavelet transform feature extraction module) added can improve the ability to extract frequency domain information (high-frequency details, edges, and textures of the image), and the small target detection head added can significantly improve the recognition effect of long-distance broken circuits and bridges.

[0045] (3) The technological breakthrough of this invention will significantly improve the perception and early warning capabilities of intelligent driving vehicles in extremely dangerous road conditions, make up for the physiological limitations of human drivers, provide a more solid guarantee for driving safety, and have significant social value and broad application prospects.

[0046] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0048] Figure 1 This is a flowchart of the open circuit and broken bridge identification method based on the improved YOLOv8 neural network model of the present invention;

[0049] Figure 2 Create detailed maps for the broken road and bridge dataset;

[0050] Figure 3Detailed diagram of the improved YOLOv8 neural network model;

[0051] Figure 4 Detailed flowchart for identifying and judging road and bridge breaks;

[0052] Figure 5 Images created using 3D modeling software;

[0053] Figure 6 This is a schematic diagram of affine transformation operations;

[0054] Figure 7 This is a diagram illustrating the effect of an affine transformation. Detailed Implementation

[0055] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0056] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0057] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0058] Please see Figures 1 to 7 This invention provides a method for identifying open circuits and broken bridges based on an improved YOLOv8 neural network model, specifically including the following steps:

[0059] S1: Collect images of broken roads and bridges, perform data annotation and data augmentation, and create a dataset of broken roads and bridges; specifically including the following steps:

[0060] S101: Conduct image retrieval on major mainstream networks to collect real images of road and bridge breaks; based on Generative Adversarial Network (GAN), use algorithmic schemes to add defects and reduce dependence on defective data to generate reliable images of road and bridge breaks; use 3D modeling software to create 3D models of road and bridge breaks based on real road conditions to obtain images.

[0061] Generative Adversarial Networks (GANs) are deep learning frameworks consisting of two core components: a generator and a discriminator. They generate high-quality simulated data through an adversarial training mechanism. The generator synthesizes data (such as images and text) from random noise, aiming to approximate the distribution of real data. The discriminator distinguishes between real and generated data through a binary classification task. Both components alternately optimize in a minimax game, ultimately enabling the generator to produce samples that are difficult to distinguish from real data.

[0062] 3D modeling software was used to create 3D models of road and bridge closures, mimicking real-world road conditions. Specifically, publicly available road condition information and images were obtained from online resources such as transportation department websites and news reports. Examples included road closures due to construction in urban areas and bridge closures caused by geological disasters in mountainous regions. These photos or videos served as references for modeling. In the 3D modeling software, the model's position, size, and shape were adjusted based on the perspective and proportions of the objects in the images, ensuring the model's appearance closely matched the real-world scene. Finally, the position and angle of objects were adjusted within the modeling software, and images were cropped. (See attached image). Figure 5 .

[0063] S102: Use OpenCV to perform affine transformation on the collected close-range images of broken roads and bridges to generate long-range images of broken roads and bridges, in order to make up for the lack of long-range, small-angle image data. Then, use professional annotation software to annotate the data, paying attention to the format of the data annotation file.

[0064] Affine transformation is a linear transformation that maps two-dimensional coordinates (x, y) to another set of two-dimensional coordinates (u, v), transforming a nearby image into an image similar to that seen from a distance through translation, rotation, and scaling.

[0065] The collected close-up images of broken roads and bridges were subjected to affine transformation using OpenCV. Specifically, a non-uniform scaling transformation matrix was applied, and appropriate parameters were selected to stretch the target (broken roads and bridges) horizontally and compress it vertically (see...). Figure 6 This makes the target appear slender, such as transforming a circular pit into an ellipse (see...). Figure 7Affine transformation processing involves scaling, rotating, and translating the obtained near-field (large angle) broken road / bridge samples to simulate the visual effect of detecting a broken road / bridge at a distance. This compensates for the lack of data in distant, small-angle images, rather than directly generating small-angle samples through affine transformation operations. Then, professional annotation software is used for data annotation, paying attention to the format of the annotation file.

[0066] S103: Data augmentation algorithms are used to augment a large batch of labeled image data, enabling functions such as flipping, rotating, scaling, cropping, noise reduction, color jittering, random grayscale conversion, and random lighting transformation. The image files and their labeled data files are then integrated to form the broken road and broken bridge dataset.

[0067] Data augmentation refers to techniques that improve the generalization ability and robustness of a model by performing a series of random transformations (such as rotation, scaling, cropping, color adjustment, etc.) on the original training data to generate diverse new samples.

[0068] S2: Improve the YOLOv8 neural network model using the MSCAM module (Multi-Scale Convolutional Attention Enhancement Module) and the LWM module (Wavelet Neural Network), and add a small target detection head; specifically including the following steps:

[0069] S201: Import the broken circuit and broken bridge dataset into various neural network models (such as YOLOv5, YOLOv8, and Faster R-CNN) for ablation experiments. Compare and analyze their loss function, precision and recall, mean precision (mAP), etc., and select the neural network model with the best broken circuit and broken bridge recognition performance. Here, the YOLOv8 neural network model is selected.

[0070] Mean Average Precision (mAP) is a core metric for measuring the overall performance of a model in object detection tasks. Essentially, it is a weighted average of the average precision (AP) for each category at different IoU thresholds (usually 0.5 to 0.95), used to comprehensively evaluate the joint accuracy of model localization and classification.

[0071] S202: See also Figure 3 The selected YOLOv8 neural network model was improved by adding an MSCAM module (multi-scale convolutional attention enhancement module) to effectively improve the model's sensitivity to targets at different scales, adding LWM (wavelet neural network) to improve the ability to extract frequency domain information (high-frequency details, edges, and textures of the image), and adding a small target detection head. Figure 3In this module, C2f_LWM is a lightweight wavelet cross-stage partial fusion module (an improved version of the original feature fusion module in YOLOv8); SPPF is a spatial pyramid pooling module; MSCAM is a multi-scale convolutional attention enhancement module; and C2f is the original feature fusion module in YOLOv8.

[0072] The MSCAM module (Multi-Scale Convolutional Attention Enhancement Module) is a module that enhances the perceptual capabilities of a model by fusing multi-scale features and channel attention mechanisms, primarily used to improve the performance of object detection models. Its core structure includes a Channel Attention Block (CAB), a Spatial Attention Block (SAB), and a Multi-Scale Convolutional Block (MSCB). By dynamically adjusting the weights of channels and space, the model becomes more flexible in complex scenes. Core structure: The Channel Attention Block (CAB) extracts channel information through global average pooling and max pooling, generates weight coefficients, and then uses the Sigmoid function to weight the original feature map, enhancing key channel features; the Spatial Attention Block (SAB) generates a spatial attention map through pooling operations, guiding the model to focus on key regions in the feature map and improving localization accuracy; the Multi-Scale Convolutional Block (MSCB) uses different convolutional kernels (1×1, 3×3, 5×5, etc.) in parallel processing, fusing multi-scale features to adapt to objects of different sizes. Core Process: Multi-scale extraction: Input features are spatially downsampled at different lengths (e.g., average pooling) to generate multi-scale feature maps; Channel attention generation: Channel weights are independently calculated for features at each scale (using a Squeeze-and-Excitation mechanism to first compress spatial information and then adaptively learn channel importance); Feature fusion: The weighted multi-scale features are upsampled back to their original size and summed, ultimately outputting key features that enhance cross-scale contextual information. Features: MSCAM works collaboratively through the above sub-modules to enhance feature representation capabilities while reducing computational costs, enabling the model to focus on both local details and global semantics, significantly improving the discriminative power of multi-scale targets in tasks such as object detection / segmentation.

[0073] The MSCB submodule uses multiple different convolutional kernels (such as 1×1, 3×3, 5×5) to perform depthwise separable convolutions on the input features to extract multi-scale information, as shown below:

[0074]

[0075] in, Indicates the kernel size as k Depth convolution operation, F For the input feature map, For the extraction of multi-scale information;

[0076] The CAB submodule (channel attention) processes the channel dimension in parallel using global average pooling and max pooling, and generates channel weights through MLP for weighting, as shown in the following formula:

[0077]

[0078] in, This represents the Channel Attention Weight Matrix, which is generated by the Channel Attention Module (CAB) through multilayer perceptron (MLP) mapping of the global average pooling and max pooling results of the input features. It is used to weight the importance of each channel. For the Sigmoid function, The ReLU activation function is used. FC1 is the first fully connected layer, used to perform dimensionality reduction mapping on the global channel features to reduce the number of parameters and extract compact channel features. FC2 is the second fully connected layer, used to restore the dimensionality-reduced features to the original channel dimension to generate complete channel weight vectors. GAP is Global Average Pooling, used to calculate the average value of the input feature map in the spatial dimension, thereby extracting the global statistical features of the channels. MaxP is Global Max Pooling, used to extract the salient feature responses of each channel. The SAB submodule (Spatial Attention) concatenates the average and maximum values ​​of the channel dimensions into a two-channel input and applies a 7×7 convolution to generate a spatial weight map.

[0079]

[0080] in, The Spatial Attention Weight Matrix is ​​used to characterize the importance of each position in the spatial dimension (H × W) of the input feature map.

[0081] The final output is:

[0082]

[0083] in, This is the output of the MSCAM module.

[0084] The MSCAM structure enhances the model's ability to respond to occluded areas, detailed textures, and multi-scale targets, making it particularly suitable for detecting irregular targets such as fractures and collapses in complex traffic environments.

[0085] LWM (Wavelet Transform Feature Extraction Module) is a feature enhancement module designed specifically for image processing tasks such as super-resolution, denoising, and enhancement. Its core lies in using Discrete Wavelet Transform (DWT) to decompose the input feature map into four sub-bands: Low-Frequency Approximation (LL), Horizontal High-Frequency (LH), Vertical High-Frequency (HL), and Diagonal High-Frequency (HH), achieving a natural multi-scale separation of image structure and detail information. Subsequently, Spatial Attention Blocks (SABs) act independently on each sub-band, learning and applying adaptive weights through lightweight attention mechanisms (such as channel or spatial attention) to enhance key information (e.g., strengthening high-frequency edges and suppressing low-frequency noise). Finally, Inverse Discrete Wavelet Transform (IDWT) accurately reconstructs the weighted sub-bands back to the spatial domain, fusing them to form the enhanced output feature map. The core advantages of this module are: 1) strict reversible multi-scale decomposition and reconstruction based on DWT / IDWT, ensuring information integrity; 2) direct adaptive weighting in the frequency domain (wavelet sub-band), specifically optimizing features at different scales and directions; 3) significantly improving the model's ability to reconstruct edge sharpness, texture details and structural fidelity in image restoration tasks.

[0086] LWM (Wavelet Neural Network) constructs an end-to-end trainable Discrete Wavelet Transform (DWT) and Inverse Wavelet Transform (IDWT) module, which replaces fixed filters by learning the gradient of wavelet filters, thereby improving the frequency domain adaptability of the network.

[0087] The filter training formula (soft constraint) is as follows:

[0088] (1) Perfect Reconstruction Conditions

[0089] Let the wavelet decomposition filter be , The reconstruction filter is , Then the following condition is met:

[0090]

[0091] In practical applications, it is written as a loss function for optimization:

[0092]

[0093] in, These are the low-frequency reconstructed image or feature map components obtained after wavelet inverse transform, used to preserve overall structural information and fuse with high-frequency details to form a complete output. It is a unit impulse response.

[0094] (2) Alias ​​Cancellation

[0095] To suppress aliasing distortion, the following should be satisfied:

[0096]

[0097] The loss function is:

[0098]

[0099] in, It is the aliasing error in the low-frequency components during the inverse wavelet transform process. It is used to constrain the filter through alias-cancellation loss during training to ensure the orthogonality of the filter and the reconstruction accuracy.

[0100] The Small Object Detection Head (SOD Head) is a module specifically designed to optimize the detection performance of small, low-resolution objects in images. Its core lies in directly utilizing the high-resolution feature maps output from the shallow layers of the backbone network (preserving rich location and detail information) and strengthening key regions and suppressing background noise through spatial attention mechanisms (such as SAM). Based on this, dense small anchor boxes (or dot matrix) and lightweight predictive convolutional layers are designed for fine-grained localization and classification. During training, FocalLoss is used to alleviate sample imbalance, GIoU / DIoU Loss to improve the accuracy of small box localization, and auxiliary supervision signals are often used to guide the network to focus on subtle features. This structure significantly improves the recall and localization accuracy of the model in dense small object scenarios such as remote sensing, surveillance, and microscopic images, effectively solving the problems of false negatives and false negatives caused by feature downsampling loss and weakened semantic information in general detection heads.

[0101] The final improved network structure (such as) Figure 3 As shown below:

[0102] The backbone network consists of 5 convolutional layers, 4 C2f_LWM modules, 1 SPPF module and 1 MSCAM module. Layers 0, 1, 3, 5 and 7 are convolutional layers, layers 2, 4, 6 and 8 are C2f_LWM modules, layer 9 is an SPPF module and layer 10 is an MSCAM module.

[0103] The neck network consists of 3 upsampling layers, 6 feature concatenation layers, 6 C2f layers, and 3 convolutional layers. Layers 11, 14, and 17 are upsampling layers; layers 12, 15, 18, 21, 24, and 27 are feature concatenation layers; layers 13, 16, 19, 22, 25, and 28 are C2f modules; and layers 20, 23, and 26 are convolutional layers.

[0104] The detection section consists of four detection heads;

[0105] The specific process includes the following steps:

[0106] (1) Input the image into the backbone network of the model; each convolutional layer uses a 3×3 convolution kernel to slide on the image to perform convolution operations and extract local features of the image; the stride of the convolutional layer is 2, and each convolutional layer will halve the size of the input feature map; the number of channels of the convolutional layer gradually increases according to (64, 128, 256, 512, 1024); the C2f_LWM module first uses 1×1 convolution to adjust the number of channels of the feature map, and then uses 3×3 convolution to extract local spatial information. The wavelet transform in the C2f_LWM module, through multi-scale spatial frequency feature fusion, lightweight compression and noise reduction optimization, enables the improved YOLOv8 to extract multi-scale target features more accurately and efficiently in complex scenes; SPPF is a fast spatial pooling pyramid layer with a pooling kernel size of 5; the output image will enter the MSCAM module. The MSCAM module uses multi-scale convolution fusion and channel-spatial attention weighting to adaptively strengthen key features and suppress redundant information, effectively improving the model's feature representation ability for multi-scale targets and complex scenes.

[0107] (2) The feature map extracted by the backbone network is input into the neck network for feature fusion. After multiple upsampling and feature fusion, the feature map is finally input to the detection head for detection at layers 19, 22, 25 and 28. The dynamic detection head predicts the bounding box coordinates of the target: 3×3 convolution operation is performed on the fused feature map, the activation function is ReLU function, and then 1×1 convolution is used to adjust the channel dimension to the regression parameters, which are 4 values: x, y, w, h; where x and y represent the horizontal and vertical coordinates of the center point of the bounding box, respectively; w and h represent the width and height of the bounding box, respectively. Finally, the target category and bounding box are output.

[0108] S3: Utilize the improved YOLOv8 neural network model to perform deep learning on the broken circuit and broken bridge dataset, extract the features of broken circuits and broken bridges, and train to obtain the optimal weight parameters.

[0109] S301: Adjust the core settings of the deep learning algorithm, such as Epoch, Batch Size, and optimizer, compare and analyze the effect of circuit and bridge break recognition under different settings, select the final setting parameters, and select the improved YOLOv8 neural network model.

[0110] S302: Write the dataset of broken circuits and bridges into the training algorithm, use the improved YOLOv8 neural network model for deep learning, extract the features of broken circuits and bridges, obtain the optimal weight parameters after training, and write them back into the model for detection.

[0111] S4: Import the video footage captured by the vehicle-mounted camera into the trained neural network model. The model autonomously determines whether the objects in the video reach a threshold. If they do, it indicates that there is a broken road or bridge, and the identification is successful.

[0112] S401: Load the optimal weight parameter file into the neural network model.

[0113] S402: Create a UI interface to facilitate parameter adjustment; adjust precision and recall by setting confidence thresholds and IoU thresholds appropriately to achieve the best recognition results.

[0114] S403: Convert the video footage captured by the vehicle-mounted camera into a different format and import it into the detection algorithm as a data source.

[0115] S404: Run the detection algorithm. The model autonomously determines whether the objects in the video have reached the threshold. If they have, it means that there is a broken circuit or bridge, the recognition is successful, and a detection success signal is output.

[0116] As can be seen from the principle, this method can effectively utilize a small amount of data for training, and by using the improved YOLOv8 neural network for training, it can achieve higher recognition accuracy and greater distance.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A neural network-based open-break identification method, characterized by, The method specifically comprises the following steps: S1: Collecting circuit breaker and bridge breakage pictures, performing data labeling and data enhancement, and making a circuit breaker and bridge breakage dataset; S2: Utilizing an MSCAM module and an LWM to improve a YOLOv8 neural network model, and adding a small target detection head; wherein the MSCAM module represents a multi-scale convolution attention enhancement module, and the LWM represents a wavelet neural network; The MSCAM module is used to replace part of the convolution structure in the YOLOv8 backbone network, and comprises an MSCB block, a CAB block and an SAB block, and through the fusion of multi-scale convolution and multi-dimensional attention mechanism in the feature extraction process, the joint enhancement of spatial features and channel features is realized; wherein the MSCB block represents a multi-scale convolution block, the CAB block represents a channel attention block, and the SAB block represents a spatial attention block; The CAB block extracts channel information through global average pooling and maximum pooling, generates a weight coefficient, and then uses a Sigmoid function to weight the original feature map to enhance the key channel features; the SAB block generates a spatial attention map through a pooling operation to guide the model to focus on the key regions in the feature map and improve the positioning accuracy; and the MSCB block adopts parallel processing of different convolution kernels to fuse multi-scale features to adapt to objects of different sizes; The LWM specifically comprises: using a discrete wavelet transform to decompose the input feature map into four subbands of low-frequency approximation, horizontal high-frequency, vertical high-frequency and diagonal high-frequency, to realize the natural multi-scale separation of image structure and detail information; subsequently, the SAB sub-module independently acts on each subband, learns and applies adaptive weights through a lightweight attention mechanism to enhance key information; finally, the inverse discrete wavelet transform accurately reconstructs the weighted subbands back to the spatial domain to form the enhanced output feature map through fusion; The image processing process of the improved YOLOv8 neural network model comprises the following steps: S21: inputting the picture into the backbone network of the model; each convolution layer uses a 3*3 convolution kernel to slide on the image for convolution operation to extract local features of the image; the stride of the convolution layer is 2, and the feature map size of each convolution layer is halved; the number of channels of the convolution layer gradually increases according to 64, 128, 256, 512 and 1024; the C2f_LWM module first uses a 1*1 convolution to adjust the number of channels of the feature map, and then uses a 3*3 convolution to extract local spatial information; the wavelet transform in the C2f_LWM module realizes multi-scale space-frequency feature fusion, lightweight compression and noise resistance optimization, so that the improved YOLOv8 can extract multi-scale target features in complex scenes; the SPPF module is a fast spatial pyramid pooling layer, and the size of the pooling kernel is 5; the output image will enter the MSCAM module, and the MSCAM module realizes adaptive reinforcement of key features and suppression of redundant information through multi-scale convolution fusion and channel-spatial attention weighting; S22: input the feature map extracted by the backbone network into the neck network for feature fusion, through multiple upsampling and feature fusion, finally input the feature map to the detection head at layers 19, 22, 25 and 28 for detection; the dynamic detection head predicts the boundary box coordinates of the target: 3x3 convolution operation is performed on the fused feature map, the activation function is ReLU function, and then 1x1 convolution is used to adjust the channel dimension to the regression parameters, which are four values: x, y, w and h; wherein x and y represent the horizontal coordinate and vertical coordinate of the center point of the boundary box respectively; w and h represent the width and height of the boundary box respectively, and finally the category and boundary box of the target are output; S3: using the improved YOLOv8 neural network model to perform deep learning on the broken bridge data set, extracting the broken bridge features, and training to obtain the best weight parameters; S4: importing the picture video captured by the vehicle-mounted camera into the improved YOLOv8 neural network model trained, and the model autonomously judges whether the object in the video reaches the threshold, if it does, it means that there is a broken bridge, and the recognition is successful.

2. The neural network-based open break identification method of claim 1, wherein, Step S1 specifically includes the following steps: S11: collecting real shooting pictures of broken bridges on the network; generating reliable broken bridge pictures using a generative adversarial network; and establishing a three-dimensional model to obtain broken bridge pictures; S12: using affine transformation to generate long-distance images of the collected broken bridge pictures, and then performing data labeling; S13: using a data enhancement algorithm to perform data enhancement on the picture data after data labeling; S14: integrating the above pictures and their data labeling files into a broken bridge data set. 3.The neural network-based open circuit breaker identification method of claim 1, wherein, In step S2, the small target detection head is a small target detection branch added on the basis of the original YOLOv8 detection head, which extracts local feature information from a high-resolution feature layer and outputs positioning and classification through deep convolution.

4. The neural network-based open break identification method of claim 1, wherein: In step S2, the improved YOLOv8 neural network model includes a backbone network and a feature enhancement module, the backbone network is composed of 5 convolution layers, 4 C2f_LWM modules, 1 SPPF module and 1 MSCAM module, wherein the 0th, 1st, 3rd, 5th and 7th layers are convolution layers, the 2nd, 4th, 6th and 8th layers are C2f_LWM modules, the 9th layer is an SPPF module, and the 10th layer is an MSCAM module; wherein the C2f_LWM module represents a light weight wave cross-stage partial module, which introduces a light weight wave neural network LWN on the basis of the traditional C2f structure, embeds a discrete wavelet transform and inverse wavelet transform mechanism in the feature extraction process, realizes multi-frequency domain decomposition and reconstruction of the input feature, and effectively separates the low-frequency background information and high-frequency edge detail information; the SPPF module represents a fast spatial pyramid pooling module, which is used for spatial feature aggregation at different scales to enhance the multi-scale feature extraction capability of the model; The neck network comprises 3 up-sampling layers, 6 feature splicing layers, 6 C2f layers and 3 convolution layers, wherein the 11th, 14th and 17th layers are up-sampling layers, the 12th, 15th, 18th, 21st, 24th and 27th layers are feature splicing layers, the 13th, 16th, 19th, 22nd, 25th and 28th layers are C2f layers, and the 20th, 23rd and 26th layers are convolution layers; The detection part comprises four detection heads.

5. The neural network-based open break identification method of claim 1, wherein, Step S3 specifically comprises the following steps: S31: adjust the Epoch, Batch Size and optimizer of the deep learning algorithm, and select the improved YOLOv8 neural network model; S32: write the circuit breaker bridge data set into the training algorithm, use the improved YOLOv8 neural network model for deep learning, and extract the circuit breaker bridge features; S33: after the training is completed, the best weight parameter is obtained, which is written back to the model for detection. 6.The neural network-based open circuit breaker identification method of claim 1, wherein, Step S4 specifically comprises the following steps: S41: load the best weight parameter file into the improved YOLOv8 neural network model; S42: adjust the precision and recall by setting the confidence threshold and the IoU threshold; S43: the picture video captured by the vehicle-mounted camera is converted in format and imported into the detection algorithm; S44: run the detection algorithm, and the model autonomously judges whether the object in the video reaches the threshold; if it does, it means that there is a circuit breaker bridge, and the recognition is successful.

Images