Bridge cable icing detection method based on machine vision with strong generalization capability
Through machine vision detection methods based on strong generalization capabilities, a multi-source multi-scale image training set is constructed using data augmentation and neural style transfer technology, the YOLOv5s network is improved, and the accuracy of bridge cable icing detection is solved and The timeliness issue has achieved real-time monitoring and early warning in harsh environments, improving bridge safety and operational efficiency.
Patent Information
- Application Number
- CN202411180428.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2025-01-03
AI Technical Summary
The existing bridge cable icing detection technology has problems such as poor accuracy, poor timeliness, high cost and difficulty in maintenance, making it difficult to achieve real-time monitoring and early warning in harsh environments.
Using machine vision detection method based on strong generalization capabilities, a multi-source and multi-scale cable icing image training set is constructed through data augmentation technology and neural style transfer technology, the YOLOv5s network is improved, and the Dropout layer, SimAM and ECA-Net modules are added. Combined with spherical network cameras to achieve real-time detection and early warning.
It realizes efficient and accurate bridge cable icing detection in a variety of complex environments, improves the safety and operational efficiency of the bridge, and has the advantages of non-contact, low-cost and fast.
Smart Images
Figure CN119251146A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing and target detection, and in particular to a bridge cable icing detection method based on machine vision with strong generalization capability. Background Art
[0002] At present, the maintenance and safety issues of bridge structures have gradually become prominent, especially in severe weather conditions such as freezing rain and low temperature environments. The problem of bridge cable icing is particularly serious. The cables of suspension bridges and cable-stayed bridges are key load-bearing components. Icing not only increases the load on the cables, but also may cause vibration and fatigue damage. At the same time, the falling of ice during melting poses a major hidden danger to driving safety. Therefore, how to efficiently and real-time detect the icing of bridge cables has become an important technical challenge to ensure the safe operation of bridges.
[0003] The appearance survey and evaluation method, calculation and evaluation method, and load test method used in traditional cable ice detection all have disadvantages such as poor accuracy, poor timeliness, long test time, and high detection costs. The existing cable ice detection technology requires the installation of a large number of sensors or cameras on the bridge, which not only increases the cost of installation and maintenance, but may also be easily damaged in harsh environments and require frequent maintenance. In addition, it performs poorly in real-time monitoring and early warning, making it difficult to detect and deal with ice problems on bridge cables in the first place, affecting the safety and operational efficiency of the bridge. Summary of the invention
[0004] Purpose of the invention: The purpose of the present invention is to provide a bridge cable icing detection method based on machine vision with strong generalization capability, so as to achieve efficient and accurate real-time icing detection of bridge cable structures under a variety of complex environmental conditions and improve the safety of bridges.
[0005] Technical solution: A bridge cable ice detection method based on machine vision with strong generalization capability, comprising the following steps:
[0006] S1, using data enhancement technology and neural style transfer technology, combined with real cable icing images and experimental simulated cable icing images to build a multi-source and multi-scale cable icing image training set;
[0007] S2, build the improved Yolov5s network;
[0008] S3, use the multi-source and multi-scale cable icing image set to train the improved Yolov5s network, obtain the optimal weight file, and verify the detection effect of the model;
[0009] S4, the optimal model is used for the bridge cable ice detection task in real scenarios.
[0010] Furthermore, the modes of cable icing include: crescent, ellipse and needle shape; the data enhancement technology is to use the real images of freezing rain disasters as negative samples in the deep learning training set, combined with simulated real cable icing experimental images, to integrate them into the cable icing image training set; the neural style transfer technology is to use Photoshop software to convert the style of unfrozen cable images into the style of thin ice on the surface, and add it to the cable icing image training set to form a multi-source and multi-scale icing image set.
[0011] Furthermore, in the improved Yolov5s network, a Dropout layer is added to the CSP1_X module in the Bottleneck layer and the CSP2_X and SPPF modules respectively; the SimAM mechanism is added after the CBS module in the Bottleneck layer and before the upsampling module; the 1D convolution operation is used instead of the full connection operation to generate the attention weights between channels, and the ECA-Net module is inserted after the convolution operation; the improved Yolov5s network is as follows:
[0012] Input layer: a linear layer with 10 input features and 64 output features;
[0013] Hidden layer: consists of two linear layers and two Dropout layers. The number of input features of each linear layer is 64 and the number of output features is 64.
[0014] Output layer: A linear layer with 64 input features and 1 output feature.
[0015] Furthermore, in the SimAM mechanism, the input sequence X, where X = {1, 2, ..., n}, each x i is an element in the input sequence. The cosine similarity is used to measure the similarity between the query sequence and the key sequence to determine the importance of each element in the input sequence. The expression is as follows:
[0016]
[0017] Among them, f(x i ,x j ) is a measure element x i and x j A function of the similarity between
[0018] Then, by i Perform weighted summation to obtain the value corresponding to each element x i The weighted vector z i , the calculation formula is as follows:
[0019]
[0020] Among them, g(xj ) is the element x in the input sequence j The representation vector of
[0021] Finally, the weighted vector z i as the output of the model.
[0022] Furthermore, the ECA-Net module uses a band matrix W k To learn channel attention:
[0023]
[0024] ω=σ(C1D k (y))
[0025] Among them, w 1,k Represents the weight of the convolution kernel on different channels. Each row corresponds to the convolution kernel weight of a channel. The number of non-zero element columns k represents the size of the convolution kernel. k () represents a one-dimensional convolution with a convolution kernel size of k, σ() is the activation function, ω represents the output after being processed by the activation function σ(), and y is the feature map.
[0026] Further, in step S3, the specific steps of using the multi-source multi-scale ice image set to train the improved Yolov5s network are as follows:
[0027] S31, setting the image size of the input network and assigning a value;
[0028] S32, create initial model;
[0029] S33, set the batch training number epoch, the initial learning rate lr, and use the adaptive strategy to adjust the learning rate;
[0030] S34, load the pre-trained weights of the YOLOv5s network;
[0031] S35, train the YOLOv5s network model and update the network parameters, retaining the best weight file;
[0032] S36, when the batch training quantity is reached, the optimal model weight file is saved and the model training ends.
[0033] Further, in step S4, the implementation steps for the bridge cable icing detection task in a real scenario are as follows:
[0034] S41, using an image acquisition device to acquire an image of the iced portion of the cable;
[0035] S42, regularly transmitting the images of the cable iced areas collected on site to the local server;
[0036] S43, separating the ice area from other areas using an image segmentation algorithm;
[0037] S44, using the saved improved YOLOv5s network model to detect the location and ice type of the icing object.
[0038] Compared with the prior art, the present invention has the following significant effects:
[0039] 1. By using data augmentation technology and neural style transfer technology, combined with real cable icing images, a multi-source, multi-scale cable icing image training set was constructed. This diversified training set solves the problem of the single traditional data set and improves the robustness and accuracy of the detection model;
[0040] 2. A Dropout layer is added to the CSP module and SPPF module of the Bottleneck layer in the existing Yolov5s network to randomly ignore the output of neurons with a probability of 0.5, forcing the model to learn more general features rather than the details of specific training samples, which significantly improves the performance of the model on new data, that is, improves the generalization ability of the model;
[0041] 3. The SimAM module and ECA-Net module are integrated into the network, which significantly improves the detection capability of bridge cable icing without increasing the complexity and size of the model. The improved model can maintain efficient and accurate detection under a variety of complex environmental conditions, has strong adaptability, and can cope with different climate changes, cable forms and icing patterns.
[0042] 4. The present invention combines a spherical network camera and a machine vision detection network with strong generalization capability to achieve real-time monitoring and early warning of icing on bridge cables. In practical applications, this real-time detection capability helps to quickly discover and respond to icing problems on cables, improves the safety and operational efficiency of bridges, and has the advantages of non-contact, low cost, and rapidity. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is the overall flow chart of the present invention;
[0044] Figure 2 It is a schematic diagram of a multi-source and multi-scale cable icing data set of the present invention;
[0045] Figure 3 This is a local structure diagram of the network after the Dropout layer is introduced into the present invention;
[0046] Figure 4 This is the Yolov5s network structure diagram after the present invention integrates SimAM and ECA-Net;
[0047] Figure 5This is a flow chart of real-time bridge cable icing detection based on the improved Yolov5s of the present invention;
[0048] Figure 6 1 is a diagram of the training results of the improved Yolov5s ice detection network of the present invention, wherein (a) is a schematic diagram of the training set border loss, (b) is a schematic diagram of the training set target loss, (c) is a schematic diagram of the training set classification loss, (d) is a schematic diagram of the accuracy, (e) is a schematic diagram of the recall rate, (f) is a schematic diagram of the validation set border loss, (g) is a schematic diagram of the validation set target loss, (h) is a schematic diagram of the validation set classification loss, (i) is a schematic diagram of the average accuracy with a confidence threshold of 0.5, and (j) is a schematic diagram of the average accuracy with confidence thresholds between 0.5 and 0.95;
[0049] Figure 7 (a) is a comparison chart of the accuracy-confidence of the network training results before improvement, and (b) is a comparison chart of the accuracy-confidence of the network training results after improvement;
[0050] Figure 8 (a) shows that the Yolov5s network did not identify the ice-covered rope before improvement.
[0051] (b) The ice-covered rope is detected with lower confidence before the Yolov5s network is improved.
[0052] (c) The icing morphology of the steel strand that was missed before the Yolov5s network was improved.
[0053] (d) The improved Yolov5s network detects the ice-covered rope with a higher confidence.
[0054] (e) The improved Yolov5s network detects the ice-covered rope with higher confidence.
[0055] (f) The improved Yolov5s network detects all the frozen cables and strands with a higher confidence. DETAILED DESCRIPTION
[0056] The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0057] The vision-based computing method is easier to meet the needs of real-time measurement due to its portability. Compared with traditional measurement methods, such as physical measuring instruments, the vision-based measurement method has the characteristics of non-contact, which is more flexible and convenient. Specifically, machine vision can automatically identify and detect ice areas through image processing and analysis technology. For example, an image segmentation algorithm can be used to separate ice areas from other areas; or a target detection algorithm can be used to detect information such as the location and size of iced objects. In addition, machine vision can also be combined with hardware devices such as sensors to monitor and warn of ice conditions in real time, helping people take timely measures to avoid traffic accidents and production accidents. In computer vision, deep learning models, especially target detection models, play a key role in object recognition and positioning. In recent years, the YouOnly Look Once (YOLO) series of models have received widespread attention due to their efficient and real-time target detection performance. YOLO was proposed by Joseph Redmon et al. in 2016. It is an end-to-end neural network model that focuses on real-time target detection and has achieved remarkable results in many fields.
[0058] The present invention uses the improved Yolov5s network in combination with a large training cable icing image data set to monitor and analyze the icing situation in real time, thereby achieving non-contact rapid icing monitoring and ensuring traffic safety and production efficiency.
[0059] like Figure 1 As shown in the figure, the specific implementation steps of a bridge cable ice detection method based on strong generalization ability machine vision are as follows:
[0060] Step 1: Create a multi-source and multi-scale cable icing dataset
[0061] Data augmentation technology and neural style transfer technology are used to construct a multi-source and multi-scale cable icing image training set in combination with real cable icing images.
[0062] Different modes of cable icing are classified and identified, and the main cable icing forms include crescent, ellipse (D-shaped), needle, etc. In this embodiment, the detection types of bridge cable icing are divided into three categories through visual algorithms, namely: cable surface ice (crescent), cable thick ice (ellipse) and ice ridges (needle).
[0063] Data enhancement technology uses real images of freezing rain disasters as negative samples in the deep learning training set. The negative samples in the training set enable the detection model to effectively distinguish between targets and backgrounds, thereby improving the model's generalization ability for different scenes. Combined with simulated real cable icing experimental images, a training set is formed that incorporates negative samples such as non-cable main body icing images.
[0064] Further preferably, in this embodiment, the icing image of the non-cable body is fused as a negative sample for deep learning network training. In the target detection task, a negative sample refers to an image area that does not contain any target or region of interest (ROI). In YOLOv5, negative samples are usually used to train the target detection model. The principle is: 1) YOLOv5 can increase the diversity and complexity of the data set by introducing negative samples. 2) During the training process, introducing more negative samples can help the model better understand the background information, thereby improving its generalization ability for real scenes. 3) The introduction of negative samples can reduce the risk of over-learning and overfitting of the model for specific targets. If the training data only contains positive samples (image areas containing targets), the model may focus too much on the detailed features of the target and ignore the background information. 4) Negative sample training enables the model to better distinguish between targets and non-targets, which helps to reduce the false detection rate, that is, reduce the situation where background or non-target objects are mistakenly identified as targets. 5) YOLOv5 improves the matching mechanism of positive and negative samples, allowing one true label box to match multiple prediction boxes. This can increase the number of positive samples while maintaining a balance between positive and negative samples, thereby improving the generalization ability of the model.
[0065] like Figure 2 As shown in the figure, the neural style transfer method is used to create a data set to obtain a style transfer image set. The principle is as follows: Photoshop (Ps for short) is used to simulate the real cable icing data set, which is one of the methods of neural style transfer. Since Ps cannot change the size of the cable image itself, it is more suitable for simulating the icing form of thin ice on the cable surface. The implementation steps are as follows:
[0066] Step A1, capture the cables from different perspectives in the video and import them into Ps.
[0067] In step A2, the lasso is used to select the main body of the lasso and isolate it from the background to avoid misidentification during training.
[0068] In step A3, add two layers and apply the filters of cable-distorted glass and plastic packaging respectively to create the icy special effect.
[0069] Step A4, select "Filter", "Blur", "Gaussian Blur" in the layer panel, select "Ice Layer" again, select "Blending Mode" in the layer panel, set the blending mode to "Overlay" or "Soft Light", adjust the opacity and fill value to make the icing effect more realistic.
[0070] Step A5, finally adjust the image's color levels, grayscale and other attributes to make it present the effect of ice reflection and complete the rendering.
[0071] Furthermore, more reliable icing data sets are collected by simulating the experimental method of cable icing, namely, the simulated cable icing atlas. In the laboratory, the real cable icing state is simulated, cables of different sizes are selected, and different icing forms of real cables (with sheaths) are simulated, including: "frost and ice on the surface of the cable", "partially melted water droplets forming "local thick ice on the cable", and "ice cones formed on the cable". The principle is as follows: the simulated training set contains a variety of different scenes, lighting conditions, target size and posture changes, and some noise or interference is added to the image to simulate the situation in the real world, which helps to improve the robustness of the model. The purpose of the simulated cable icing experiment is to simulate the "multi-type" conditions caused by icing, including: the shedding of thick ice caused by the hanging of ice on the cable, the falling of ice cones, and the change of the cross-sectional shape of the cable, so as to verify the sensitivity of the icing monitoring model to the three cable icing forms.
[0072] Finally, the four types of data sets mentioned above, namely negative samples, style transfer image sets, simulated cable icing images and real cable icing images, were merged to produce a multi-source and multi-scale icing image set that incorporates negative samples such as non-cable main body icing images.
[0073] Step 2: Build the improved Yolov5s network model
[0074] In order to improve the generalization ability and detection accuracy of the basic Yolov5s network model, the improved Yolov5s network model in the present invention integrates SimAM and ECA-Net modules, and introduces the Dropout layer into the network.
[0075] like Figure 3 As shown, the present invention adopts the method of adding a Dropout layer to the CSP1_X and CSP2_X modules and the SPPF module in Bottleneck respectively. The Dropout parameter uses 0.5 to represent the discard ratio. The improved Yolov5s network is as follows:
[0076] Input layer: a linear layer with 10 input features and 64 output features.
[0077] Hidden layer: It consists of two linear layers and two Dropout layers. The number of input features of each linear layer is 64, and the number of output features is 64. The probability of the Dropout layer is 0.5.
[0078] Output layer: a linear layer with an input feature number of 64 and an output feature number of 1. In the present invention, the output of the Yolov5s model includes the category, location and confidence of the icy object.
[0079] The SimAM mechanism is added after the CBS module and before the upsampling module in the Bottleneck layer of the Yolov5s network, such as Figure 4 As shown in the figure, the network's attention to features of different scales is strengthened. 1D convolution operations are used instead of full connection operations to generate attention weights between channels. The ECA-Net module is inserted after the convolution operation. Similarly, it can also be inserted after the CBS module in the Bottleneck layer and before the upsampling module. Finally, the SimAM module and the ECA-Net module are integrated, and channels are added before feature fusion. By paying attention to both spatial and channel information at the same time, the model can learn richer and more robust feature representations.
[0080] The Dropout layer is introduced on the basis of the existing Yolov5s network to enhance the generalization ability of the network. The principle is as follows: Dropout is a simple and effective regularization technique for achieving better generalization of deep neural networks. The original dropout and various similar techniques are widely used to train neural networks for various tasks due to their simplicity and efficiency. The use of Dropout prevents the training network from overfitting the training data by randomly dropping, that is, "discarding" 50% of the neurons in each training iteration. Therefore, neurons cannot depend on each other, and the trained network has better generalization effect. During the inference process, neurons are not discarded and all information is retained; instead, each output value is multiplied by 0.5 to make the average consistent with the training time. The network used for inference can be regarded as a collection of many sub-networks randomly created during the training process.
[0081] Further preferably, on the basis of improving the Yolov5s network, SimAM and ECA-Net are integrated to improve the detection accuracy of the model and enhance the generalization ability. The principle is as follows:
[0082] In the SimAM mechanism, the input sequence X, where X = {1, 2, ..., n}, each x i is an element in the input sequence. SimAM uses cosine similarity to measure the similarity between the query sequence and the key sequence to determine the importance of each element in the input sequence. Using the dot product attention mechanism, it can be expressed as follows:
[0083]
[0084] Among them, f(x i ,x j ) is a measure element x i and x j The function of the similarity between .
[0085] Then, by i Perform weighted summation to obtain the value corresponding to each element x i The weighted vector z i , and its calculation formula is as follows:
[0086]
[0087] Among them, g(x j ) is the element x in the input sequence j The representation vector of .
[0088] Finally, the weighted vector z i as the output of the model.
[0089] ECA-Net (Efficient Channel Attention Module) is an effective channel attention (ECA) module for deep CNN. The principle is as follows: This module avoids dimensionality reduction and captures cross-channel interactions in an effective way. Without dimensionality reduction, by considering each channel and its k neighbors, its ECA captures the local cross-channel interactions. In ECA-Net, a band matrix W is used. k To learn channel attention:
[0090]
[0091] ω=σ(C1D k (y)) (4)
[0092] Among them, w 1,1 、w 1,k etc. represent the weights of the convolution kernel on different channels. Each row corresponds to the convolution kernel weight of a channel. The number of non-zero element columns k represents the size of the convolution kernel. k () represents a one-dimensional convolution with a convolution kernel size of k, σ() is the activation function, ω represents the output after being processed by the activation function σ(), and y is the feature map.
[0093] After the ECA module uses GAP to aggregate convolution features without dimensionality reduction, it first adaptively determines the kernel size k, then performs one-dimensional convolution, and then uses the Sigmoid function to learn channel attention. The specific approach is as follows: first apply SimAM to learn spatial attention, and then use ECA-Net to refine features based on channel attention. The two focus on different aspects. SimAM focuses on enhancing the model's perception of spatial features. By allocating attention to different areas of the image, the model pays more attention to areas that are useful for the final task (such as target detection); ECA-Net effectively enhances the model's attention to important feature channels and suppresses unimportant channels by learning the dependencies between channels, thereby improving the expressiveness of features.
[0094] Step 3: Network training
[0095] Use the multi-source and multi-scale ice image set to train the improved Yolov5s network, obtain the optimal weight file, and verify the detection effect of the model; the specific steps are as follows:
[0096] Step C1, define the width w and height h of the input network image, and assign w=640, h=640;
[0097] Step C2, parse all XML files in the training and validation sets to obtain train.txt and val.txt files;
[0098] Step C3, use an adaptive strategy to adjust the learning rate, and set the batch size to 16; set SGD (Stochastic Gradient Descent) as the optimization function of the network, the number of batch training epochs is 200, and the initial learning rate lr is 0.001;
[0099] Step C4, load the pre-trained weights of the improved YOLOv5s network; define the initial epoch as 0;
[0100] Step C5, perform training to update the network model parameters, update the best weight file, and set epoch = epoch + 1;
[0101] Step C6, when epoch = 200, save the model weight last, which means that the training of the improved Yolov5s network model is completed.
[0102] Step 4: Real-time detection
[0103] The trained optimal model is used for bridge cable ice detection tasks in real scenarios, and combined with spherical network cameras to achieve real-time detection and early warning of bridge cable ice. The specific implementation steps are as follows:
[0104] Step D1, using a spherical network camera image acquisition device to acquire images of the frozen portion of the cable.
[0105] Step D2, regularly transmitting the images of the cable iced areas collected on site to the local server.
[0106] Step D3, using an image segmentation algorithm to separate the ice area from other areas.
[0107] Step D4, using the improved target detection algorithm to detect the position and ice type of the frozen object.
[0108] like Figure 5 As shown in the figure, the specific steps for using the trained optimal model for bridge cable ice detection tasks in real scenarios are as follows:
[0109] (41) Deploy YOLOv5s network
[0110] The first step is to download the trained YOLOv5s model and related weight files to the local server. Then, configure the operating environment on the local server, including but not limited to installing necessary libraries such as Python, PyTorch, and CUDA. Subsequently, set up an API service to call the model for image detection through a network interface.
[0111] (42) Camera configuration
[0112] First, the on-site monitoring camera is set to capture real-time images of the cables at a rate of at least one image per hour. Then, the processor in the camera encodes the captured raw image data into a format that can be properly transmitted and decoded. In order to ensure that the cable icing monitoring camera can access the local server and send image data, FTP (File Transfer Protocol) is used to send image data to the server, and finally the on-site cable icing images are transmitted back to the local server, so as to achieve real-time monitoring of the actual bridge on-site cables.
[0113] (43)Integration and testing
[0114] Run and test the improved YOLOv5s model on the local server, and use some sample images to perform basic tests on the model to verify whether the cable ice detection model can run normally and return correct results. This process can ensure that the model can correctly receive and process image data and return detection results. Then, use an FTP client or command line tool to upload the image to the selected server directory, test whether the data transmission process from the camera to the server is normal, and verify whether the returned detection results meet expectations.
[0115] The following uses an actual bridge cable ice detection case to illustrate the implementation steps of the proposed bridge cable ice detection method based on strong generalization ability machine vision:
[0116] In order to meet the computing needs, a local server equipped with NVIDIA RTX 3060 GPU and 50GB memory capacity was used. The learning rate was set to 5×10 -4, 200 epochs were trained before the model was tested, and the batch size of the training was set to 2. The model was trained on a computer with a CPU of i9-12900H, a GPU of RTX3060, and 16GRAM. The framework used for training was Pytorch, and the default initialization method of Pytorch was used for the model parameter initialization during training. Labelimg was used for image segmentation and labeling in the Conda environment. Subsequently, the YOLOv5s model for ice detection and classification was trained, so that the ice part of the cable can be quickly located and the ice category in the image can be automatically identified. Establishing a data set is the basis for applying deep learning methods. The database images established are derived from actual engineering sites and laboratory simulation scenes. After repeated annotation and proofreading, the final exported data set contains 500 cable ice images. Finally, the annotated ice sampling map is sent to the network for training.
[0117] Through the model training results, during the training process, the loss function gradually decreased, while the accuracy continued to improve, showing the model's excellent ability in learning data features. Through the evaluation of test data, the model has high detection accuracy in various categories of targets. It can not only accurately identify targets, but also has excellent positioning accuracy, demonstrating excellent target detection capabilities. In addition, through visual analysis, the model has shown robustness and generalization capabilities in various scenarios. It can accurately detect targets of different sizes, postures and lighting conditions, and has strong adaptability and universality. Figure 6 (a)-(j) in the figure are the training results of 200 epochs of the improved ice detection network with strong generalization ability. Figure 7 (a) and (b) in the figure are the accuracy-confidence graphs of the Yolov5s detection network before and after the improvement. In the comparison of the graphs before and after the improvement, all categories achieved an accuracy of 1.00 at the confidence levels of 0.901 and 0.882, which means that the model's predictions for all categories at this confidence threshold are accurate. And compared to the confidence threshold of 0.901, the improved model can achieve perfect accuracy at a slightly lower confidence threshold (0.882), which shows that its performance on detection tasks is better than the existing YOLOv5s model.
[0118] Table 1. Yolov5s training and validation loss values before and after improvement
[0119]
[0120] from Figure 6 , Figure 7From the curve comparison and the data analysis table in Table 1, it can be seen that high accuracy can still be maintained at a lower confidence threshold, indicating that the model has high sensitivity in practical applications and can detect potential icing problems in a timely manner. This high sensitivity is very important for early warning and risk management in industrial measurement.
[0121] The comparison between the bridge cable icing detection results of the present invention and the traditional visual detection results is as follows: Figure 8 As shown in (a)-(f) in the figure, it can be concluded that the present invention can more effectively perform safety assessment and maintenance management on bridges, improve bridge detection efficiency, and is a very potential new method for detecting cable icing.
[0122] The technical means disclosed in the present invention are not limited to the technical means disclosed in the above-mentioned embodiments, but also include technical solutions composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.
Claims
1. A bridge cable icing detection method based on machine vision with strong generalization capability, characterized in that: The steps include: S1, using data enhancement technology and neural style transfer technology, combined with real cable icing images and experimental simulated cable icing images to build a multi-source and multi-scale cable icing image training set; S2, build the improved Yolov5s network; S3, use the multi-source and multi-scale cable icing image set to train the improved Yolov5s network, obtain the optimal weight file, and verify the detection effect of the model; S4, the optimal model is used for the bridge cable ice detection task in real scenarios.
2. The bridge cable icing detection method based on machine vision with strong generalization capability according to claim 1 is characterized in that: The modes of cable icing include: crescent, ellipse and needle; the data enhancement technology is to use the real images of freezing rain disasters as negative samples in the deep learning training set, combined with simulated real cable icing experimental images, to integrate them into the cable icing image training set; the neural style transfer technology is to use Photoshop software to convert the style of unfrozen cable images into the style of thin ice on the surface, and add it to the cable icing image training set to form a multi-source and multi-scale icing image set.
3. The bridge cable icing detection method based on machine vision with strong generalization capability according to claim 1 is characterized in that: In the improved Yolov5s network, a Dropout layer is added to the CSP1_X module in the Bottleneck layer and the CSP2_X and SPPF modules respectively; the SimAM mechanism is added after the CBS module in the Bottleneck layer and before the upsampling module; the 1D convolution operation is used instead of the full connection operation to generate the attention weights between channels, and the ECA-Net module is inserted after the convolution operation; the improved Yolov5s network is as follows: Input layer: a linear layer with 10 input features and 64 output features; Hidden layer: consists of two linear layers and two Dropout layers. The number of input features of each linear layer is 64 and the number of output features is 64. Output layer: A linear layer with 64 input features and 1 output feature.
4. The bridge cable icing detection method based on machine vision with strong generalization capability according to claim 3 is characterized in that: In the SimAM mechanism, the input sequence X, where X = {1, 2, ..., n}, each x i is an element in the input sequence. The cosine similarity is used to measure the similarity between the query sequence and the key sequence to determine the importance of each element in the input sequence. The expression is as follows: Among them, f(x i ,x j ) is a measure element x i and x j A function of the similarity between Then, by i Perform weighted summation to obtain the value corresponding to each element x i The weighted vector z i , the calculation formula is as follows: Among them, g(x j ) is the element x in the input sequence j The representation vector of Finally, the weighted vector z i as the output of the model.
5. The bridge cable icing detection method based on machine vision with strong generalization capability according to claim 3 is characterized in that: The ECA-Net module uses a band matrix W k To learn channel attention: ω=σ(C1D k (y)) Among them, w 1,k Represents the weight of the convolution kernel on different channels. Each row corresponds to the convolution kernel weight of a channel. The number of non-zero element columns k represents the size of the convolution kernel. k () represents a one-dimensional convolution with a convolution kernel size of k, σ() is the activation function, ω represents the output after being processed by the activation function σ(), and y is the feature map.
6. The bridge cable icing detection method based on machine vision with strong generalization capability according to claim 1 is characterized in that: In step S3, the specific steps of using the multi-source multi-scale ice image set to train the improved Yolov5s network are as follows: S31, setting the image size of the input network and assigning a value; S32, create initial model; S33, set the batch training number epoch, the initial learning rate lr, and use the adaptive strategy to adjust the learning rate; S34, load the pre-trained weights of the YOLOv5s network; S35, train the YOLOv5s network model and update the network parameters, retaining the best weight file; S36, when the batch training quantity is reached, the optimal model weight file is saved and the model training ends.
7. The bridge cable icing detection method based on machine vision with strong generalization capability according to claim 1 is characterized in that: In step S4, the implementation steps for the bridge cable ice detection task in a real scenario are as follows: S41, using an image acquisition device to acquire an image of the iced portion of the cable; S42, regularly transmitting the images of the cable iced areas collected on site to the local server; S43, separating the ice area from other areas using an image segmentation algorithm; S44, using the saved improved YOLOv5s network model to detect the location and ice type of the icing object.