Concrete telegraph pole hidden danger identification method based on deep learning

Abstract

The invention discloses a concrete telegraph pole hidden danger identification method based on deep learning, and the method comprises the steps: constructing a comprehensive feature vector through fusing ultrasonic signal features and infrared thermal image temperature field features, carrying out the modeling of the ultrasonic signal features and the infrared thermal image temperature field features in combination with a heat conduction theory, and effectively improving the recognition capability of the internal and surface defects of a concrete telegraph pole. Meanwhile, based on the Mask R-CNN model, the particle swarm optimization algorithm is adopted to dynamically adjust hyper-parameter configuration, so that the model training efficiency is improved, and the detection precision is also remarkably enhanced. Besides, by introducing normalization processing and multi-channel expansion methods of feature vectors, the data processing flow is simplified, and the consistency of input data is ensured, so that the reasoning efficiency and performance of the model are further improved.

Description

Technical Field

[0001] The present invention belongs to the field of hidden danger identification of electric poles, and specifically, relates to a method for identifying hidden dangers of concrete electric poles based on deep learning. Background Art

[0002] Concrete poles are an important part of the power transmission system, and their structural safety directly affects the stability and reliability of power supply. However, as the service life increases, concrete poles often have hidden dangers such as cracks, rust, and peeling. In severe cases, they may even cause the poles to collapse, causing major economic losses and safety accidents. Therefore, how to efficiently and accurately identify the potential hazards of concrete poles has become an urgent problem to be solved.

[0003] Current deep learning detection models (such as Faster R-CNN) are mainly used for single-modal data processing, which makes it difficult to fully utilize the complementary advantages of multi-source data. In the detection of hidden dangers in concrete poles, single-modal models are difficult to take into account both internal and surface hidden dangers at the same time.

[0004] Infrared thermal imaging detection captures the temperature field distribution on the concrete surface and infers structural defects. However, this method is highly dependent on ambient temperature and detection conditions, and is difficult to be applied alone to the identification of hidden dangers in complex structures, especially internal defects.

[0005] Traditional ultrasonic testing methods use signal reflection to identify internal defects in concrete, but they rely on fixed statistical features and cannot deeply mine information about hidden defects in complex structures. In addition, single ultrasonic testing is easily interfered by external noise, and the accuracy and stability of the test results are difficult to guarantee.

[0006] In view of this, the present invention is proposed. Summary of the invention

[0007] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a method for identifying hidden dangers of concrete utility poles based on deep learning, thereby solving the problems raised in the above-mentioned background technology.

[0008] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is:

[0009] A method for identifying hidden dangers of concrete electric poles based on deep learning, comprising the following steps:

[0010] The ultrasonic sensor collects the reflected signal inside the concrete pole, extracts the signal features, calculates the corresponding statistical features, and generates the ultrasonic signal feature vector;

[0011] The temperature field data on the surface of the utility pole is collected through infrared thermal imaging sensors, and the temperature field changes are modeled based on the heat conduction equation to form corresponding temperature field characteristics;

[0012] The characteristics of the ultrasonic signal are fused with the temperature change data of the infrared thermal image to obtain a comprehensive characteristic phasor;

[0013] Based on the theory of heat conduction, the relationship between ultrasonic signals and temperature fields is modeled by combining their mutual influence, thus obtaining the combined features of fused ultrasonic and infrared thermal images.

[0014] Establish and train the MaskR-CNN model, and use the particle swarm optimization algorithm to optimize the training process of the MaskR-CNN model, integrate the features of ultrasound and infrared thermal imaging, and train the model through the optimization function;

[0015] The optimized Mask R-CNN model is used to perform target detection and defect segmentation on concrete poles. By combining the changes in the temperature field and the characteristics of ultrasonic signals, hidden dangers inside and on the surface of the poles are identified, and the defect location and segmentation results are output.

[0016] According to a method for identifying hidden dangers of concrete utility poles based on deep learning according to claim 1, it is characterized in that the statistical features include a mean value μ, a standard deviation σ, a skewness S, a kurtosis K, an energy proportion β, and a mean value μ, wherein the mean value μ, the standard deviation σ, the skewness S, the kurtosis K, the energy proportion β, and the mean value μ are fused to generate an ultrasonic signal feature vector U.

[0017] According to a method for identifying hidden dangers of concrete utility poles based on deep learning according to claim 1, it is characterized in that after obtaining the comprehensive feature vector, a Z-score standardization method needs to be used to eliminate the dimensional differences between feature data, and its expression is: Among them, μ combined and σ cpmbined are the mean and standard deviation of the feature vector respectively, and Z is the standardized feature data.

[0018] Based on the theory of heat conduction, combined with the mutual influence of ultrasonic signals and temperature fields, the steps for modeling the relationship between the two using the following equations are:

[0019] The temperature field model inside the concrete pole is established according to the heat conduction equation, and its expression is: Where: ρ is the density of concrete, C p is the specific heat capacity of concrete, T is the temperature field, t is the time, k is the thermal conductivity of concrete, and Q is the heat source term;

[0020] The propagation process of ultrasonic signals will be affected by the temperature of the medium. Assuming that the propagation speed v of ultrasonic waves in concrete is affected by the temperature T, it can be expressed by the formula v(T)=v0(1+α(T-T0)), where v(T) is the wave speed at temperature T (unit: m / s), v0 is the wave speed at reference temperature T0, and α is the influence coefficient of temperature on the wave speed;

[0021] Moreover, the interaction between ultrasonic wave velocity v(T) and temperature field T is expressed by the coupling of temperature gradient and wave velocity change. The coupling equation is expressed as follows: Among them, U reflects the change of ultrasonic signal; is the gradient of the temperature field, which indicates the change of temperature in space; is the heat flux, which indicates the flow of heat in the concrete; is the heat accumulation term, describing the change of temperature with time.

[0022] According to the method for identifying hidden dangers of concrete electric poles based on deep learning in claim 1, it is characterized in that after obtaining the fused comprehensive feature F fusion After that, the integrated feature F fusion Transformed into a two-dimensional matrix representation, where each element represents the value of the fusion feature at the spatial position, the matrix size is H×W, where H is the height and W is the width. Then, the fusion feature matrix F fusion For visualization, the matrix values ​​are mapped to pixel intensity values. Subsequently, the eigenvalues ​​are normalized to a range of [0,255] in order to be converted into a standard image format. The expression is: Among them, F′ fusion is the normalized feature matrix;

[0023] According to the input format requirements, the two-dimensional feature matrix is ​​expanded to a multi-channel image. If MaskR-CNN requires RGB format, the single-channel feature matrix F′ is fusion Copy it into three channels, the expression is: FRGB = Stack (F' fusion , F′ fusion , F′ fusion ), where the generated F RGB is a three-channel image of shape H×W×3;

[0024] The feature image F RGB The size is adjusted to the standard size of 512×512. Finally, multiple converted images are organized into batches to form the standard Mask R-CNN input format with the shape of: Batch Shape = (N, H target , W target , 3), where N is the batch size.

[0025] According to a method for identifying hidden dangers of concrete utility poles based on deep learning according to claim 1, it is characterized in that the steps of establishing the MaskR-CNN model are:

[0026] Collect and organize multimodal datasets containing ultrasonic data and infrared thermal imaging data. Then, annotate the location and category of the targets in the dataset, generate corresponding segmentation masks and bounding boxes, use ResNet as the backbone network, and combine RegionProposalNetwork to generate candidate regions.

[0027] According to a method for identifying hidden dangers of concrete utility poles based on deep learning according to claim 1, it is characterized in that when training the MaskR-CNN model, a particle swarm optimization algorithm is used to optimize the training process of the MaskR-CNN model, and the steps of integrating the features of ultrasonic and infrared thermal images and training the model through the optimization function are as follows:

[0028] Initialize a particle swarm, where each particle represents a set of hyperparameter configurations of the Mask R-CNN model, the position of the particle represents the hyperparameter, and the speed of the particle controls the adjustment direction and amplitude of the hyperparameter;

[0029] The total loss function is calculated through the MaskR-CNN model and used as the fitness function of the particle swarm optimization. The loss function includes the classification loss L classification , regression loss L regression , mask segmentation loss L mask , and the velocity update term of particle swarm optimization, which is expressed as: Among them, λ is the hyperparameter of the particle swarm optimization process, which controls the impact of the particle swarm on network training, V i (t) is the velocity of particle ii at time t;

[0030] In each iteration of particle swarm optimization, the Mask R-CNN model is trained using the hyperparameter configuration represented by the current particle, the classification, regression, and mask segmentation accuracy are calculated, the fitness value is obtained, and the position and velocity of the particle are updated to optimize the training process;

[0031] According to the input feature map B fusion And particle swarm optimization algorithm, the final loss function expression is: L final =L Mask R-CNN (B fusion )+α·PSO(B fusion , θ), where α is the weight factor that controls the intensity of particle swarm optimization and θ is the hyperparameter of the MaskR-CNN model.

[0032] The method for identifying hidden dangers of concrete electric poles based on deep learning according to claim 1 is characterized in that, at each iteration of particle swarm optimization, the position and velocity of the particles are updated by the following update rule, and the expression is: V i (t+1)=ω·V i (t)+c1·r1·(P best -X i (t))+c2·r2·(G best -X i (t)), where V i (t) and X i (t) are the velocity and position of particle i at time t, ω is the inertia weight, c1, c2 are learning factors, r1, r2r are random numbers, P best is the historical optimal position of the particle, G best is the global optimal position.

[0033] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art. Of course, any product implementing the present invention does not necessarily need to achieve all the advantages described below at the same time:

[0034] The present invention constructs a comprehensive feature vector by fusing the ultrasonic signal features with the infrared thermal image temperature field features, and models the two in combination with the heat conduction theory, which effectively improves the ability to identify internal and surface defects of concrete poles. At the same time, based on the Mask R-CNN model, the particle swarm optimization algorithm is used to dynamically adjust the hyperparameter configuration, which not only improves the efficiency of model training, but also significantly enhances the detection accuracy. In addition, by introducing the normalization processing and multi-channel expansion method of the feature vector, the data processing process is simplified, the consistency of the input data is ensured, and the reasoning efficiency and performance of the model are further improved.

[0035] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings described below are only some embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0037] In the figure:

[0038] Figure 1 This is a flow chart of the method for identifying hidden dangers of concrete utility poles.

[0039] It should be noted that these drawings and textual descriptions are not intended to limit the conceptual scope of the present invention in any way, but are intended to illustrate the concept of the present invention for those skilled in the art by referring to the embodiments. DETAILED DESCRIPTION

[0040] The present invention will now be described in further detail with reference to the accompanying drawings.

[0041] See also Figure 1 As shown, in this embodiment, a method for identifying hidden dangers of concrete electric poles based on deep learning is provided, comprising the following steps:

[0042] The ultrasonic sensor collects the reflected signal inside the concrete pole, extracts the signal characteristics, including wave speed, propagation time, frequency, amplitude, and calculates the corresponding statistical characteristics to generate the ultrasonic signal feature vector;

[0043] The temperature field data on the surface of the utility pole is collected through infrared thermal imaging sensors, and the temperature field changes are modeled based on the heat conduction equation to form corresponding temperature field characteristics;

[0044] The characteristics of the ultrasonic signal are fused with the temperature change data of the infrared thermal image to obtain a comprehensive feature vector; by fusing the statistical characteristics of the ultrasonic signal with the change information of the temperature field of the infrared thermal image, a comprehensive feature vector is established, and the two are modeled based on the heat conduction theory. This combination of multimodal data can not only accurately capture the internal hidden dangers of concrete poles (such as cracks, voids, etc.), but also indirectly infer structural defects through surface temperature anomalies. Experimental results show that the accuracy of the present invention reaches 95%, which is significantly higher than the 65% of traditional manual detection and 78% of single ultrasonic detection. Secondly, through a modeling method based on heat conduction theory, the dynamic characteristics of the ultrasonic signal are associated with the changes in the temperature field, which solves the problem that a single detection method is difficult to fully identify internal hidden dangers and surface defects. The recall rate of the present invention is as high as 93%, especially in the detection of complex scenes and hidden defects (such as deep cracks or edge peeling), it shows excellent capabilities.

[0045] Based on the theory of heat conduction, the relationship between ultrasonic signals and temperature fields is modeled by combining their mutual influence, thus obtaining the combined features of fused ultrasonic and infrared thermal images.

[0046] Establish and train the MaskR-CNN model, and use the particle swarm optimization algorithm to optimize the training process of the MaskR-CNN model, integrate the features of ultrasound and infrared thermal imaging, and train the model through the optimization function;

[0047] Through the optimized Mask R-CNN model, the concrete poles are detected and defect segmented. The hidden dangers inside or on the surface of the poles are identified by combining the changes in the temperature field and the characteristics of the ultrasonic signal, and the defect location and segmentation results are output. By fusing the ultrasonic signal characteristics with the infrared thermal image temperature field characteristics, a comprehensive feature phasor is constructed, and the two are modeled in combination with the heat conduction theory, which effectively improves the ability to identify defects inside and on the surface of concrete poles. At the same time, based on the Mask R-CNN model, the particle swarm optimization algorithm is used to dynamically adjust the hyperparameter configuration, which not only improves the efficiency of model training, but also significantly enhances the detection accuracy. In addition, by introducing the normalization processing of the feature vector and the multi-channel expansion method, the data processing process is simplified, the consistency of the input data is ensured, and the reasoning efficiency and performance of the model are further improved.

[0048] In this embodiment, the statistical features include the average value μ, the standard deviation σ, the skewness S, the kurtosis K, the energy proportion β, the average value μ, wherein the average value μ, the standard deviation σ, the skewness S, the kurtosis K, the energy proportion β, and the average value μ are fused to generate the ultrasonic signal feature vector U, and the average value μ is used to represent the mean value of the sub-band energy, and the calculation formula is: The standard deviation σ is used to measure the fluctuation of sub-band energy and is calculated as follows: The skewness S represents the skewness of the sub-band energy distribution and is calculated as: Kurtosis K, which indicates the sharpness of the sub-band energy distribution, is calculated as: Energy proportion β represents the proportion of each sub-band energy to the total energy, and the calculation formula is: Wherein, μ is the average value of sub-band energy, which indicates the mean value of all sub-band energies and reflects the overall energy distribution of the signal; j is the number of wavelet packet decomposition layers, each wavelet packet decomposition layer represents a different frequency band; i is the wavelet packet node number, which is used to identify each sub-band; E iis the energy of the ith sub-band, indicating the energy value of the ith band extracted from the signal, 2j is the total number of sub-bands, which is determined by the decomposition level j, indicating the number of all sub-bands after decomposition, σ is the standard deviation of the sub-band energy, which measures the discreteness or fluctuation of the energy. The larger the standard deviation, the more dispersed and unstable the energy distribution. S is the skewness of the sub-band energy, indicating the skewness of the energy distribution. A skewness of zero indicates a symmetrical distribution, a skewness greater than zero indicates a right skew (positive skewness), and a skewness less than zero indicates a left skew (negative skewness). K is the sub-band energy. The kurtosis of the frequency band energy indicates the sharpness of the energy distribution. The larger the kurtosis value, the more concentrated the signal distribution is near the mean and the more extreme values ​​there are. If the kurtosis is 3, it indicates a normal distribution. n is the exponent of the kurtosis, which is usually 4, indicating high-order sharpness. β is the energy proportion of each sub-band, which is used to indicate the ratio of the energy of all sub-bands to the energy of the first sub-band. The larger this value is, the greater the energy proportion of other bands. E1 is the energy of the first sub-band. As a reference value, it is usually taken as the energy of the first band after decomposition.

[0049] In this embodiment, after obtaining the comprehensive feature vector, a Z-score standardization method needs to be used to eliminate the dimensional differences between the feature data, and its expression is: Among them, μ combined and σ combined are the mean and standard deviation of the feature vector, respectively, and Z is the standardized feature data. The main benefit of using the Z-score standardization method is that it can effectively eliminate the dimensional differences between different feature data in the comprehensive feature vector, making the distribution of feature data more uniform and avoiding interference with model training and prediction due to differences in numerical ranges or units. By normalizing each feature value according to its mean and standard deviation, the Z-score method can map the feature value to a distribution with zero mean and unit variance, thereby enhancing the comparability between features and improving the sensitivity and adaptability of the model to various features.

[0050] Based on the theory of heat conduction, combined with the mutual influence of ultrasonic signals and temperature fields, the steps for modeling the relationship between the two using the following equations are:

[0051] The temperature field model inside the concrete pole is established according to the heat conduction equation, and its expression is: Where: ρ is the density of concrete (unit: kg / m 3 ), C p is the specific heat capacity of concrete (unit: J / kgK), T is the temperature field (unit: K), t is the time (unit: s), k is the thermal conductivity of concrete (unit: W / mK), Q is the heat source term (unit: W / m 3 ), which can be ignored if there is no external heat source;

[0052] The propagation process of ultrasonic signals will be affected by the temperature of the medium, and the ultrasonic wave velocity is usually related to the change of the temperature field. Assuming that the propagation velocity v of ultrasonic waves in concrete is affected by the temperature T, it can be expressed by the following formula: v(T) = v0(1+α(T-T0)), where v(T) is the wave velocity at temperature T (unit: m / s), v0 is the wave velocity at reference temperature T0, and α is the coefficient of influence of temperature on the wave velocity;

[0053] Moreover, the interaction between ultrasonic wave velocity v(T) and temperature field T is expressed by the coupling of temperature gradient and wave velocity change. The coupling equation is expressed as follows: Among them, U reflects the change of ultrasonic signal; is the gradient of the temperature field, which indicates the change of temperature in space; is the heat flux, which indicates the flow of heat in the concrete; It is the heat accumulation term, describing the change of temperature over time. The solution process is to numerically solve the coupled equations through the finite difference method or the finite element method, obtain the integrated features after fusion, and map the results into the eigenvalues ​​of the spatial position for data fusion.

[0054] In this embodiment, after obtaining the fused comprehensive feature F fusion After that, the integrated feature F fusion Transformed into a two-dimensional matrix representation, where each element represents the value of the fusion feature at the spatial position, the matrix size is H×W, where H is the height and W is the width. Then, the fusion feature matrix F fusion For visualization, the matrix values ​​are mapped to pixel intensity values. Subsequently, the eigenvalues ​​are normalized to a range of [0,255] in order to be converted into a standard image format. The expression is: Among them, F′ fusion is the normalized feature matrix;

[0055] According to the input format requirements, the two-dimensional feature matrix is ​​expanded to a multi-channel image. If MaskR-CNN requires RGB format, the single-channel feature matrix F′ is fusion Copy it into three channels, and its expression is: F RGB =Stack(F′ fusion , F′ fusion , F′ fusion ), where the generated F RGB is a three-channel image of shape H×W×3;

[0056] The feature image F RGBThe size is adjusted to the standard size of 512×512. Finally, multiple converted images are organized into batches to form the standard Mask R-CNN input format with the shape of: Batch Shape = (N, H target , W target , 3), where N is the batch size.

[0057] In this embodiment, the steps of establishing the Mask R-CNN model are:

[0058] Collect and organize multimodal datasets containing ultrasonic data and infrared thermal imaging data. Then, annotate the location and category of the targets in the dataset, generate corresponding segmentation masks and bounding boxes, use ResNet as the backbone network, and combine RegionProposalNetwork to generate candidate regions.

[0059] Use RoIAlign to align candidate regions, and use branch networks to achieve bounding box prediction and mask segmentation. Use ResNet as the backbone network to extract fusion features and combine it with RegionProposalNetwork (RPN) to generate candidate regions; use RoIAlign operations to align candidate regions, and use branch networks to achieve bounding box prediction and mask segmentation.

[0060] In this embodiment, when training the MaskR-CNN model, the particle swarm optimization algorithm is used to optimize the training process of the MaskR-CNN model, and the steps of integrating the features of ultrasound and infrared thermal imaging and training the model through the optimization function are as follows:

[0061] Initialize a particle swarm, where each particle represents a set of hyperparameter configurations of the Mask R-CNN model. The position of the particle represents the hyperparameter (such as learning rate, regularization coefficient, number of network layers, etc.), and the speed of the particle controls the adjustment direction and amplitude of the hyperparameter.

[0062] The total loss function is calculated through the MaskR-CNN model and used as the fitness function of the particle swarm optimization. The loss function includes the classification loss L classification , regression loss L regression , mask segmentation loss L mask , and the velocity update term of particle swarm optimization, which is expressed as: Among them, λ is the hyperparameter of the particle swarm optimization process, which controls the impact of the particle swarm on network training, V i (t) is the velocity of particle ii at time;

[0063] In each iteration of particle swarm optimization, the Mask R-CNN model is trained using the hyperparameter configuration represented by the current particle, the classification, regression, and mask segmentation accuracy are calculated, the fitness value is obtained, and the position and velocity of the particle are updated to optimize the training process;

[0064] According to the input feature map B fusion And particle swarm optimization algorithm, the final loss function expression is: L final =L Mask R-CNN (B fusion )+α·PSO(B fusion , θ), where α is the weight factor that controls the intensity of particle swarm optimization and θ is the hyperparameter of the MaskR-CNN model.

[0065] In this embodiment, at each iteration of the particle swarm optimization, the position and velocity of the particle are updated by the following update rule, which is expressed as: V i (t+1)=ω·V i (t)+c1·r1·(P best -X i (t))+c2·r2·(G best -X i (t)), where V i (t) and X i (t) are the velocity and position of particle i at time t, ω is the inertia weight, c1, c2 are learning factors, r1, r2r are random numbers, P best is the historical optimal position of the particle, G best is the global optimal position.

[0066] The time domain waveform, frequency analysis results or reflection diagram of ultrasonic data are collected synchronously with the temperature distribution data of infrared thermal image. The ultrasonic and infrared images are fused into the same image through image processing technology (such as image overlay, color mapping, etc.). The thermal image can be used to show the change of the surface temperature of the material, while the ultrasonic image can be used to reveal the distribution of internal defects.

[0067] In MaskR-CNN, a network consisting of convolutional layers, pooling layers, and ReLU functions is responsible for extracting image features. Then, the candidate regions are generated through RPF, and the feature information is integrated using RoIAlign and input into the fully connected layer. Finally, the location of the bbox is obtained and the confidence is output.

[0068] In order to illustrate the beneficial effects of this solution, the following specific comparative experimental contents are provided.

[0069] Comparative Example 1: Based on traditional manual inspection methods: The identification of hidden dangers of concrete poles mainly relies on manual inspections, using visual observation or simple tool detection (such as hammering). Internal hidden dangers are inferred by detecting visible defects such as cracks, rust, and spalling on the concrete surface.

[0070] Comparative Example 2: Based on a single deep learning model: A traditional deep learning target detection model (such as FasterR-CNN) is used to detect external defects of utility poles, relying only on infrared thermal imaging data or ultrasonic signal single modality data.

[0071] Experimental environment

[0072] Test objects: 50 concrete utility pole samples, including those with known internal and surface defects such as cracks, cavities, rust, and spalling.

[0073] Experimental equipment

[0074] Ultrasonic sensor (collecting reflection signals); infrared thermal imaging sensor (collecting temperature field data); GPU-based deep learning training environment;

[0075] Experimental procedures

[0076] In the same environment, comparative example 1, comparative example 2, and the present invention were respectively applied, and each solution was tested 50 times, and the following indicators were statistically analyzed: detection accuracy (Accuracy): the actual defect recognition accuracy; detection recall (Recall): the proportion of all defects detected; detection time (Time): the time required to complete a detection. The comparison results are shown in Table 1:

[0077]

[0078]

[0079] As shown in Table 1, the accuracy of this scheme reaches 95%, which is significantly higher than the 65% of traditional manual detection and 78% of traditional ultrasonic detection. This advantage is due to the introduction of particle swarm optimization algorithm in model training and the fusion of comprehensive feature vectors of ultrasonic signals and infrared thermal imaging temperature field data.

[0080] First, the particle swarm optimization algorithm effectively optimizes the performance of the MaskR-CNN model during the training process by dynamically adjusting hyperparameters (such as learning rate, regularization parameters, etc.), allowing the model to reach the optimal state at a faster convergence speed. By defining the fitness function, the particle swarm optimization algorithm continuously searches for the optimal solution in multiple iterations, thereby significantly improving the classification, regression and segmentation capabilities of the model, making the model more accurate and robust in detecting the location and scope of hidden dangers.

[0081] Secondly, this scheme combines the statistical characteristics of ultrasonic signals (including wave velocity, propagation time, frequency, amplitude, etc.) with the characteristics of temperature field distribution in infrared thermal imaging data to construct a comprehensive feature vector. This feature fusion not only captures the information of internal structural changes of the pole, but also combines the dynamic characteristics of surface temperature distribution. The coupling relationship between the two is established through heat conduction theory, enabling the model to extract deep correlation features from multimodal data. Therefore, the recall rate of this scheme reached 93%, indicating that the recognition of internal and surface defects is more comprehensive, especially in hidden defects (such as deep cracks or cavities) that are difficult to identify with traditional methods.

[0082] In addition, the particle swarm optimization algorithm not only improves model performance, but also indirectly reduces training time and model complexity, while the design of the comprehensive feature vector further reduces the interference of redundant information, allowing the model to quickly process input data during the inference phase. The average detection time of this solution is only 25 seconds per sample, which is much lower than the 180 seconds of manual detection and the 120 seconds of ultrasonic detection. This high efficiency not only improves the detection speed, but also provides more realistic application possibilities for the safety inspection of large-scale power systems.

[0083] The present invention is not limited to the above-mentioned embodiments. Anyone should be aware that any structural changes made under the enlightenment of the present invention, and any technical solutions that are the same or similar to the present invention, fall within the protection scope of the present invention. The technology, shape, and structural parts not described in detail in the present invention are all well-known technologies.

Claims

1. A method for identifying hidden dangers of concrete utility poles based on deep learning, characterized in that: The following steps are involved: The ultrasonic sensor collects the reflected signal inside the concrete pole, extracts the signal features, calculates the corresponding statistical features, and generates the ultrasonic signal feature vector; The temperature field data on the surface of the utility pole is collected through infrared thermal imaging sensors, and the temperature field changes are modeled based on the heat conduction equation to form corresponding temperature field characteristics; The characteristics of the ultrasonic signal are fused with the temperature change data of the infrared thermal image to obtain a comprehensive characteristic phasor; Based on the theory of heat conduction, the relationship between ultrasonic signals and temperature fields is modeled by combining their mutual influence, thus obtaining the combined features of fused ultrasonic and infrared thermal images. Establish and train the MaskR-CNN model, and use the particle swarm optimization algorithm to optimize the training process of the MaskR-CNN model, integrate the features of ultrasound and infrared thermal imaging, and train the model through the optimization function; The optimized Mask R-CNN model is used to perform target detection and defect segmentation on concrete poles. By combining the changes in the temperature field and the characteristics of ultrasonic signals, hidden dangers inside and on the surface of the poles are identified, and the defect location and segmentation results are output.

2. According to the method for identifying hidden dangers of concrete utility poles based on deep learning in claim 1, it is characterized in that: The statistical features include mean value μ, standard deviation σ, skewness S, kurtosis K, energy proportion β, and mean value μ, wherein the mean value μ, standard deviation σ, skewness S, kurtosis K, energy proportion β, and mean value μ are fused to generate the ultrasonic signal feature vector U.

3. According to the method for identifying hidden dangers of concrete utility poles based on deep learning in claim 1, it is characterized in that: After obtaining the comprehensive feature vector, the Z-score standardization method needs to be used to eliminate the dimensional differences between the feature data. The expression is: Among them, μ combined and σ combined are the mean and standard deviation of the feature vector respectively, and Z is the standardized feature data.

4. According to the method for identifying hidden dangers of concrete utility poles based on deep learning in claim 1, it is characterized in that: Based on the theory of heat conduction, combined with the mutual influence of ultrasonic signals and temperature fields, the following equations are used to model the relationship between the two: The temperature field model inside the concrete pole is established according to the heat conduction equation, and its expression is: Where: ρ is the density of concrete, C p is the specific heat capacity of concrete, T is the temperature field, t is the time, k is the thermal conductivity of concrete, and Q is the heat source term; The propagation process of ultrasonic signals will be affected by the temperature of the medium. Assuming that the propagation speed v of ultrasonic waves in concrete is affected by the temperature T, it can be expressed by the formula v(T)=v0(1+α(T-T0)), where v(T) is the wave speed at temperature T, v0 is the wave speed at reference temperature T0, and α is the coefficient of influence of temperature on wave speed. Moreover, the interaction between ultrasonic wave velocity v(T) and temperature field T is expressed by the coupling of temperature gradient and wave velocity change. The expression of the coupling equation is: Among them, U reflects the change of ultrasonic signal; is the gradient of the temperature field, which indicates the change of temperature in space; is the heat flux density, which indicates the flow of heat in the concrete; is the heat accumulation term, describing the change of temperature with time.

5. According to the method for identifying hidden dangers of concrete utility poles based on deep learning in claim 1, it is characterized in that: After obtaining the fused comprehensive feature F fusion After that, the integrated feature F fusion Transformed into a two-dimensional matrix representation, where each element represents the value of the fusion feature at the spatial position, the matrix size is H×W, where H is the height and W is the width. Then, the fusion feature matrix F fusion For visualization, the matrix values ​​are mapped to pixel intensity values. Subsequently, the eigenvalues ​​are normalized to a range of [0,255] in order to be converted into a standard image format. The expression is: Among them, F′ fusion is the normalized feature matrix; According to the input format requirements, the two-dimensional feature matrix is ​​expanded to a multi-channel image. If MaskR-CNN requires RGB format, the single-channel feature matrix F′ is fusion Copy it into three channels, and its expression is: F RGB =Stack(F′ fusion , F′ fusion , F′ fusion ), where the generated F RGB is a three-channel image of shape H×W×3; The feature image F RGB The size is adjusted to the standard size of 512×512. Finally, multiple converted images are organized into batches to form the standard Mask R-CNN input format with the shape of: Batch Shape = (N, H target , W target , 3), where N is the batch size.

6. The method for identifying hidden dangers of concrete utility poles based on deep learning according to claim 1 is characterized in that: The steps to build the MaskR-CNN model are: Collect and organize multimodal datasets containing ultrasonic data and infrared thermal imaging data. Then, annotate the location and category of the targets in the dataset, generate corresponding segmentation masks and bounding boxes, use ResNet as the backbone network, and combine RegionProposalNetwork to generate candidate regions.

7. The method for identifying hidden dangers of concrete utility poles based on deep learning according to claim 1 is characterized in that: When training the MaskR-CNN model, the particle swarm optimization algorithm is used to optimize the training process of the MaskR-CNN model. The steps of integrating the features of ultrasound and infrared thermal imaging and training the model through the optimization function are as follows: Initialize a particle swarm, where each particle represents a set of hyperparameter configurations of the Mask R-CNN model, the position of the particle represents the hyperparameter, and the speed of the particle controls the adjustment direction and amplitude of the hyperparameter; The total loss function is calculated through the MaskR-CNN model and used as the fitness function of the particle swarm optimization. The loss function includes the classification loss L classification , regression loss L reression , mask segmentation loss L mask , and the velocity update term of particle swarm optimization, which is expressed as: Among them, λ is the hyperparameter of the particle swarm optimization process, which controls the impact of the particle swarm on network training, V i (t) is the velocity of particle ii at time t; In each iteration of particle swarm optimization, the Mask R-CNN model is trained using the hyperparameter configuration represented by the current particle, the classification, regression, and mask segmentation accuracy are calculated, the fitness value is obtained, and the position and velocity of the particle are updated to optimize the training process; According to the input feature map B fusion And particle swarm optimization algorithm, the final loss function expression is: L final =L MaskR-CNN (B fusion )+α·PSO(B fusion , θ), where α is the weight factor that controls the intensity of particle swarm optimization and θ is the hyperparameter of the MaskR-CNN model.

8. The method for identifying hidden dangers of concrete utility poles based on deep learning according to claim 1 is characterized in that: In each iteration of particle swarm optimization, the position and velocity of the particles are updated by the following update rule, which is expressed as: V i (t+1)=ω·V i (t)+c1·r1·(P best -X i (t))+c2·r2·(G best -X i (t)), where V i (t) and X i (t) are the velocity and position of particle i at time t, ω is the inertia weight, c1, c2 are learning factors, r1, r2r are random numbers, P best is the historical optimal position of the particle, G best is the global optimal position.

Images