SAR transmission tower identification method based on deep learning and three-dimensional electromagnetic scattering model

Abstract

The invention discloses an SAR (Synthetic Aperture Radar) transmission tower identification method based on deep learning and a three-dimensional electromagnetic scattering model, solves the problems that a data set is single and the data set cannot well simulate an actual condition in the existing method, and belongs to the technical field of radar remote sensing and image processing. The method comprises the following steps: constructing SAR data sets of various transmission towers; constructing three-dimensional electromagnetic scattering models of the various transmission towers at different angles, different terrains and different scenes according to the radar images of the various transmission towers, projecting target three-dimensional scattering echoes obtained by using the three-dimensional electromagnetic scattering models onto a two-dimensional imaging plane, and obtaining two-dimensional data including a distance direction and an azimuth direction as a simulation sample; forming a training data set by the SAR data set and the simulated sample, and training a transmission tower identification model by using the training data set; and inputting an SAR image of a to-be-detected power transmission tower into the trained power transmission tower identification model, and outputting the type by the power transmission tower identification model.

Description

Technical Field

[0001] The invention relates to a SAR transmission tower recognition method based on deep learning and a three-dimensional electromagnetic scattering model, and belongs to the technical field of radar remote sensing and image processing. Background Art

[0002] The automatic identification of transmission towers is of great significance, which can realize real-time monitoring and management of transmission lines and towers, and ensure the safe operation of transmission towers. The three-dimensional electromagnetic scattering model can model and identify the characteristics of transmission towers, thereby realizing automatic identification and analysis. Deep learning can learn and understand data by simulating the neural network structure of the human brain, and use appropriate neural network structures and large-scale annotated data to realize automatic identification and positioning of targets in complex scenes. SAR technology can generate high-resolution two-dimensional or three-dimensional images, which are not affected by weather and daylight conditions, and can therefore provide reliable surface information in various environments. This creates a good basic condition for the identification of transmission towers.

[0003] However, most of the current transmission tower recognition models rely on a single SAR data set, which is not rich enough. Therefore, the recognition ability of transmission towers is poor. At the same time, since the data set used by most transmission tower recognition models cannot simulate the actual situation well, they can only recognize transmission towers in certain specific terrains or scenarios, so they cannot accurately identify transmission towers. Summary of the invention

[0004] In order to solve the problems that the data set in SAR transmission tower identification based on deep learning is single and the data set cannot simulate the actual situation well, and can only identify transmission towers in certain specific terrains or scenarios, the present invention provides a SAR transmission tower identification method based on deep learning and a three-dimensional electromagnetic scattering model.

[0005] A SAR transmission tower recognition method based on deep learning and three-dimensional electromagnetic scattering model of the present invention comprises:

[0006] Construct SAR datasets of various types of transmission towers;

[0007] According to the radar images of various transmission towers, three-dimensional electromagnetic scattering models of various transmission towers at different angles, different terrains and different scenes are constructed. The three-dimensional scattering echo of the target obtained by the three-dimensional electromagnetic scattering model is projected onto the two-dimensional imaging plane to obtain two-dimensional data including range and azimuth as simulation samples;

[0008] The SAR data set and the simulated samples together constitute a training data set, wherein the input data is the SAR image data of the transmission tower, and the output is the type of the corresponding transmission tower;

[0009] Establish a transmission tower recognition model based on a deep learning network, and use the training data set to train the transmission tower recognition model;

[0010] The SAR image of the transmission tower to be tested is input into the trained transmission tower recognition model, and the transmission tower recognition model outputs the type.

[0011] 2. The SAR transmission tower recognition method based on deep learning and three-dimensional electromagnetic scattering model according to claim 1 is characterized in that the method of constructing three-dimensional electromagnetic scattering models of various transmission towers at different angles, different terrains and different scenes according to radar images of various transmission towers comprises:

[0012] The radar emits electromagnetic waves to illuminate various transmission towers at different angles, different terrains, and different scenes, and receives their echoes to obtain radar images, and then obtains the actual scattering data of various transmission towers based on the radar images;

[0013] Build a 3D electromagnetic scattering model:

[0014]

[0015] In the formula, represents the three-dimensional scattered echo of the target, f is the real-time frequency of the radar transmitting signal, is the azimuth angle, θ is the elevation angle, A i represents the scattering intensity corresponding to the i-th scattering center, α i is the scattering type parameter corresponding to the i-th scattering center, x i ,y i 、z i Respectively represent the lateral distance, longitudinal distance and vertical distance of the scattering point in the reference coordinate system of the transmission tower, is additive Gaussian white noise; c represents the speed of light, j represents an imaginary number, and l represents the number of scattering centers of transmission towers;

[0016] Through optimization search, the actual scattering data of various transmission towers are used to optimize the parameters in the three-dimensional electromagnetic scattering model, and the various scattering centers in the three-dimensional electromagnetic scattering model are integrated to achieve the optimal match between the three-dimensional electromagnetic scattering model and the actual scattering data.

[0017] Preferably, the method for constructing SAR data sets of various types of transmission towers includes:

[0018] Obtaining a set of N original SAR images containing transmission towers, preprocessing the SAR image set, annotating the preprocessed SAR images, and augmenting the annotated SAR images to obtain an augmented SAR image set;

[0019] According to the parameters of SAR data from different platforms, the time baseline and space baseline are set, and the azimuth and range multi-view windows are selected for the SAR images in the augmented SAR image set to obtain the SAR data sets of various types of transmission towers.

[0020] Preferably, the SAR image set is preprocessed, including cropping the SAR images in the SAR image set according to the region of interest, and denoising, correcting and registering the cropped SAR images;

[0021] The annotated SAR images are augmented, including image rotation, image scaling, image translation, random cropping, and adding noise.

[0022] Preferably, the recognition model of the transmission tower is implemented using the YOLOv5s-FSS model.

[0023] Preferably, the present invention further comprises using a test set to verify the recognition result of the transmission pole tower recognition model, and evaluating the performance of the transmission pole tower recognition model according to the recognition result, wherein the performance indicators include accuracy, recall rate, and F1 score.

[0024] As a preference, the accuracy f accuracy for:

[0025]

[0026] Among them, T TP is the number of samples that are detected as true and are actually true, T FP is the number of samples that are detected as true but are actually false, T TN is the number of samples that are detected as false but are actually false, T FN is the number of samples that are tested as false but are actually true;

[0027] Recall rate f recall for:

[0028]

[0029] F1 score F1score for:

[0030]

[0031] The beneficial effect of the present invention is that a data set is formed by using SAR images and three-dimensional electromagnetic scattering models of transmission towers, wherein the transmission towers in the SAR images are marked out to form real samples, and the three-dimensional electromagnetic scattering models of the transmission towers are simulated samples, which together constitute a training data set, and automatic recognition of various types of transmission towers in different scenes and terrains is realized based on a deep learning model, which solves the problem of poor recognition ability caused by a single data set, and overcomes the limitation that the data set used by the transmission tower recognition model cannot simulate the actual situation well and can only recognize transmission towers in certain specific terrains or scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of the identification method of the present invention;

[0033] Figure 2 It is a structural diagram of the YOLOv5s-FSS model used in the present invention;

[0034] Figure 3 are real samples used in the present invention, where (a), (b), and (c) are labeled SAR images of various types of transmission towers;

[0035] Figure 4 The results of the transmission tower recognition model, where (a) is the tower recognition result; (b) is the location of the actual tower. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0038] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0039] refer to Figure 1 The present invention provides a SAR image transmission tower recognition method based on deep learning and three-dimensional electromagnetic scattering model. The method obtains SAR images containing various types of transmission towers, annotates the transmission towers in the SAR images, and obtains real samples. At the same time, the transmission towers are modeled using the three-dimensional electromagnetic scattering model to obtain simulated samples. The real samples and simulated samples together constitute a training data set, which is added to the YOLOv5s-FSS model for training. Training data set reference Figure 2 and Figure 3 The specific steps include:

[0040] S1: Obtain N scenes of original SAR image data containing transmission towers, and perform preprocessing, including denoising, correction, and registration, to ensure data quality and consistency. Annotate the preprocessed SAR images, and augment the annotated SAR images to obtain an augmented SAR image set. The preprocessing method is as follows:

[0041] Import SAR data into the processing environment, crop the SAR image according to the area of ​​interest, and select a suitable denoising method to denoise the SAR image;

[0042] Select a certain scene image of VV polarization as the main image, register the slave image to the main image space based on the feature points, and use a geometric transformation method to make the spatial position of the slave image consistent with that of the main image to ensure that they are aligned in the same coordinate system. The geometric transformation method includes translation, rotation and scaling;

[0043] Augment the annotated SAR images, including image rotation, image scaling, image translation, random cropping, and adding noise;

[0044] According to the parameters of SAR data from different platforms, the time baseline and space baseline are set, and the azimuth and range multi-view windows are selected for the SAR images in the augmented SAR image set to obtain the SAR data sets of various types of transmission towers.

[0045] S2: Based on the radar images of various transmission towers, a three-dimensional electromagnetic scattering model of various transmission towers at different angles, different terrains, and different scenes is constructed, and the three-dimensional scattering echo of the target obtained by the three-dimensional electromagnetic scattering model is projected onto the two-dimensional imaging plane to obtain two-dimensional data including range and azimuth as the simulation sample. The specific method is as follows:

[0046] The total scattering of the transmission tower can be approximated as the superposition of the contributions of several scattering centers. The characteristics of each scattering center are related to the geometric shape and physical properties such as the structure, curvature and dielectric constant at that location, so each scattering center corresponds to a specific electromagnetic scattering mechanism. The radar emits electromagnetic waves to illuminate the target and receives its echo, thereby obtaining the scattering signal of the target. The collected data is preprocessed, such as removing noise, correcting data, and removing ground interference. The diffraction geometry theory (GTD) model can not only consider the change of the target scattering response with frequency, but also take into account the influence of azimuth and elevation angles on the target scattering intensity. Using the GTD scattering model and the collected data, a three-dimensional electromagnetic scattering model of various types of transmission towers at different angles, different terrains, and different scenarios is established. The specific formula of the target three-dimensional echo signal based on the GTD model is:

[0047]

[0048] In the formula, represents the three-dimensional scattered echo of the target, A i represents the scattering intensity corresponding to the i-th scattering center, α i is the scattering type corresponding to the i-th scattering center. Different scattering structures correspond to different parameters. f0 is the initial frequency of the radar transmitting signal. f=f0+m*Δf is the real-time frequency of the radar transmitting signal at frequency point n. is the azimuth, where is the initial azimuth angle of the target relative to the radar, is the small turning angle of the azimuth target relative to the radar, θ=θ0+k·Δθ, is the pitch angle, θ0 is the initial pitch angle of the target relative to the radar, k·Δθ is the small turning angle of the pitch target relative to the radar, x i ,y i ,z i , respectively represent the horizontal distance, longitudinal distance and vertical distance of the scattering point in the reference coordinate system of the transmission tower, is additive white Gaussian noise.

[0049] The target electromagnetic scattering model is obtained by superimposing several scattering centers. Each scattering center represents the local similarity between the three-dimensional electromagnetic scattering model and the actual data. The existence of each scattering center is combined to obtain the overall matching similarity between the three-dimensional electromagnetic scattering model and the actual data. The established model is parameter estimated and optimized, and the matching degree between the model projection data and the actual SAR data is maximized through optimization search, so as to achieve the optimal matching between the model and the data.

[0050] The three-dimensional scattering echo of the target obtained by the three-dimensional electromagnetic scattering model is projected onto the two-dimensional imaging plane. The three-dimensional parameterized electromagnetic scattering model of the target describes the scattering of the target in three-dimensional space, while the actual SAR data is a combination of two-dimensional data in the range and azimuth directions, which is in the two-dimensional imaging plane. The dimensions of the two do not match. In order to make the two comparable, they need to be transformed into the same domain.

[0051] S3. A transmission tower recognition model is established based on a deep learning model (DL). The transmission towers in the preprocessed SAR images are annotated as real samples, and the three-dimensional electromagnetic scattering model of the transmission towers is used as a simulated sample to form a data set, which is divided into a sample set and a test set. The specific method is as follows:

[0052] In order to obtain a more robust transmission tower recognition model, it is necessary to use SAR data to create training samples. The transmission towers in the preprocessed SAR data are marked with rectangular boxes, and augmentation techniques are used to increase the number of training sets. Augmentation techniques include image rotation, image scaling, image translation, random cropping, adding noise, etc., so as to obtain real samples.

[0053] The three-dimensional electromagnetic scattering model of various transmission towers in different directions, different scenes and different terrains is established by using radar, which describes the interaction between the towers and electromagnetic waves and is optimized so that the model can best fit the actual observed SAR data; the three-dimensional electromagnetic scattering model describes the conditions of different types of transmission towers at different angles, different terrains and different scenes, which is a simulation sample;

[0054] The real samples and simulated samples together constitute the sample set, and the data set is divided into training set, validation set and test set. 75% of the data in the data set is divided into training set, 15% is validation set, and the remaining 10% is test set.

[0055] S4. Input the training set into the transmission tower recognition model, train the tower recognition model and optimize the parameters so that it can accurately identify the transmission tower. The specific method is as follows:

[0056] The YOLOv5s-FSS model is used to build a recognition model for transmission towers. YOLOv5s-FSS is an improvement on the YOLOv5 version. The YOLOv5s-FSS model has the advantages of a small network and the fastest calculation speed. First, the CBS module in the YOLOv5s-FSS network is replaced with a CBF module. The difference is that the activation function after the convolution Conv and BN is different. The SiLu function is replaced with the FReLU function, which has better context capture capabilities and better understanding of small and long objects than the original network. Secondly, the spatial pyramid is improved from SPP to SPP in the YOLOv7 version, which improves the detection performance of the system.

[0057] The YOLOv5s-FSS network is divided into four parts: input, backbone network Backone, neck network Neek, and prediction network Prediction. For the input, YOLOv5s-FSS uses Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling. For the neck network Neek, a top-down FPN structure and a bottom-up PAN structure are used.

[0058] Compared with the activation function used in previous versions of YOLOv5, FReLU can improve the performance of gradient explosion caused by the large memory capacity of the transmission tower dataset. FReLU uses a two-dimensional funnel-shaped condition. FReLU expands the condition from the original one-dimensional to a two-dimensional condition, depending on the spatial context of each element, enhancing the sensitivity of the activation space, improving image vision, and further improving the detection ability of tower targets; FReLU can also effectively extract the spatial structure of the object through the pixel modeling capability provided by the spatial condition, and can well cope with the various shapes of images appearing in the tower target, thereby improving the network's detection accuracy of the tower target.

[0059] The spatial pyramid is improved from SPP to SPPSPC in the YOLOv7 version. Although the SPP module can solve the problem of input image distortion caused by image cropping, stretching and other operations, and change the original features of the image, the SPPSPC in the YOLOv7 version can adjust the original calculation amount of the image based on the advantages of the original SPP module, extract more useful feature information, and thus improve the detection accuracy.

[0060] S5. Based on the trained transmission tower recognition model, use the test set to verify the recognition results of the model. The performance evaluation indicators are accuracy, recall rate, and F1 score. The specific method is as follows:

[0061] The performance of the model is evaluated using a test set that is independent of the training set and validation set. The test set is added to the optimal deformation monitoring model, and the recognition results are compared with the actual deformation to evaluate the performance of the deformation monitoring model. The performance evaluation indicators are accuracy, recall, and F1 score.

[0062]

[0063]

[0064] In the formula, f accuracy is the accuracy, f recall is the recall rate, f F1score is the F1 score, T TP is the number of samples that are detected as true and are actually true, T FP is the number of samples that are detected as true but are actually false, T TN is the number of samples that are detected as false but are actually false, T FN is the number of samples that are detected as false but are actually true.

[0065] This embodiment uses SAR images and three-dimensional electromagnetic scattering models of transmission towers to form a data set, in which the transmission towers in the SAR images are marked to form real samples, and the three-dimensional electromagnetic scattering model of the transmission towers is a simulated sample. The YOLOv5s-FSS model is used to construct the recognition model of the transmission tower, which fully utilizes the advantages of the small network and fast calculation speed of the YOLOv5s-FSS to achieve automatic recognition of the transmission tower. In tower detection, the data set composed of a single SAR data is not rich enough. The three-dimensional scattering model of the transmission tower is used to form simulated data, which increases the richness of the data set. In addition, the three-dimensional electromagnetic scattering model can simulate the conditions of various types of transmission towers at different angles, different scenes, and different terrains, making the model more accurate in identifying the transmission tower.

[0066] Please refer to Figure 4 , Figure 4 is the recognition result of the transmission tower recognition model, (a) is the tower recognition result, and (b) is the location of the actual tower.

[0067] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. SAR transmission tower recognition method based on deep learning and three-dimensional scattering model, It is characterized in that include: Construct SAR datasets of various types of transmission towers; According to the radar images of various transmission towers, three-dimensional scattering models of various transmission towers at different angles, different terrains and different scenes are constructed. The three-dimensional scattering echo of the target obtained by the three-dimensional electromagnetic scattering model is projected onto the two-dimensional imaging plane to obtain two-dimensional data including range and azimuth as simulation samples; The SAR data set and the simulated samples together constitute a training data set, wherein the input data is the SAR image data of the transmission tower, and the output is the type of the corresponding transmission tower; Establish a transmission tower recognition model based on a deep learning network, and use the training data set to train the transmission tower recognition model; The SAR image of the transmission tower to be tested is input into the trained transmission tower recognition model, and the transmission tower recognition model outputs the type.

2. According to the SAR transmission tower identification method based on deep learning and three-dimensional scattering model according to claim 1, It is characterized in that Methods for constructing three-dimensional scattering models of various transmission towers at different angles, different terrains, and different scenes based on radar images of various transmission towers include: The radar emits electromagnetic waves to illuminate various transmission towers at different angles, different terrains, and different scenes, and receives their echoes to obtain radar images, and then obtains the actual scattering data of various transmission towers based on the radar images; Build a 3D scattering model: In the formula, represents the three-dimensional scattered echo of the target, f is the real-time frequency of the radar transmitting signal, is the azimuth angle, θ is the elevation angle, A i represents the scattering intensity corresponding to the i-th scattering center, α i is the scattering type parameter corresponding to the i-th scattering center, x i ,y i 、z i Respectively represent the lateral distance, longitudinal distance and vertical distance of the scattering point in the reference coordinate system of the transmission tower, is additive Gaussian white noise; c represents the speed of light, j represents an imaginary number, and l represents the number of scattering centers of transmission towers; Through optimization search, the actual scattering data of various transmission towers are used to optimize the parameters in the three-dimensional scattering model, and the various scattering centers in the three-dimensional scattering model are integrated to achieve the optimal match between the three-dimensional scattering model and the actual scattering data.

3. According to the SAR transmission tower identification method based on deep learning and three-dimensional scattering model according to claim 1, It is characterized in that Methods for constructing SAR datasets of various types of transmission towers include: Obtaining a set of N original SAR images containing transmission towers, preprocessing the SAR image set, annotating the preprocessed SAR images, and augmenting the annotated SAR images to obtain an augmented SAR image set; According to the parameters of SAR data from different platforms, the time baseline and space baseline are set, and the azimuth and range multi-view windows are selected for the SAR images in the augmented SAR image set to obtain the SAR data sets of various types of transmission towers.

4. The SAR transmission tower identification method based on deep learning and three-dimensional scattering model according to claim 3, It is characterized in that Preprocessing the SAR image set includes cropping the SAR images in the SAR image set according to the region of interest, and denoising, correcting and registering the cropped SAR images; The annotated SAR images are augmented, including image rotation, image scaling, image translation, random cropping, and adding noise.

5. The SAR transmission tower identification method based on deep learning and three-dimensional scattering model according to claim 1, It is characterized in that The recognition model of transmission towers is implemented using the YOLOv5s-FSS model.

6. The SAR transmission tower identification method based on deep learning and three-dimensional scattering model according to claim 1, It is characterized in that The method also includes using a test set to verify the recognition result of the transmission pole tower recognition model, and evaluating the performance of the transmission pole tower recognition model according to the recognition result, wherein the performance indicators include accuracy, recall rate, and F1 score.

7. The SAR transmission tower identification method based on deep learning and three-dimensional scattering model according to claim 6, It is characterized in that Accuracy accuracy for: Among them, T TP is the number of samples that are detected as true and are actually true, T FP is the number of samples that are detected as true but are actually false, T TN is the number of samples that are detected as false but are actually false, T FN is the number of samples that are tested as false but are actually true; Recall rate f recall for: F1 score F1score for:

8. A computer-readable storage device, the storage device storing a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the SAR transmission tower identification method based on deep learning and a three-dimensional scattering model as claimed in any one of claims 1 to 7 are implemented.

9. A SAR transmission tower identification device based on deep learning and three-dimensional scattering model, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, It is characterized in that The processor executes the computer program to implement the steps of the SAR transmission tower identification method based on deep learning and three-dimensional scattering model as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the SAR transmission tower identification method based on deep learning and a three-dimensional scattering model as described in any one of claims 1 to 7 are implemented.

Images