A power transmission line dangerous area detection method based on simulation and real data hybrid training

Abstract

This application provides a method for detecting hazardous areas under power transmission lines based on hybrid training using simulation and real data. It solves the technical problems of low detection accuracy caused by the inherent subjectivity, large errors, and susceptibility to failure due to changes in the original equipment in existing manually labeled hazardous areas. The method includes constructing a 3D model of the power tower, transmission line, and terrain; preprocessing the model to generate simulation data; constructing a convolutional neural network model based on a regression strategy; performing three-stage training to obtain the trained convolutional neural network model; labeling the four vertices of the simulation data using the trained convolutional neural network model; fitting a centerline based on the four vertices of the simulation data; and laterally expanding the line segment according to the base width of the transmission line and the expansion ratio of the voltage level to close the line and obtain the hazardous area of ​​the transmission line at that voltage level. This application can be widely applied in the field of hazardous area detection technology.

Description

Technical Field

[0001] This application belongs to the field of hazardous area detection technology, and more specifically, it relates to a method for detecting hazardous areas under power transmission lines based on a hybrid training of simulation and real data. Background Technology

[0002] With the continuous expansion of high-voltage transmission networks, automatic early warning systems for hazardous areas beneath high-voltage transmission lines are becoming increasingly important. Hazardous areas beneath high-voltage transmission lines are defined as the area formed by expanding outwards from the vertical projection of the transmission line on the ground (the transmission corridor) according to the voltage level. Figure 2 As shown, the left image represents the vertical projection of the transmission line onto the ground, while the right image shows the area expanded outwards by a certain proportion according to the voltage level. These hazardous areas are prohibited from containing tarpaulins, tarpaulins, wire mesh, engineering vehicles used for elevated operations (such as cranes), or long-term parked motor vehicles. These items can easily cause foreign objects to suspend on the lines or cause high-voltage air breakdown, resulting in serious accidents.

[0003] To ensure the safety of hazardous areas, the power system installs cameras on each power tower to capture real-time images of dangerous areas on the ground. The traditional method relies on staff manually reviewing the surveillance footage and issuing alerts when danger is detected, which is labor-intensive and prone to delays.

[0004] With the in-depth development of artificial intelligence technology, methods for real-time early warning using algorithms have begun to be widely used, greatly reducing labor costs. However, such automatic alarm methods for safety hazards heavily rely on the accuracy of hazard zone labeling. If the hazard zone labeling is inaccurate, whether a target within the zone is marked as outside the zone or vice versa, the reliability of the algorithm will be severely reduced. Currently, the mainstream method for safety zone labeling is manual labeling based on experience after observing the images. This method has two major drawbacks: first, individual labeling is based on intuition and deviates significantly from the actual situation. Specifically, realizing the inference of the transmission channel prediction model requires a large number of training samples with safety zone labeling information. Although a large number of original photos taken by transmission tower cameras can be easily obtained, the labeling of the vertical projection of the outer power lines on the ground is difficult to achieve, mainly because there are no clear and usable judgment labels, such as... Figure 3-4 As shown. Due to the lack of real-world parameters, labeling the projection of outer power lines onto the ground solely with the naked eye leads to significant errors. Furthermore, individual differences among labelers cause instability in the labeling results. Such labeled samples cannot train an accurate automatic prediction algorithm. Secondly, if the camera rotates, the previously labeled area becomes instantly unusable due to errors. Therefore, there is an urgent need to provide a method for automatic prediction of hazardous areas to solve these problems. Summary of the Invention

[0005] The purpose of this application is to provide a method for detecting dangerous areas under power transmission lines based on a hybrid training of simulation and real data, in order to solve the technical problems of low detection accuracy of manually marked dangerous areas in the prior art, which are characterized by strong subjectivity, large errors, and easy failure due to changes in the original equipment.

[0006] To achieve the above objectives, this application provides a method for detecting hazardous areas under power transmission lines based on a hybrid training method using simulation and real data, comprising the following steps: Construct three-dimensional models of power towers, transmission lines, and terrain, and perform preprocessing to generate simulation data; A convolutional neural network model is constructed based on a regression strategy. The model is then trained in three stages to obtain the trained convolutional neural network model. The trained convolutional neural network model is then used to label the four vertices of the simulation data. The centerline is obtained by fitting the four vertices of the simulation data. The line segment is then extended laterally according to the base width of the transmission line and the expansion ratio of the voltage level, and the dangerous area of ​​the transmission line at that voltage level is obtained by closing the line.

[0007] Preferably, the three-stage training process includes: the first stage, freezing the domain discriminator and training the keypoint detection head with simulated data; the second stage, fixing the keypoint detection head, calculating the detection loss of the keypoint detection head using simulated data, extracting features from real and simulated data from the backbone network, and performing domain adversarial training on the features through a gradient inversion layer to complete the adversarial update; and the third stage, jointly optimizing the backbone network, keypoint detection head, and domain discriminator based on real and simulated data.

[0008] Preferably, the process of jointly optimizing the backbone network, keypoint detection head, and domain discriminator includes: preprocessing and enhancing the simulation data and real data respectively, inputting them into the backbone network to extract features, and dividing the features extracted from the backbone network into two detection paths. One path is input into the keypoint detection head to calculate the keypoint loss; the other path is input into the domain discriminator to calculate the domain classification loss. The total loss is obtained by combining the keypoint loss and the domain classification loss, and the model weights are updated based on the total loss.

[0009] Preferably, the process of labeling the four vertices of the simulation data includes: using the simulation data as input to the trained convolutional neural network model, and using the trained convolutional neural network model to regress and output the coordinates of the four vertices of the power transmission channel quadrilateral; The four vertices are the projections of the intersection of the outermost tower arm and the power line on the ground. The labeling order is from the lower left corner to the lower right corner, and the key points 1, 2, 3, and 4 are labeled in a clockwise direction. The horizontal and vertical coordinates of the four points are normalized to the range of [0,1] according to the length and width of the image.

[0010] Preferably, the base width of the transmission line refers to: based on the extension direction of the line connecting key point 2 and key point 3 as the lateral reference direction, drawing horizontal parallel line segments perpendicular to the center line at different depth positions of the center line as the base width.

[0011] Preferably, the process of lateral expansion of the line segment includes: expanding outwards at both ends of the base width according to the expansion ratio corresponding to the current transmission line voltage level, so as to obtain an expanded lateral line segment with a total length equal to the product of the expansion ratio and the base width.

[0012] Preferably, the process of obtaining the dangerous area of ​​the transmission line at this voltage level includes: sequentially constructing and connecting the four vertices of the two extended horizontal line segments above and below to form a closed polygonal area, which is the dangerous area of ​​the transmission line at the current voltage level.

[0013] Preferably, the centerline of the transmission line is obtained by geometric fitting based on the coordinates of the four vertices of the simulation data.

[0014] Preferably, the process of preprocessing to generate simulation data includes: changing the position of the power tower in the ground scene, the number of power lines, the slack, and the position and orientation of the virtual camera to obtain power line data of different shapes; Change the materials, textures, and scene rendering lighting of the power transmission line data to increase the diversity of the simulation data; Using the vertical projection method, the projection boundary is calibrated, the projection of the outermost wire on the ground plane is solved, and the simulation data is output.

[0015] Preferably, the backbone network adopts a shared weight structure, combined with a domain discriminator and a gradient inversion layer, and achieves feature distribution alignment between simulated data and real data through domain adversarial training.

[0016] The beneficial effects of this application are as follows: This application provides a method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data. It constructs a 3D model of the power tower, transmission line, and terrain, and preprocesses it to generate simulation data. When the original equipment parameters or scene change, only the model parameters need to be adjusted to automatically generate new simulation data samples, eliminating the need for re-collection of on-site data and manual annotation. This avoids the invalidation of existing simulation data due to changes in the original equipment and eliminates the uncertainty caused by subjective differences in the annotation process. A convolutional neural network model is constructed based on a regression strategy and trained in three stages to obtain the trained model. The trained model automatically annotates the four vertices of the simulation data, avoiding the subjective errors of manual point-by-point annotation. Furthermore, the three-stage progressive training method ensures that the model can learn the geometric features and key point information of the mixed simulation and real data, outputting accurate vertex coordinates and improving the annotation accuracy of key point detection. The centerline is obtained by fitting the four vertices of the simulation data, and the lateral expansion ratio is determined according to the base width of the transmission line and the voltage level. After closure, the dangerous area of ​​the transmission line is obtained. The entire process of delineating the dangerous area is based on a fixed geometric fitting algorithm and the expansion rules corresponding to the voltage level. No manual intervention is required, which avoids the uncertainty of manually delineating the dangerous area and ensures the consistency of dangerous area delineation under different voltage levels and different scenarios, thereby improving the accuracy of dangerous area detection under the transmission line. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data, provided in an embodiment of this application; Figure 2 A schematic diagram illustrating the division of hazardous areas according to an embodiment of this application; Figure 3 A field diagram of a power transmission channel in a first scenario provided in an embodiment of this application; Figure 4 A field diagram of a power transmission channel in a second scenario provided in an embodiment of this application; Figure 5 A three-dimensional model of a power transmission tower, transmission line, and terrain provided in an embodiment of this application; Figure 6 A rendering of simulation data provided in one embodiment of this application; Figure 7 A simulation diagram showing the annotations of the four vertices of a power transmission channel provided in an embodiment of this application; Figure 8 A front view showing the four vertices of a power transmission line provided in an embodiment of this application; Figure 9 A side view showing the four vertices of a power transmission line provided in an embodiment of this application; Figure 10 A flowchart of YOLO12m training provided in one embodiment of this application; Figure 11 This is a schematic diagram illustrating the geometric principle of the outward expansion of the dangerous area of ​​a power transmission line according to an embodiment of this application. Detailed Implementation

[0019] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0020] Please see Figure 1 An embodiment of this application provides a method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data, comprising: S1: Construct a 3D model of the power tower, transmission line, and terrain, and perform preprocessing to generate simulation data.

[0021] First, 3D modeling software was used to model the power transmission towers, transmission lines, and ground terrain. The modeling results are as follows: Figure 5 As shown, where, Figure 5 The upper sub-image shows the baseline model of the power line towers and transmission lines from the initial viewpoint, while the lower sub-image shows the urban building complex terrain scene modeled in conjunction with the sub-image, thus completing the initial construction of the basic simulation scene. This application uses 3DMax modeling software for 3D modeling, but the software used for modeling is not limited here and can be selected according to the actual situation.

[0022] Next, by changing the positions of the power towers in the ground scene, the number and slack of the transmission lines, and the position and orientation of the virtual camera, power transmission line data with varying shapes was created. Secondly, the materials, textures, scene rendering lighting, and sky sphere effects of the power transmission line data were changed to make the overall appearance of the scene more varied, thereby increasing the diversity of the simulation data. The results are as follows: Figure 6 The two sub-images on the left and right are shown below.

[0023] Finally, using the vertical projection method of the rendering engine, the projection boundary is marked with red lines to solve for the projection of the outermost power line onto the ground plane. This projection is accurately generated according to physical definitions and is unaffected by changes in viewing angle, solving the problems of strong subjectivity and annotation failure after changes in posture in manual annotation. The final output is simulation data with the original width of the power transmission channel annotated, such as... Figure 6 The two sub-images on the right side of the middle section are shown below.

[0024] S2: Convolutional neural network model is constructed based on regression strategy, and three-stage training is performed to obtain the trained convolutional neural network model. The four vertices of the simulation data are labeled using the trained convolutional neural network model.

[0025] A convolutional neural network (CNN) is constructed based on a regression strategy. Simulated data is used as input to the network, which then regresses to output the coordinates of the four vertices of the quadrilateral of the power transmission channel. The predicted vertices are the ground projections of the intersection of the outermost tower arm and the power line, as labeled accordingly. Figure 7-9 As shown. The annotation order is uniformly based on the bottom left corner as the starting point and the bottom right corner as the ending point, clockwise annotating key points 1, 2, 3, and 4. The horizontal and vertical coordinates of the four points are normalized to the range [0,1] according to the image's width and height to eliminate scale differences caused by different image resolutions. Based on this annotation method, simulation data after annotating the four vertices is automatically generated. Among them, Figure 7 This is a schematic diagram of the principle of perspective simulation, which intuitively shows the four vertices and the lines connecting them. Figure 8 This is to simulate the effect of annotation from the perspective of power line towers. Figure 9 To observe the effect of simulated annotations from the perspective of a power line tower from the side.

[0026] In one optional embodiment, this application generates 1000 simulation data images through 3D modeling, selects 900 images for the training set and 100 images for the test set. In a real-world environment, 100 real data images under different power transmission scenarios are also collected, with 90 images sent to the training set and the remaining 10 images sent to the test set.

[0027] This application introduces domain adaptation technology to train a backbone network with shared weights, aligning the feature distributions of simulated and real data to reduce the distribution differences between real and simulated data. Furthermore, a multi-task collaborative training mechanism is used to simultaneously optimize task loss and domain adversarial loss, improving the model's generalization performance and training effectiveness. The overall training process is as follows: Figure 10 As shown.

[0028] Specifically, the network uses a YOLO12m regression network with an input image size of 960×960 and a batch size of 16 images. The Adam optimizer is used, and the network is trained for a total of 200 epochs. The entire training process is divided into three stages: The first stage: train a basic, usable keypoint detection model on simulation data, freeze the domain discriminator, set the learning rate to 0.001, train for 50 epochs, and obtain the initial model.

[0029] Phase Two: Conduct adversarial training to achieve cross-domain feature alignment, adapting the model to the real data distribution. At this stage, the keypoint detection head is fixed, and the backbone feature extraction network simultaneously extracts features from both real and simulated data. The keypoint detection head is used only on the simulated data to calculate the detection loss. The features extracted from both datasets are then fed into the domain discriminator to calculate the domain classification loss. Adversarial updates are implemented through a gradient inversion layer to confuse the simulated and real data, thereby making the features extracted by the backbone network more consistent. In this phase, the backbone network learning rate is set to 0.0001, the domain discriminator learning rate is set to 0.0005, and training is performed for 100 epochs.

[0030] Phase 3: Global joint fine-tuning is conducted. Based on cross-domain feature alignment, all data (all simulation and real data) are used with a learning rate of 0.0001 to simultaneously optimize the backbone network, keypoint detection head, and domain discriminator. The learning rate uses a cosine annealing strategy overall, with separate scheduling for each phase. After model training, model performance is validated only using a validation set composed of real data. Specifically, firstly, simulation and real data are preprocessed to enhance the data, and then input into the backbone network with shared weights to extract features. The features output from the backbone network are fed into two detection paths: one path is fed into the keypoint detection head to calculate the keypoint loss; the other path is fed into the domain discriminator to calculate the domain classification loss. The total loss is obtained by combining the keypoint loss and the domain classification loss. During training, the model weights are updated based on the total loss, and the network's ability to segment regions is improved through iterative updates.

[0031] S3: The centerline is obtained by fitting the four vertices of the simulation data. The line segment is then extended laterally according to the base width of the transmission line and the expansion ratio of the voltage level, and the dangerous area of ​​the transmission line at that voltage level is obtained by closing the line segment.

[0032] The width of the danger zone varies depending on the voltage level; the higher the voltage level, the wider the danger zone extends outward. To accommodate the differentiated zoning of different lines, this embodiment uses a multiplier and base width to estimate the outward expansion of the danger zone. First, based on the coordinates of the four vertices of the transmission channel profile predicted by the trained convolutional neural network model, the centerline of the transmission line is solved through geometric fitting, such as... Figure 11 The central axis is divided by the vertical point.

[0033] Next, based on the extension direction of the line connecting key points 2 and 3 predicted by the model as the lateral reference direction, horizontal parallel line segments perpendicular to the center line are drawn at different depth positions of the center line. This line segment is the original width of the transmission channel and is defined as the base width.

[0034] Then, based on the preset voltage level of the current transmission line and the corresponding expansion ratio, the line is simultaneously expanded outward at equal intervals at both ends of the base width, resulting in an expanded horizontal segment with a total length of expansion ratio × base width, such as... Figure 11 The magnification × base width dimensions are indicated at the top and bottom.

[0035] Finally, the four vertices of the two extended horizontal line segments are constructed and connected sequentially to form a closed polygonal region. This polygonal region is the transmission hazard area for the current voltage level. The relationship between voltage level and extension ratio is shown in Table 1.

[0036] Table 1: Relationship between voltage level and extension ratio

[0037] The calculation process is as follows Figure 11 As shown in the diagram, the pink area represents the original power transmission channel, while the blue areas on either side represent the extended danger zones. Since the voltage level corresponding to each image is known in advance and will not change, the predicted danger zone can be calculated by inputting the corresponding voltage level.

[0038] Example 1: Comparison and verification of training accuracy using simulation and real data.

[0039] To verify the accuracy of the hazardous area delineation in this application, tests were conducted on a test set consisting of a mixture of real and simulated data. The polygon overlap ratio (IOU) was used as the evaluation criterion. The higher the overlap ratio between two images, the closer the IOU value is to 1. When two images have no overlap, the IOU is 0. The formula is: ; In the formula, To predict the intersection-union ratio of polygons between the hazardous area and the actually labeled hazardous area, The area of ​​the actual danger zone is manually marked. The model outputs the predicted area of ​​the hazardous region for this application.

[0040] This application conducted comparative tests on a test set composed of mixed real and simulated data. The average accuracy of the model trained using only simulated data was 87.4%, while the average accuracy of the model trained using a fusion of simulated and real data was 93.2%. This demonstrates the effectiveness of training with a mixture of simulated and real data, and the effectiveness of fusion of simulated and real data in improving accuracy.

[0041] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0042] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A power line dangerous area detection method based on simulation and real data hybrid training, characterized in that, Includes the following steps: Construct three-dimensional models of power towers, transmission lines, and terrain, and perform preprocessing to generate simulation data; A convolutional neural network model is constructed based on a regression strategy, and a three-stage training process is performed to obtain the trained convolutional neural network model. The four vertices of the simulation data are then labeled using the trained convolutional neural network model. The centerline is obtained by fitting the four vertices of the simulation data. The line segment is then extended laterally according to the base width of the transmission line and the expansion ratio of the voltage level, and the dangerous area of ​​the transmission line at that voltage level is obtained by closing the line.

2. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 1, characterized in that, The three-stage training process includes: First, freezing the domain discriminator and training the keypoint detection head using the simulation data; Second, fixing the keypoint detection head, calculating the detection loss of the simulation data using the keypoint detection head, extracting features from the real data and the simulation data from the backbone network, and performing domain adversarial training on the features through a gradient inversion layer to complete the adversarial update; Third, jointly optimizing the backbone network, keypoint detection head, and domain discriminator based on the real data and simulation data.

3. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 2, characterized in that, The process of jointly optimizing the backbone network, keypoint detection head, and domain discriminator includes: preprocessing and enhancing the simulation data and real data respectively, inputting them into the backbone network to extract features, and dividing the features extracted from the backbone network into two detection paths. One path is input into the keypoint detection head to calculate the keypoint loss; the other path is input into the domain discriminator to calculate the domain classification loss. The total loss is obtained by combining the keypoint loss and the domain classification loss, and the model weights are updated based on the total loss.

4. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 1, characterized in that, The process of labeling the four vertices of the simulation data includes: using the simulation data as input to the trained convolutional neural network model, and using the trained convolutional neural network model to regress and output the coordinates of the four vertices of the power transmission channel quadrilateral; The four vertices are the projections of the intersection of the outermost tower arm and the power line on the ground. The labeling order is from the lower left corner to the lower right corner, and the key points 1, 2, 3, and 4 are labeled in a clockwise direction. The horizontal and vertical coordinates of the four points are normalized to the range of [0,1] according to the length and width of the image.

5. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 4, characterized in that, The base width of the transmission line refers to: based on the extension direction of the line connecting key point 2 and key point 3 as the lateral reference direction, drawing horizontal parallel line segments perpendicular to the center line at different depth positions of the center line, which serve as the base width.

6. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 5, characterized in that, The process of the lateral expansion segment includes: expanding outwards at both ends of the base width according to the expansion ratio corresponding to the current transmission line voltage level, so as to obtain an expanded lateral segment with a total length equal to the product of the expansion ratio and the base width.

7. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 6, characterized in that, The process of obtaining the dangerous area of ​​the transmission line at this voltage level includes: sequentially constructing and connecting the four vertices of the two extended horizontal line segments above and below to form a closed polygonal area, which is the dangerous area of ​​the transmission line at the current voltage level.

8. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 1, characterized in that, Based on the coordinates of the four vertices of the simulation data, the centerline of the transmission line is obtained through geometric fitting.

9. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 1, characterized in that, The process of generating the simulation data through the preprocessing includes: changing the position of the power tower in the ground scene, the number and slack of the power transmission lines, and the position and orientation of the virtual camera to obtain power transmission line data with different shapes. The materials, textures, and scene rendering lighting of the power transmission line data are changed to increase the diversity of the simulation data; Using the vertical projection method, the projection boundary is calibrated, the projection of the outermost wire on the ground plane is solved, and the simulation data is output.

10. The method for detecting dangerous areas under power transmission lines based on hybrid training of simulation and real data as described in claim 2, characterized in that, The backbone network adopts a shared weight structure, and combined with the domain discriminator and the gradient inversion layer, the feature distribution of the simulated data and the real data are aligned through domain adversarial training.

Images