Power transmission line engineering state identification method based on satellite remote sensing data and related device

By optimizing the combination of YOLOv5 network and cascade twin neural network, the problem of insufficient sample library and low recognition accuracy in transmission line engineering operation recognition is solved, high-precision transmission line engineering status recognition is achieved, and the quality and cost of power grid construction is improved. control.

CN119048918APending Publication Date: 2024-11-29XIAN POWER TRANSMISSION & TRANSFORMATION PROJECT ENVIRONMENTAL IMPACT CONTROL TECHN CENT CO LTD +1
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202410821238.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-24
Publication Date
2024-11-29

Smart Images

  • Figure CN119048918A_ABST
    Figure CN119048918A_ABST
Patent Text Reader

Abstract

The invention belongs to an image recognition method, and provides a power transmission line project state recognition method based on satellite remote sensing data and a related device aiming at the technical problems that an existing power transmission line construction project progress management method is difficult to realize accurate regulation and control of project cost and guarantee of power grid construction quality and is easily influenced by an external environment. Based on image data of satellite remote sensing, a detector module based on a YOLOv5 network is optimized, and a boundary prediction result and a state category can be quickly obtained. According to the method, positioning of the power transmission line project in the image and accurate judgment of the working state of the project are realized only by means of satellite remote sensing data, and an innovative scheme is provided for high-precision risk monitoring of unplanned operation of power grid construction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to an image recognition method, and specifically to a method and related device for identifying the status of a power transmission line project based on satellite remote sensing data. Background Art

[0002] As the hub of the power supply lifeline, the precision and intelligence of operation control of power transmission and transformation projects plays a vital role in the construction of digital power grids. As the scale of transmission line projects under construction continues to expand, it is extremely important to carry out scientific, precise, and real-time dynamic monitoring of progress intelligent control.

[0003] At present, the management of the progress of transmission line construction projects is mainly based on manual control and regular reporting at each level. It is difficult to achieve accurate control of project costs and ensure the quality of power grid construction. It is easily affected by the external environment and it is difficult to achieve full coverage of progress supervision in multiple regions. Therefore, a method for managing the progress of transmission line construction projects with the help of remote sensing technology has been extended. However, the existing remote sensing-based transmission line engineering operations lack a rich and reliable sample library, and the data set preparation is difficult and costly, resulting in low application and recognition accuracy of intelligent recognition algorithms in this regard. For example, in the Chinese invention patent application with application publication number CN116630153A, a method and system for assessing the risk of unplanned operations of transmission line satellite remote sensing is disclosed, and it is proposed to identify the transmission line operation site and compare the operation plan database with high-resolution satellite remote sensing data to realize the full process of comparison and judgment of unplanned operation projects. However, in the preprocessing stage, it is necessary to supplement with longitude and latitude geographic coordinates as the local positioning of the transmission line engineering operation area in the whole map. If the longitude and latitude of the tower are not entered in time in the plan database, the algorithm cannot obtain the area to which the project belongs, and cannot make status judgment in the corresponding area of ​​the image. In addition, under the condition of sample limitation, it is difficult to ensure high classification accuracy by identifying multiple states of the basic stage based on the Alex Net model. The robustness is poor when only relying on the frequency spectral density method to characterize the binary classification characteristics of the on-site operation wiring stage and tower assembly stage. Summary of the invention

[0004] This application aims to address the technical problems that existing methods for managing the progress of power transmission line construction projects are difficult to achieve accurate control of project costs and quality assurance of power grid construction, and are easily affected by the external environment. It provides a method and related devices for identifying the status of power transmission line projects based on satellite remote sensing data.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In the first aspect, the present application proposes a method for identifying the state of a power transmission line project based on satellite remote sensing data, comprising:

[0007] Collect satellite remote sensing image data;

[0008] Use sliding window to traverse and read sub-images of image data to generate image queue;

[0009] Performing de-clouding and clarity processing on the images in the image queue that do not meet the clarity requirement, to obtain a processed image queue;

[0010] The sub-images in the processed image queue are input into the optimized detector module based on the YOLOv5 network to obtain the boundary prediction results and status categories of the transmission line engineering operation area;

[0011] Wherein, the optimized detector module based on the YOLOv5 network includes:

[0012] A detection layer is added to the input side of the backbone network of the YOLOv5 network, a BiFPN structure is introduced into the neck network, and a CBAM attention mechanism is embedded in the C3 module of the YOLOv5 network;

[0013] The YOLOv5 network includes multiple prediction heads, corresponding to all operating states of the transmission line project.

[0014] Furthermore, the training method of the optimized detector module based on the YOLOv5 network includes:

[0015] Taking the transmission line project as the target, a transmission line project detection network model is trained without the need to subdivide the state categories;

[0016] Freeze the weights of the backbone network of the YOLOv5 network, and then train each prediction head.

[0017] Furthermore, after obtaining the status category of the transmission line engineering operation area through the optimized detector module based on the YOLOv5 network, it also includes:

[0018] Set the confidence interval [c min ,c max ];

[0019] If the confidence of the state category is less than c min , discard the corresponding state category;

[0020] If the confidence of the state category is c min and c max In between, secondary classification is performed through a cascaded twin neural network, and the result of the secondary classification is used as the state category result;

[0021] If the confidence of the state category is greater than c max , retain the corresponding status category.

[0022] Furthermore, the cascaded twin neural network adopts ResNet50.

[0023] Furthermore, the training method of the cascade twin neural network includes:

[0024] The local graph size of the transmission line engineering area under all operating states is adjusted to the input size of the cascaded twin neural network as a training sample;

[0025] The training samples are randomly classified into two groups, and the two samples belonging to the same category are assigned a label of 0, otherwise they are assigned a label of 1, and the arranged samples are obtained;

[0026] Inputting the arranged samples into the cascade twin neural network for training until convergence, and saving the weights of the cascade twin neural network;

[0027] Filter multiple samples of each category and input them into ResNet50 for prediction. Set a new feature space for each category of samples, calculate the average value, and determine the position of all category data in the new feature space.

[0028] Furthermore, the secondary classification by the cascaded twin neural network includes:

[0029] The state category results are used as test samples and mapped to the corresponding feature space through the cascaded twin neural network to obtain the mapping results;

[0030] The Euclidean distance between the mapping result and each category of data is calculated in turn, and the category with the closest Euclidean distance is selected as the result of the secondary classification.

[0031] Furthermore, the data set acquisition method used in the training of the optimized detector module based on the YOLOv5 network includes:

[0032] Based on satellite remote sensing technology, image data of all operating conditions of the transmission line project are obtained;

[0033] Performing geometric correction on the image data, and then converting the geometrically corrected image data into an RGB image;

[0034] The RGB image is cropped by sliding the window through a rectangular window to generate a sub-image of a fixed size;

[0035] Filter the sub-maps that do not contain transmission line projects, and use the remaining sub-maps as benchmark data;

[0036] Verifying the reference data, extracting data with clarity or contrast lower than a threshold or containing clouds and fog, and performing clarity processing based on dark channel priors to obtain cleared reference data;

[0037] Combine the RGB image and the clarified reference data to perform style migration data amplification to obtain style migration data;

[0038] The transmission line engineering dataset is composed of benchmark data, clarified benchmark data, and style transfer data;

[0039] A rectangular frame is made on the transmission line project on the benchmark data for optimized YOLOv5 network-based detector module training.

[0040] In the second aspect, the present application proposes a transmission line engineering status identification system based on satellite remote sensing data, comprising:

[0041] The acquisition module is used to collect satellite remote sensing image data;

[0042] A queue module is used to traverse and read sub-images of image data using a sliding window to generate an image queue;

[0043] A clarity module, used to perform de-clouding and clarity processing on images in the image queue that do not meet the clarity requirements, to obtain a processed image queue;

[0044] A recognition module is used to input the sub-images in the processed image queue into the optimized detector module based on the YOLOv5 network to obtain the boundary prediction results and state categories of the transmission line engineering operation area;

[0045] Wherein, the optimized detector module based on the YOLOv5 network includes:

[0046] A detection layer is added to the input side of the backbone network of the YOLOv5 network, a BiFPN structure is introduced into the neck network, and a CBAM attention mechanism is embedded in the C3 module of the YOLOv5 network;

[0047] The YOLOv5 network includes multiple prediction heads, corresponding to all operating states of the transmission line project.

[0048] In a third aspect, the present application provides an electronic device, comprising:

[0049] Memory for storing computer programs;

[0050] A processor is used to implement the steps of the above-mentioned method for identifying the state of a power transmission line project based on satellite remote sensing data when executing the computer program.

[0051] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for identifying the status of a transmission line project based on satellite remote sensing data are implemented.

[0052] Compared with the prior art, this application has the following beneficial effects:

[0053] This application proposes a method for identifying the status of a power transmission line project based on satellite remote sensing data. Based on satellite remote sensing image data, and by optimizing the detector module based on the YOLOv5 network, the boundary prediction results and status categories can be quickly obtained. This application relies solely on satellite remote sensing data to achieve the positioning of the power transmission line project in the image and the accurate judgment of the operation status of the project, providing an innovative solution for high-precision risk monitoring of unplanned operations in power grid construction.

[0054] The present application also proposes a transmission line engineering status identification system based on satellite remote sensing data, a computer storage medium and an electronic device, which have all the advantages of the above-mentioned transmission line engineering status identification method based on satellite remote sensing data. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 A schematic diagram of a flow chart of a method for identifying the state of a power transmission line project based on satellite remote sensing data in this application;

[0057] Figure 2 This is a schematic diagram of a power transmission line project identification process based on sliding window and dark channel de-clouding self-judgment in an embodiment of the present application;

[0058] Figure 3 This is a schematic diagram of the network structure of the optimized detector module based on the YOLOv5 network in the embodiment of the present application;

[0059] Figure 4 This is a schematic diagram of a specific network structure of a single-category prediction head in an embodiment of the present application;

[0060] Figure 5 This is a flow chart of engineering state recognition of a cascade of a detector and a twin neural network classifier in an embodiment of the present application;

[0061] Figure 6 This is a schematic diagram of the state recognition result based on the twin network classifier in the embodiment of the present application; wherein (1) to (8) are schematic diagrams of the state recognition result of the eight states of not started, foundation excavation, foundation pouring, foundation completion, tower assembly construction, tower assembly completion, wire hanging and tightening, and construction completion;

[0062] Figure 7 The present invention discloses a connection diagram of a power transmission line engineering status identification system based on satellite remote sensing data. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0064] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0065] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0066] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0067] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0068] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0069] The management of the progress of transmission line construction projects is mainly based on manual control and regular reporting. Due to the long construction period, wide construction area, and high mobility of personnel and equipment, it is difficult to report dynamic information on the construction site in a timely manner. In addition, some engineering management indicators are extensive, and unplanned operations that slow down, catch up, or speed up the construction progress occur from time to time, making it difficult to accurately control the project cost and ensure the quality of power grid construction. In addition, the transmission line operation area is wide, with many points and long lines, distributed in plains, mountains, hills and other multi-address landforms. Traditional frequent manual monitoring or on-site video monitoring is easily constrained by climate and natural environment, making it difficult to achieve full coverage of progress supervision over large areas in multiple regions.

[0070] Remote sensing technology, with its advantages of wide monitoring range and fast data transmission, has shown high application value in power grid construction and operation and maintenance. In particular, it can automatically identify the status of power transmission line projects based on remote sensing data by combining artificial intelligence algorithms, which can overcome the limitations of artificial, spatial scope and time continuity faced by project progress supervision, and avoid the drawbacks of manual image interpretation that relies on experience and is highly subjective. However, the existing power transmission line engineering operations based on remote sensing lack a rich and reliable sample library, and the preparation of data sets is difficult and costly, resulting in a low application degree and recognition accuracy of intelligent recognition algorithms in this area.

[0071] This application is based on the practical difficulties and pain points exposed in the process of supervising the progress of the above-mentioned power transmission line project, and provides a power transmission line project status identification method and related devices based on satellite remote sensing data, which realizes the automatic identification and positioning of the power transmission line project in the remote sensing image and the accurate judgment of its operation status, overcomes the limitations of small sample classification of power transmission and transformation remote sensing, and improves the accuracy of judgment on unplanned operation of the power transmission line project. The following is a detailed description of this application in conjunction with the embodiments and drawings.

[0072] like Figure 1 As shown, it is a flow chart of a method for identifying the state of a power transmission line project based on satellite remote sensing data of the present application, which may include:

[0073] S101, collecting satellite remote sensing image data.

[0074] S102, using a sliding window to traverse and read sub-images of the image data to generate an image queue.

[0075] It should be noted that sliding window techniques can be used in image processing to analyze smaller sub-regions within a larger image.

[0076] S103, performing de-clouding and clarity processing on the images in the image queue that do not meet the clarity requirement, to obtain a processed image queue.

[0077] S104, inputting the sub-images in the processed image queue into an optimized detector module based on the YOLOv5 network to obtain boundary prediction results and state categories of the transmission line engineering operation area.

[0078] Among them, the optimized detector module based on the YOLOv5 network includes:

[0079] A detection layer is added to the input side of the backbone network of the YOLOv5 network, a BiFPN structure is introduced into the neck network, and a CBAM attention mechanism is embedded in the C3 module of the YOLOv5 network;

[0080] The YOLOv5 network includes multiple prediction heads, corresponding to all operating states of the transmission line project.

[0081] In practical applications, YOLOv5 usually uses CSPNet as the backbone network to improve the learning ability and reasoning speed of the model. PANet is also used as the neck structure to enhance feature extraction and fusion capabilities. The output layer is usually a convolutional layer for predicting bounding boxes, categories, and confidence scores.

[0082] The following is another embodiment of a method for identifying the state of a power transmission line project based on satellite remote sensing data of the present application, which may include:

[0083] S201, collects high-resolution image data of power transmission line projects based on satellite remote sensing technology.

[0084] In actual applications, the collected image data includes eight different operation states of power transmission line engineering work not started, foundation excavation, foundation pouring, foundation completion, tower assembly, tower assembly completion, line hanging and tightening, and construction completion, ensuring full coverage of the power transmission line range. The image data is in TIFF format containing geographic coordinates and is based on the GCS_WGS_1984 or Geodetic 2000 coordinate system.

[0085] S202, using a sliding window to crop the image data, performing dark channel declouding preprocessing and style transfer transformation on the sub-image, and constructing a data set of power transmission line engineering projects based on this.

[0086] S202-1, perform geometric correction preprocessing on the image data collected in S201, and use GDAL (Geospatial Data Abstraction Library) to convert the image data in TIFF format containing geographic coordinates into RGB images.

[0087] Since satellite images have a high resolution and cannot be directly used as the input of the transmission line project detection algorithm, a rectangular window with an overlap ratio of 10% is set to perform sliding window cropping on the image to generate sub-images of fixed size. Most of the sub-images that do not contain transmission line projects are manually checked and filtered, and the remaining sub-images are used as benchmark data.

[0088] S202-2, the transmission line project has to go through multiple stages from foundation excavation to completion of line erection. Due to the temporal relationship of engineering construction, different types of projects may appear simultaneously in a relatively close area and lack significant classification features. For example, the completion and construction of tower assembly when there are residual building materials on the ground, as well as the completion of line hanging and tightening and construction, increase the difficulty of state classification. Therefore, the benchmark data is preprocessed at the beginning of the data set construction, and the data with low clarity and contrast or containing fog is manually verified and extracted for clarity preprocessing based on dark channel prior. Considering that the foundation stage operation involves soil excavation, backfilling and other processes, the background is similar to the visual features of the transmission line project target, which increases the risk of subsequent network misjudgment. Therefore, weighted guided filtering is used to optimize the transmittance during the clarity processing.

[0089] S202-3, combining the original data with the clarified reference data to perform style transfer data augmentation. The clarified reference data is input into the pre-trained VGG-19 network model for feature learning, and the corresponding feature maps of the original training style image and the content image are obtained. The style transfer image is reconstructed based on the feature image to enrich the sample features.

[0090] After the above processing, the benchmark data, the clarified benchmark data, and the style transfer data are used to form a transmission line engineering dataset. The labeling tool LabelImg is used to mark the transmission line projects in the transmission line engineering dataset with rectangular frames.

[0091] Satellite remote sensing image data has a high resolution, the grayscale change trend between the power transmission line project target and the background is weak, and the difference with other ground objects is not obvious. The current target detection algorithm has great difficulty in handling such small target tasks. In addition, the direct use of general models to directly identify the state of power transmission line projects has poor accuracy, and the model can only detect targets with a large number of specific projects or state samples, and its generalization and robustness are limited. Through image clarity preprocessing and style transfer to enrich samples, the detection performance of the native network for small targets of power transmission line projects is optimized.

[0092] S203, design a transmission line engineering identification process based on sliding window and dark channel declouding self-judgment.

[0093] like Figure 2 As shown, it is a schematic diagram of the power transmission line engineering identification process based on sliding window and dark channel de-clouding self-judgment, which may include:

[0094] S203-1, input the original high-resolution satellite remote sensing TIFF image, traverse and read the sub-images according to the same sliding window size as step S202-1, use the GDAL library to generate an image queue, and determine in turn whether the images in the queue need to be declouded and clarified.

[0095] In practical applications, the average grayscale value of the dark channel map can be calculated. Since the ground objects in the satellite image are far away, the transmission line project appears grayish white. In this case, according to statistical experience, the average grayscale value of the dark channel of most sub-images is greater than 0.2. Compared with the normal clear state without fog, the dark channels of the engineering targets are mostly black or dark gray. The average grayscale value of the dark channel is not greater than 0.2. The dark channel declouding threshold σ0 is set. When the average grayscale value of the dark channel of the sub-image is greater than σ0, it is declouded. Otherwise, it does not need to be clarified. The calculation formula of σ0 is as follows:

[0096]

[0097] Among them, J dark is the dark channel image of the sub-image, σ represents its average grayscale value, the threshold σ0 is 0.2, and defog represents whether to perform declouding and clarity processing.

[0098] S203-2, the sub-image is sent to the trained and optimized detector module based on the YOLOv5 network to identify the transmission line project, and the boundary prediction results [x, y, w, h] and state category i of the transmission line project operation area are output. Among them, the boundary prediction results are used for local positioning of the transmission line project area, and a local map containing only the project area is obtained. Considering that the small network has low data fault tolerance performance, the misdetected local map can be manually eliminated, and the difference between i and the true label can be compared. The traditional data enhancement such as flipping, rotating, and mirroring is performed on the category misjudgment to lay the data foundation for the subsequent twin neural network training.

[0099] When training the optimized detector module based on the YOLOv5 network, the transmission line project data set with the transmission line project annotated with a rectangular frame is used to train the detector module based on the YOLOv5 network, so that the detector module based on the YOLOv5 network can output the boundary prediction results and state categories of the transmission line project operation area.

[0100] The optimized detector module based on the YOLOv5 network introduces a small target detection layer and an attention mechanism to optimize the YOLOv5 project area positioning based on the YOLOv5 network.

[0101] (1) Adding a 160×160 detection layer to improve the original network's attention to the location information of small targets. The remote sensing sub-image input to the original YOLOv5 model will produce 8 times, 16 times and 32 times downsampling results, generating 80×80, 40×40 and 20×20 prediction feature maps respectively to correspond to the detection of small, medium and large targets. Considering that most of the wires in the sub-images of the towers that have been erected or completed have fewer pixels, the sub-images only have missing feature information after multiple downsampling processes. Therefore, the original three detection layers are difficult to meet the detection of small targets, resulting in poor network detection performance. Therefore, a 160×160 detection layer is introduced on the basis of the original network to focus on the location information of small targets.

[0102] (2) In order to solve the problem of limited accuracy in spatial positioning of small targets due to background interference and other factors, the BiFPN structure is introduced to optimize the original neck network, and bidirectional fusion of deep and shallow features is achieved from top to bottom and bottom to top. The features of multiple levels in step S205-1 are balanced by cross-layer connection and feature weighting to provide a more global feature representation with stronger semantic information, thereby improving the positioning performance of power transmission line engineering targets of different scales in complex environments. On this basis, the CBAM attention mechanism is embedded in the C3 module to further enhance the spatial structure information of small targets, capture the correlation characteristics between the aforementioned multiple spatial features of different scales with the help of the spatial attention mechanism, and normalize the channel attention in the channel dimension of the multi-level feature map, thereby improving the overall positioning capability of the transmission line engineering operation area. Figure 3 , which is a schematic diagram of the network structure of the optimized detector module based on the YOLOv5 network.

[0103] (3) In combination with the above improvements, Focal-EIoU Loss can also be used to improve the stability of network training and effectively measure the real difference between the detection box and the confidence. The loss function combination of Focal Loss and EIoU Loss is used. Focal Loss reduces the difference between the width and height of the predicted box and the minimum external box by performing a difference operation, and EIoU Loss can reduce the difference between the predicted result and the real box.

[0104] The calculation formula of EIoU Loss is:

[0105] L EIoU =L IoU +L dis +L asp

[0106] Among them, L EIoURepresents the EIoU Loss loss function, L IoU , L dis , L asp They represent IoU loss, distance loss, and width-height loss respectively.

[0107] Integrate Focal Loss into EIoU Loss and select high-quality anchor frames for loss calculation. The calculation method is:

[0108] L focal-EIoU =IoU γ L EIoU

[0109] Among them, L focal-EIoU It is the loss function combination of Focal Loss and EIoU Loss, and γ is the hyperparameter for curve curvature control.

[0110] In this embodiment, a small target detection layer is added, a CBAM attention mechanism is embedded, and the feature fusion module is replaced with a BiFPN structure, which increases the model parameters and the average inference time by 1.5ms. In addition, the training process is stable, and the mAP value is significantly improved by 1.5%.

[0111] In addition, the prediction head of the YOLOv5 network is optimized;

[0112] (1) In this application, a transmission line engineering state detector with multiple prediction heads is constructed. Traditional target detection algorithms need to score multiple categories while completing the position prediction of the target. In contrast, this application adopts a detection method in which multiple engineering state categories are parallel. A separate prediction head is trained for each of the eight operating states of the transmission line in order to predict the bounding box of each category. According to this detection strategy, when new operating states need to be added in the foundation, tower assembly and other stages, only the corresponding new prediction heads need to be added, thereby realizing the reuse of the network structure. For any type of transmission line engineering state, the output category is set to i∈{0,1}, where 0 represents the background and 1 represents the engineering state. Independent IoU intersection-over-union thresholds are set for multiple operating stages, and the single-category recall rate of different operating states is improved based on category correlation.

[0113] (2) When training the detector module based on the YOLOv5 network, all transmission line projects can be taken as targets during the training process to train a transmission line project detection network model that does not require subdividing the operating status categories. After the training reaches convergence, the weights of the feature extraction network with the small target detection layer added are frozen, and separate prediction heads are trained for different transmission line project operating states, and the prediction heads that have been trained to converge are merged. Among them, Conv, C3, upsampling and Concat modules are set in the single-category prediction head to further extract and fuse the convolution features of different engineering targets. Furthermore, the input part of the single-category prediction head is the image features extracted by the last four C3 modules in the feature extraction network in which the small target detection layer is added in step S4, which correspond to the convolution features of four different scales of 160×160, 80×80, 40×40 and 20×20 respectively. The features of the four scales are fused uniformly in the fusion layer, and a feature map of a unified scale is output. After passing through a specific discrimination module, the boundary prediction result of the transmission line engineering operation area is output, including the target position [x, y, w, h], and the category i and confidence c output by the single-category corresponding prediction head, where x, y, w, h correspond to the horizontal and vertical positions, frame width and frame height of the target respectively. The specific network structure of the single-category prediction head is as follows: Figure 4 shown.

[0114] S204, construct an engineering state recognition module that cascades a detector and a twin neural network classifier. Figure 5 Shown is the flow chart of engineering state recognition of the cascade of detector and twin neural network classifier.

[0115] Since the categories of power transmission line engineering status are easily confused, the same area in the detector results may present high scores in multiple categories. Therefore, a secondary judgment of the category can be made based on the confidence c in the detector detection results to determine the category to which the engineering status finally belongs.

[0116] Set the confidence interval [c min ,c max ], when the confidence c of the detector category judgment is less than c min When c∈[c min ,c max ], it indicates that the engineering status category output by the prediction head is ambiguous. The cascaded twin neural network is used as a classifier for secondary classification. When c is greater than c max , it means that the current prediction head determines the state category of the current project with a higher score probability. At this time, the current category information is retained. The specific confidence setting formula is as follows:

[0117]

[0118] Among them, i is the project status category determined by the prediction head, 0 means discard, -1 means there is ambiguity, 1 means keep, c min The value is 0.3, c max The value is 0.65.

[0119] The trained twin neural network is used as a classifier for c∈[c min ,c max ] is corrected. When the classifier result is consistent with the state type of the corresponding prediction head, it means that the state prediction of the transmission line project is correct. Otherwise, the state type of the corresponding target is modified to the type predicted by the classification network.

[0120] As an embodiment, the twin neural network uses ResNet50 as a feature extraction network, adjusts the size of all local graphs of the transmission line project area in the eight states to the model input size of 200×200, and randomly classifies them in pairs as training samples. The two samples belonging to the same class are assigned a label of 0, and vice versa. The arranged samples are sent to the model for training. After convergence, the weights of the twin network are saved, and multiple typical samples of each category are screened and input into the ResNet50 of the twin network structure for prediction. After mapping each type of typical samples to the new feature space, the average value is calculated to determine the position of all class data in the feature space. When the classifier corrects the state type of the project, the local graph containing only the project area obtained based on the local positioning in step S203-2 is used as a test sample, mapped to the corresponding feature space through the twin network, and the Euclidean distance between it and each type of typical data is calculated in turn, and the nearest category is selected as the prediction output. The final category of the ambiguous transmission line project is corrected accordingly. The state recognition result based on the twin network classifier is as follows: Figure 6 shown.

[0121] The engineering status recognition module constructed by the detector and the twin neural network classifier cascade constructed in this application optimizes the network structure in a targeted manner and designs a parallel recognition strategy for multiple independent detection heads, which can achieve high-quality small target detection while having high classification accuracy. The secondary classification based on the twin neural network effectively makes up for the poor accuracy of small remote sensing samples in judging the status of power transmission line engineering. It can improve the robustness of the small sample satellite image recognition algorithm and ultimately achieve the purpose of high-precision remote sensing power transmission line engineering operation status recognition.

[0122] This application uses a sliding window to crop satellite images, performs dark channel declouding preprocessing and style transfer transformation on image sub-images, and constructs a data set of power transmission line projects based on this. It introduces a small target detection layer and an attention mechanism to optimize the YOLOv5 project area positioning, and constructs a project state recognition module that cascades detectors and twin neural network classifiers to achieve multi-state discrimination of all power transmission line projects in the entire satellite remote sensing image. In addition, visual detection algorithms are used to solve the problem of poor accuracy in locating power transmission line projects based on remote sensing data and low accuracy in identifying small samples of project operation status, providing accurate guidance information for high-quality power grid construction and intelligent supervision of unplanned operations.

[0123] According to the unplanned operation engineering status of transmission lines in satellite remote sensing data, pre-processing such as sliding window and cloud removal self-judgment are designed taking into account the target environment characteristics, so as to improve the engineering status recognition effect in a targeted manner from the data level. Through the task logic of cascading small target detectors and small sample twin neural network classifiers, a detection head with high detection accuracy and reusability is designed to optimize the detection and classification modules, which can effectively solve the automatic recognition and positioning of power transmission and transformation remote sensing images, improve the limitations of small sample classification, and ensure the accuracy of status judgment results.

[0124] Based on the above-mentioned transmission line engineering status identification method based on satellite remote sensing data, such as Figure 7 As shown, the present application also proposes a power transmission line engineering status identification system based on satellite remote sensing data, which may include:

[0125] The acquisition module is used to collect satellite remote sensing image data;

[0126] A queue module is used to traverse and read sub-images of image data using a sliding window to generate an image queue;

[0127] A clarity module, used to perform de-clouding and clarity processing on images in the image queue that do not meet the clarity requirements, to obtain a processed image queue;

[0128] A recognition module is used to input the sub-images in the processed image queue into the optimized detector module based on the YOLOv5 network to obtain the boundary prediction results and state categories of the transmission line engineering operation area;

[0129] Wherein, the optimized detector module based on the YOLOv5 network includes:

[0130] A detection layer is added to the input side of the backbone network of the YOLOv5 network, a BiFPN structure is introduced into the neck network, and a CBAM attention mechanism is embedded in the C3 module of the YOLOv5 network;

[0131] The YOLOv5 network includes multiple prediction heads, corresponding to all operating states of the transmission line project.

[0132] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0133] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0134] An electronic device provided in an embodiment of the present application includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the method for identifying the status of a power transmission line project based on satellite remote sensing data as described in any of the above embodiments are implemented.

[0135] Another electronic device provided in the embodiment of the present application may also include: an input port connected to the processor, used to transmit multimodal data collected by an external acquisition device to the processor; and a display unit connected to the processor, used to display the processing results of the processor to the outside world; a communication module connected to the processor, used to realize communication between the electronic device and the outside world. The display unit can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module includes but is not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless connection: wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s.

[0136] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the method for identifying the status of a transmission line project based on satellite remote sensing data as described in any of the above embodiments are implemented.

[0137] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0138] For the description of the relevant parts of the power transmission line engineering state identification system based on satellite remote sensing data, the electronic device and the computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the power transmission line engineering state identification method based on satellite remote sensing data provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0139] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying the status of a power transmission line project based on satellite remote sensing data. It is characterized in that include: Collect satellite remote sensing image data; Use sliding window to traverse and read sub-images of image data to generate image queue; Performing de-clouding and clarity processing on the images in the image queue that do not meet the clarity requirement, to obtain a processed image queue; The sub-images in the processed image queue are input into the optimized detector module based on the YOLOv5 network to obtain the boundary prediction results and status categories of the transmission line engineering operation area; Wherein, the optimized detector module based on the YOLOv5 network includes: A detection layer is added to the input side of the backbone network of the YOLOv5 network, a BiFPN structure is introduced into the neck network, and a CBAM attention mechanism is embedded in the C3 module of the YOLOv5 network; The YOLOv5 network includes multiple prediction heads, corresponding to all operating states of the transmission line project.

2. According to claim 1, a method for identifying the state of a power transmission line project based on satellite remote sensing data, It is characterized in that The optimized detector module based on the YOLOv5 network, the training method includes: Taking the transmission line project as the target, a transmission line project detection network model is trained without the need to subdivide the state categories; Freeze the weights of the backbone network of the YOLOv5 network, and then train each prediction head.

3. According to claim 2, a method for identifying the state of a power transmission line project based on satellite remote sensing data, It is characterized in that After obtaining the status category of the transmission line engineering operation area through the optimized detector module based on the YOLOv5 network, it also includes: Set the confidence interval [c min ,c max ]; If the confidence of the state category is less than c min , discard the corresponding state category; If the confidence of the state category is c min and c max In between, secondary classification is performed through a cascaded twin neural network, and the result of the secondary classification is used as the state category result; If the confidence of the state category is greater than c max , retain the corresponding status category.

4. According to claim 3, a method for identifying the state of a power transmission line project based on satellite remote sensing data, It is characterized in that The cascaded twin neural network adopts ResNet50.

5. According to claim 4, a method for identifying the state of a power transmission line project based on satellite remote sensing data, It is characterized in that The training method of the cascade twin neural network comprises: The local graph size of the transmission line engineering area under all operating states is adjusted to the input size of the cascaded twin neural network as a training sample; The training samples are randomly classified into two groups, and the two samples belonging to the same category are assigned a label of 0, otherwise they are assigned a label of 1, and the arranged samples are obtained; Inputting the arranged samples into the cascade twin neural network for training until convergence, and saving the weights of the cascade twin neural network; Filter multiple samples of each category and input them into ResNet50 for prediction. Set a new feature space for each category of samples, calculate the average value, and determine the position of all category data in the new feature space.

6. According to claim 5, a method for identifying the state of a power transmission line project based on satellite remote sensing data, It is characterized in that The secondary classification is performed by cascaded twin neural networks, including: The state category results are used as test samples and mapped to the corresponding feature space through the cascaded twin neural network to obtain the mapping results; The Euclidean distance between the mapping result and each category of data is calculated in turn, and the category with the closest Euclidean distance is selected as the result of the secondary classification.

7. A method for identifying the state of a power transmission line project based on satellite remote sensing data according to claim 6, It is characterized in that The data set acquisition method used when training the optimized detector module based on the YOLOv5 network includes: Based on satellite remote sensing technology, image data of all operating conditions of the transmission line project are obtained; Performing geometric correction on the image data, and then converting the geometrically corrected image data into an RGB image; The RGB image is cropped by sliding the window through a rectangular window to generate a sub-image of a fixed size; Filter the sub-maps that do not contain transmission line projects, and use the remaining sub-maps as benchmark data; Verifying the reference data, extracting data with clarity or contrast lower than a threshold or containing clouds and fog, and performing clarity processing based on dark channel priors to obtain cleared reference data; Combine the RGB image and the clarified reference data to perform style migration data amplification to obtain style migration data; The transmission line engineering dataset is composed of benchmark data, clarified benchmark data, and style transfer data; A rectangular frame is made on the transmission line project on the benchmark data for optimized YOLOv5 network-based detector module training.

8. A power transmission line engineering status identification system based on satellite remote sensing data, It is characterized in that include: The acquisition module is used to collect satellite remote sensing image data; A queue module is used to traverse and read sub-images of image data using a sliding window to generate an image queue; A clarity module, used to perform de-clouding and clarity processing on images in the image queue that do not meet the clarity requirements, to obtain a processed image queue; A recognition module is used to input the sub-images in the processed image queue into the optimized detector module based on the YOLOv5 network to obtain the boundary prediction results and state categories of the transmission line engineering operation area; Wherein, the optimized detector module based on the YOLOv5 network includes: A detection layer is added to the input side of the backbone network of the YOLOv5 network, a BiFPN structure is introduced into the neck network, and a CBAM attention mechanism is embedded in the C3 module of the YOLOv5 network; The YOLOv5 network includes multiple prediction heads, corresponding to all operating states of the transmission line project.

9. An electronic device, It is characterized in that include: Memory for storing computer programs; A processor is used to implement the steps of the method for identifying the state of a power transmission line project based on satellite remote sensing data as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for identifying the state of a power transmission line project based on satellite remote sensing data as claimed in any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Improved YOLOv5l-based cow dung identification and detection method

    CN120071088A

  • Power transmission and transformation line remote sensing image enhancement system based on digital twinning

    CN120387941A

  • Transmission and transformation project progress identification method and device based on space-air-ground integration technology

    CN120656068A

  • A method and device for identifying the progress of power transmission and transformation projects based on integrated space-air-ground technology

    CN120656068B

  • Transmission line inspection data processing system based on satellite images

    CN121329152A