A multi-ground multi-resolution fan detection method

Abstract

The application discloses a kind of multi-ground multi-resolution fan detection methods, comprising S1: obtaining fan splicing image data;S2: fan splicing image data is handled to image, obtains the fan dataset of multi-ground multi-resolution, and fan dataset is divided into data training set and data verification set according to preset proportion;S3: construct fan detection network model;S4: based on data training set and data verification set, fan detection network model is trained and model is verified to obtain optimal fan detection network model;S5: obtain the remote sensing image of the wind power plant to be detected, and obtain a plurality of remote sensing sub-images by segmentation processing, and the output detection graph is obtained by fan feature detection based on optimal fan detection network model to each remote sensing sub-image.Solved with the rapid development of wind power industry, due to the influence of geological, climatic environment and wind energy total reserves and other factors in each region, make wind power installation present regionalization, centralization and other trends, leading to the difference of wind power development in each region is remarkable, cannot efficiently identify the problem of regional fan.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fan detection, and in particular to a multi-surface multi-resolution fan detection method. BACKGROUND

[0002] There are mainly two ways to obtain fan data at present: the first is to obtain the number and distribution information of the fan according to the data provided by the wind farm; the second is to obtain the number and point information of the fan in an artificial recognition manner based on remote sensing images. Both of these two methods have great limitations and greatly waste human resources; the first method needs to communicate with major wind farms from time to time to obtain fan data, and cannot detect and obtain multi-surface multi-resolution fan data in real time according to a small amount of fan data in the early stage, and the second method needs human recognition of images, which is time-consuming and laborious and may produce some errors through artificial recognition. Therefore, it is increasingly important to develop a multi-surface multi-resolution fan detection technology for the above problems. SUMMARY

[0003] The present application provides a multi-surface multi-resolution fan detection method to overcome the above technical problems.

[0004] In order to achieve the above purpose, the technical scheme of the present application is:

[0005] A multi-surface multi-resolution fan detection method, comprising the following steps:

[0006] S1: obtaining fan splicing image data;

[0007] The fan splicing image data includes wind farm tile coordinates and fan contour features;

[0008] S2: performing image processing on the fan splicing image data to obtain a multi-surface multi-resolution fan data set, and dividing the fan data set into a data training set and a data validation set according to a preset proportion;

[0009] S3: constructing a fan detection network model;

[0010] The fan detection network model includes a YOLOv5 backbone network for extracting different scales of multi-surface multi-resolution fan features, an improved YOLOv5 neck network for multi-scale feature fusion of the fan features extracted by the backbone network, and a prediction end for multi-surface multi-resolution fan detection, and the prediction end is provided with an ASFF-4 module for adaptive feature fusion of the output features of the improved YOLOv5 neck network;

[0011] S4: model training and model validation of the fan detection network model based on the data training set and the data validation set to obtain an optimal fan detection network model;

[0012] and the model training comprises: based on a YOLOv5 backbone network, extracting wind fan features of different scales from multi-surface and multi-resolution wind fan data in a data training set, and obtaining a multi-scale wind fan feature map;

[0013] based on an improved YOLOv5 neck network, training the multi-scale wind fan feature map to obtain a wind fan fusion feature map of different scales;

[0014] based on an ASFF-4 module in the prediction end, performing adaptive feature fusion on the wind fan fusion feature map of different scales to obtain an adaptive fusion feature map, which is used to realize wind fan detection training of multi-surface and multi-resolution;

[0015] The model verification comprises: verifying the wind fan detection network model after model training by using a data verification set to obtain the optimal weight parameters of the wind fan detection network model and reconstruct the wind fan detection network model to obtain an optimal wind fan detection network model;

[0016] S5: obtaining a remote sensing image of a wind farm to be detected, performing segmentation processing to obtain a plurality of remote sensing sub-images, and performing wind fan feature detection on each remote sensing sub-image based on the optimal wind fan detection network model to obtain an output detection map.

[0017] Further, the S1 specifically comprises the following steps:

[0018] S11: obtaining wind farm coordinate information of multi-surface and multi-resolution;

[0019] The wind farm coordinate information comprises coordinate information of multiple types of surfaces and resolutions;

[0020] and performing Mercator coordinate conversion to obtain Google map wind farm tile coordinate information, and setting the map order of the Google map tile coordinate information;

[0021] S12: calling Google API to obtain wind farm tile data map according to the wind farm tile coordinate information and the map order;

[0022] S13: based on an image stitching strategy, stitching the wind farm tile data map to obtain wind fan stitching image data;

[0023] and the image stitching strategy specifically comprises

[0024] setting the stitching range of image stitching as m×n, wherein m represents the number of rows of the image stitching range; and n represents the number of columns of the image stitching range;

[0025] the obtained wind farm tile data map has a quantity of m×n, and the wind farm tile data map is divided into n groups, and each group includes wind farm tile data map with a quantity of m.

[0026] vertically splicing the m wind farm tile data maps to obtain n groups of vertically spliced maps;

[0027] The grouping method of the wind farm tile data maps is to arrange and group the wind farm tile data maps in each group in descending order of image length with the same image width, and if the image lengths are the same, the wind farm tile data maps with the same image length are randomly sorted;

[0028] vertically splicing the n groups of vertically spliced maps to obtain a wind turbine spliced image.

[0029] Further, the wind turbine spliced image data is processed to obtain a multi-terrain and multi-resolution wind turbine dataset in S2, specifically

[0030] S21: manually identifying and labeling the wind turbine spliced image based on a labelme labeling tool to obtain a labeled image;

[0031] The labeling rule is to label the wind turbine body in the wind turbine spliced image with a rectangular frame, and to label the center point of the wind turbine wing and the wind turbine tower base as key points;

[0032] S22: sequentially performing data cutting, data screening and data augmentation on the labeled image to obtain a multi-terrain and multi-resolution wind turbine data map;

[0033] Data cutting: cutting at the center point of the rectangular labeled rectangular frame to obtain a plurality of cutting subgraphs of a preset resolution size;

[0034] Data screening: based on a pre-created data screening device, cutting subgraphs that are confirmed to have no wind turbines or incomplete wind turbine images are deleted, and cutting subgraphs in which the same wind turbine is close to the center of the image are retained;

[0035] The cutting subgraphs after data screening are divided into multiple types of ground surface subgraphs according to the ground surface environment;

[0036] The ground surface environment at least includes forest land, mountain land, grassland and wasteland;

[0037] Data augmentation: using a mean resampling image processing algorithm to perform data augmentation on each type of ground surface subgraph, that is, extracting every other pixel point and every other two pixel points to obtain different resolution subgraphs of the same image;

[0038] Further, a multi-terrain and multi-resolution wind turbine data map is obtained.

[0039] Further, the improved YOLOv5 neck network in S3 is that a neck end network layer for further sampling operation on the output feature map of the original YOLOv5 neck network is added to the upper layer of the original YOLOv5 neck network;

[0040] The neck end network layer is used for fusion with the output feature of the YOLOv5 backbone network;

[0041] The prediction end includes a plurality of ASFF-4 modules and a KeyDetect module connected in sequence for adaptive feature fusion;

[0042] The ASFF-4 module is used for weighted fusion of the output feature map of the improved YOLOv5 neck network to obtain adaptive fusion feature maps of different scales;

[0043] The output feature map includes four feature maps of different scales output by four C3 modules connected with the prediction end in the improved YOLOv5 neck network;

[0044] The KeyDetect module is used for fan detection and positioning of key points of the fan according to the output of the ASFF-4 module.

[0045] Further, the optimal fan detection network model in S4 is obtained by the following steps:

[0046] S41: input the data training set into the fan detection network model for model training to obtain the trained fan detection network model;

[0047] S42: verify the trained fan detection network model by the data test set;

[0048] That is, to determine whether the output of the trained fan detection network model converges;

[0049] If the output of the trained fan detection network model converges, it is confirmed that the trained fan detection network model at this time is the optimal fan detection network model;

[0050] Otherwise, based on the back propagation algorithm, the weight parameters of the trained fan detection network model are adaptively adjusted, and step S41 is repeatedly executed until the weight parameters of the trained fan detection network model whose output converges are the optimal weight parameters and the fan detection network model is reconstructed to obtain the optimal fan detection network model.

[0051] Further, the loss function loss & is

[0052] loss &= lambda1loss Obj + lambda2loss Conf + lambda3loss Class + lambda4loss key

[0053]

[0054] In the formula: lambda1, lambda2, lambda3, lambda4 represent weight coefficients; loss Obj loss represents fan positioning loss; loss Conf loss represents fan detection confidence loss; loss Class loss represents fan multi-surface multi-resolution classification loss; loss key loss represents fan key point detection loss; loss represents the fan key point coordinates predicted by the model; loss represents the real coordinates of the fan key point.

[0055] Beneficial effects: The application provides a fan detection method with multi-surface multi-resolution, which acquires a multi-surface multi-resolution fan dataset by image processing on fan spliced image data, so as to train the constructed fan detection network model by using fan data of different surface types and resolutions, so as to acquire an optimal fan detection network model, and facilitate the diversity detection and recognition accuracy of fans of different surface types and resolutions; and the output detection graph of the diversity detection of the fan features in the remote sensing image of the to-be-detected wind power plant is greatly improved. The application solves the problem that with the rapid development of the wind power industry, due to the influence of factors such as regional geology, climate environment, and total wind energy reserves, the wind power installation presents a regionalization and centralization trend, which leads to significant differences in the development of wind power in different regions, and the regional fan cannot be efficiently recognized; the application can detect and acquire multi-surface multi-resolution fan data in real time according to a small amount of fan data in the early stage, greatly reduces the waste of human resource costs, and ensures efficient recognition of regional fans while significantly improving the efficiency and accuracy of fan recognition. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0057] Figure 1 The flowchart of the fan detection method with multi-surface multi-resolution of the application;

[0058] Figure 2A core flow block diagram of the multi-terrain multi-resolution fan detection method in the embodiment;

[0059] Figure 3 A structural schematic diagram of the fan detection network model constructed in the embodiment;

[0060] Figure 4 An experimental result diagram of the multi-terrain multi-resolution fan detection in the embodiment. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in a clear and complete manner with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0062] The embodiment provides a multi-terrain multi-resolution fan detection method, as shown in the following formula: Figure 1 The method comprises the following steps:

[0063] S1: Obtain fan splicing image data;

[0064] The fan splicing image data comprises wind farm tile coordinates and fan contour features;

[0065] As shown in the following formula: Figure 2 The method comprises the following steps:

[0066] S11: Obtain multi-terrain multi-resolution wind farm coordinate information;

[0067] The wind farm coordinate information comprises coordinate information of multiple types of terrains and resolutions;

[0068] The coordinate information is subjected to Mercator coordinate conversion to obtain Google map wind farm tile coordinate information, and the map order of the Google map tile coordinate information is set, and the name of the wind farm tile data is "x_y_z";

[0069] The formula for the Mercator coordinate conversion is

[0070] n=2^zoom

[0071] x=n*((lon+180) / 360)

[0072] y=n*(1-(log(tan(lat)+sec(lat)) / π)) / 2

[0073] In the formula: x, y represent the horizontal and vertical coordinates of the Google map tile; z represents the tile level; lon and lat represent the longitude and latitude of the wind farm coordinate respectively;

[0074] S12: calling the Google API to obtain the wind farm tile data image according to the wind farm tile coordinate information and the map level;

[0075] S13: based on the image stitching strategy, the wind turbine stitching image data is obtained by stitching the wind farm tile data image; and the image stitching strategy is specifically

[0076] The stitching range m x n of image stitching is set, wherein m represents the number of rows of the image stitching range; and n represents the number of columns of the image stitching range;

[0077] The obtained m x n wind farm tile data images are divided into n groups, and each group includes m wind farm tile data images;

[0078] The m wind farm tile data images are vertically stitched to obtain n groups of vertically stitched images;

[0079] The grouping method of the wind farm tile data image is that the wind farm tile data images in each group are arranged in descending order according to the image length with the same image width; and if the image lengths are the same, the wind farm tile data images with the same image length are randomly sorted;

[0080] The n groups of vertically stitched images are horizontally stitched to obtain the wind turbine stitching image;

[0081] The vertical stitching and the horizontal stitching are the picture stitching according to the preset horizontal and vertical directions, which are well-known technical means and will not be described in detail here;

[0082] S2: performing image processing on the wind turbine stitching image data to obtain a wind turbine data set with multiple terrains and multiple resolutions, and dividing the wind turbine data set into a data training set and a data verification set according to a preset proportion;

[0083] Specifically, the wind turbine stitching image data is processed to obtain a wind turbine data set with multiple terrains and multiple resolutions, which includes the following steps:

[0084] S21: manually identifying and labeling the wind turbine stitching image based on the labelme labeling tool to obtain a labeled image;

[0085] The labeling rule is to label the wind turbine body in the wind turbine stitching image with a rectangular frame, and to label the center point of the wind turbine wing and the wind turbine tower base as key points;

[0086] The reason for selecting the two points as key points is that the fan wing center point and the fan tower base have obvious image features in the remote sensing image, and are less affected by remote sensing imaging and surface environment, so the fan wing center point and the fan tower base are respectively taken as key points for labeling;

[0087] S22: sequentially performing data cutting, data screening and data augmentation on the labeled image to obtain wind turbine data graphs of multiple terrains and multiple resolutions;

[0088] Data cutting: cutting the center point of the rectangular labeled rectangular frame to obtain a plurality of cutting subgraphs of preset resolution sizes; preferably, the preset resolution is 1024x1024;

[0089] Data screening: based on a pre-created data screening unit, cutting subgraphs in which no wind turbine exists or the wind turbine image is incomplete are deleted, and cutting subgraphs in which the wind turbine center is close to the image center when the same wind turbine exists in different cutting subgraphs are retained;

[0090] The pre-created data screening unit is a unit module set according to existing data screening technology, and is only used for screening data according to the existing data screening function. The data screening technology itself is not the invention point of the present application, and will not be described in detail here;

[0091] The cutting subgraphs after data screening are divided into multiple types of terrain subgraphs according to the terrain environment, and multiple types of terrain subgraphs are obtained;

[0092] The terrain environment at least includes forest land, mountain land, grassland and wasteland; and the classification of the multiple types of terrain subgraphs is 1:1:1:1 for forest land, mountain land, grassland and wasteland;

[0093] Data augmentation: using a mean resampling image processing algorithm to perform data augmentation on each type of terrain subgraph, that is, extracting every other pixel point and every other two pixel points to obtain different resolution subgraphs of the same image;

[0094] Further, wind turbine data graphs of multiple terrains and multiple resolutions are obtained;

[0095] S3: constructing a wind turbine detection network model;

[0096] As Figure 3As shown, the fan detection network model comprises a YOLOv5 backbone network for different scale extraction of fan features of multiple terrains and multiple resolutions, an improved YOLOv5 neck network for multi-scale feature fusion of fan features extracted by the backbone network, and a prediction end for fan detection of multiple terrains and multiple resolutions, and the prediction end is provided with an ASFF-4 module for adaptive feature fusion of output features of the improved YOLOv5 neck network;

[0097] Specifically, the improved YOLOv5 neck network is a neck end network layer added to the upper layer of the original YOLOv5 neck network for further sampling operation on the output feature map of the original YOLOv5 neck network.

[0098] The neck end network layer is used for fusion with the output features of the YOLOv5 backbone network, so that the model can better capture the feature information of the fan.

[0099] The prediction end comprises a plurality of ASFF-4 modules and a KeyDetect module connected in sequence for adaptive feature fusion.

[0100] The ASFF-4 module is used for weighted fusion of the output feature map of the improved YOLOv5 neck network to obtain adaptive fusion feature maps of different scales.

[0101] The output feature map comprises four feature maps of different scales output by four C3 modules connected with the prediction end in the improved YOLOv5 neck network; the C3 module is a C3 module in the original YOLOv5 neck network, and in this embodiment, only the number of C3 modules is increased to further implement the sampling operation of the neck end network layer.

[0102] Specifically, the different level features of the original YOLOv5 model are not fully fused, which can cause serious loss of fan image features in the ground environment, and therefore the ASFF-4 adaptive feature fusion module is added to solve the problem of fan feature loss.

[0103] The ASFF-4 module is explained as follows:

[0104] Y n =F ASFF-4 (P 2→n ,P 3→n ,P 4→n ,P 5→n )

[0105] Wherein, Y n is the feature map after the ASFF-4 model; the value of n is 2-5, P 2→n ,P 3→n ,P 4→nP 5→n represent the feature maps after scale unification; scale unification refers to, before being output to each ASFF-4 module, for P2, P3, P4, P5 four feature layers, i.e. four different scale feature maps output by the four C3 modules connected with the neck end network layer; since different feature layers have different sizes, scale unification needs to be performed first, and the feature map at level n is represented as P n , for different levels x, P x needs to be adjusted to the same size as P n , to obtain P x→n ;

[0106] After obtaining four feature maps of the same size, they are input into the ASFF-4 module for weighted fusion, and the fusion formula is as follows:

[0107] F ASFF-4 = α n × P 2→n + β n × P 3→n + γ n × P 4→n + δ n × P 5→n

[0108] Wherein, α n , β n , γ n , δ n are learning parameters learned by the network model, and are shared in the channel, and the embodiment defines that the four learning parameters satisfy:

[0109] α n + β n + γ n + δ n = 1

[0110] Each parameter takes a value in the range of 0-1 and satisfies:

[0111]

[0112] The formula is a softmax function, and the parameter weight is obtained through the softmax function, and this process can be learned through the standard back propagation algorithm (BP algorithm), and β n , γ n , δ n and α n are calculated in the same way, since BP algorithm is satisfied, the weight can obtain appropriate parameters through network training, and adaptive parameter adjustment is performed;

[0113] The KeyDetect module is used to realize fan detection and key point positioning of the fan according to the output of the ASFF-4 module.

[0114] Specifically, for the prediction end of the detection head end, the channel dimension of each output feature layer is expanded, and the output length is increased to 10, which is mainly realized by adding the coordinates of two groups of fan key points to the original channel dimension. Since each feature layer corresponds to three anchor boxes, the total output length in the whole channel dimension reaches 30. This improvement method can simultaneously and accurately locate the positions of the target bounding box and the key points, thereby realizing the synchronous processing of fan detection and key point positioning. Since the key point information of the fan is introduced, the key point loss is added in the YOLO5 loss, and the KeyDetect module is set;

[0115] S4: model training and model verification are performed on the fan detection network model based on the data training set and the data verification set to obtain an optimal fan detection network model;

[0116] The model training comprises: based on the YOLOv5 backbone network, extracting different scale fan features from the multi-ground and multi-resolution fan data in the data training set to obtain a multi-scale fan feature map;

[0117] Based on the improved YOLOv5 neck network, the multi-scale fan feature map is subjected to feature fusion training to obtain a fan fusion feature map of different scales;

[0118] Based on the ASFF-4 module in the prediction end, the fan fusion feature map of different scales is subjected to adaptive feature fusion to obtain an adaptive fusion feature map, which is used to realize the fan detection training of the multi-ground and multi-resolution;

[0119] The model verification comprises: verifying the fan detection network model after the model training by using the data verification set to obtain the optimal weight parameters of the fan detection network model and reconstruct the fan detection network model, thereby obtaining the optimal fan detection network model;

[0120] The optimal fan detection network model is obtained by the following steps:

[0121] S41: inputting the data training set into the fan detection network model for model training to obtain the fan detection network model after the training;

[0122] S42: verifying the fan detection network model after the training by using the data test set;

[0123] That is, whether the output of the fan detection network model after the training converges is determined;

[0124] If the output of the fan detection network model after the training converges, it is confirmed that the fan detection network model after the training at this time is the optimal fan detection network model;

[0125] Otherwise, based on the back propagation algorithm, the weight parameters of the trained wind turbine detection network model are adaptively adjusted, and step S41 is repeated until it is confirmed that the weight parameters of the trained wind turbine detection network model that have converged are the optimal weight parameters and the wind turbine detection network model is reconstructed to obtain the optimal wind turbine detection network model;

[0126] The loss function in S42 determines whether the output of the wind turbine detection network model after training converges. & for

[0127] loss & =λ1loss Obj +λ2loss Conf +λ3loss Class +λ4loss key

[0128]

[0129] Where: λ1,λ2,λ3,λ4 represent weight coefficients; loss Obj Indicates fan positioning loss; loss Conf Indicates the wind turbine detection confidence loss; loss Class Indicates the classification loss of wind turbines with multiple surfaces and multiple resolutions; loss key Indicates the loss of key point detection of the fan; Represents the coordinates of key points of the wind turbine predicted by the model; represents the true coordinates of the key points of the wind turbine; the λ1 parameter is set to 0.05, the λ2 parameter is set to 0.4225, the λ3 parameter is set to 0.125, and the λ4 parameter is set to 0.025. Because the loss of the key points of the wind turbine only works in the prediction box, its weight is small; the loss is based on the wing loss function, which is very sensitive to small errors and helps improve the network's ability to cope with small errors during training, thereby improving the accuracy of key point positioning;

[0130] S5: Obtain the remote sensing image of the wind farm to be detected, and perform segmentation processing on it to obtain several remote sensing sub-images. Based on the optimal wind turbine detection network model, perform wind turbine feature detection on each remote sensing sub-image to obtain the output detection map, that is, cut the wind farm image to be detected into 1024×1024 small images. Figure 1 The wind turbine detection images are input into the optimal wind turbine detection network model for identification and detection. The model outputs the detection images in sequence and splices them to generate the wind farm wind turbine detection images and wind turbine coordinates, so as to realize wind turbine detection on multiple surfaces and with multiple resolutions.

[0131] like Figure 4 There are 74 wind turbines in the remote sensing image of the wind farm shown.

[0132] Fan coordinates: (113.94886493648528E, 41.16947185963589N), confidence: 0.734785; Fan coordinates: (113.95386189651487E, 41.16941330707751N), confidence: 0.692275; Fan coordinates: (113.94509106788632E, 41.16213009122498N), confidence: 0.739686;

[0133] Fan coordinates: (113.94401013679504E, 41.15897788924774N), confidence: 0.671113; Fan coordinates: (113.93262147903442E, 41.17717418163292N), confidence: 0.746163; Fan coordinates: (113.93672526397705E, 41.17996415492976N), confidence: 0.740697;

[0134] Fan coordinates: (113.92947256736755E, 41.176719939629095N), confidence: 0.71142; Fan coordinates: (113.95129233169558E, 41.14485673689699N), confidence: 0.845698; Fan coordinates: (113.95771622314453E, 41.14452143721771N), confidence: 0.822277; Fan coordinates: (113.87741625137329E, 41.1613829467599N), confidence: 0.730238; Fan coordinates: (113.88051956634519E, 41.1607852266059N), confidence: 0.722215;

[0135] Fan coordinates: (113.87910604820252E, 41.16374551756207N), confidence: 0.720683; Fan coordinates: (113.91804635490416E, 41.124627001175N), confidence: 0.797804; Fan coordinates: (113.92097532920837E, 41.16779197277381N), confidence: 0.755559; Fan coordinates: (113.89520734634402E, 41.12904357167062N), confidence: 0.711692;

[0136] Fan coordinates: (113.92084389495852E, 41.160375295507585N), confidence: 0.819443; Fan coordinates: (113.9259991024017E, 41.16071657020764N), confidence: 0.800595; Fan coordinates: (113.92083317298892E, 41.15389890467738N), confidence: 0.75565;

[0137] Fan coordinates: (113.92723292198184E, 41.154213954492135N), confidence: 0.707532; Fan coordinates: (113.95440637893677E, 41.16295194495096N), confidence: 0.836703; Fan coordinates: (113.95039379768372E, 41.162273464694714N), confidence: 0.712345; Fan coordinates: (113.9349147651672E, 41.16855115491449N), confidence: 0.730213;

[0138] Fan coordinates: (113.92933577728269E, 41.168666245650556N), confidence: 0.712321; Fan coordinates: (113.94238740119931E, 41.17342105153266N), confidence: 0.7786; Fan coordinates: (113.93974274139401E, 41.17848035210261N), confidence: 0.732996; Fan coordinates: (113.91650408248898E, 41.16037731938633N), confidence: 0.822986;

[0139] Fan coordinates: (113.91090631828308E, 41.16110428328173N), confidence: 0.717739; Fan coordinates: (113.91590059318543E, 41.173853104845286N), confidence: 0.766677; Fan coordinates: (113.9115527263641E, 41.12829402525912N), confidence: 0.694032;

[0140] Fan coordinates: (113.90873104591373E, 41.12930822859406N), confidence: 0.682637; Fan coordinates: (113.89872104187015E, 41.12199833531127N), confidence: 0.759567; Fan coordinates: (113.90038669624329E, 41.12501896985209N), confidence: 0.712079;

[0141] Fan coordinates: (113.94146472473147E, 41.12445929984121N), confidence: 0.79548; Fan coordinates: (113.93873691558838E, 41.119909065092614N), confidence: 0.705838; Fan coordinates: (113.89296233482361E, 41.12423301292081N), confidence: 0.760336;

[0142] Fan coordinates: (113.93853843246461E, 41.12952844679393N), confidence: 0.691041; Fan coordinates: (113.92723559989929E, 41.179166742232574N), confidence: 0.77211; Fan coordinates: (113.95293116569519E, 41.1198686556321N), confidence: 0.743587;

[0143] Fan coordinates: (113.94892930950164E, 41.12478257549202N), confidence: 0.708825; Fan coordinates: (113.91689032058713E, 41.16769707051098N), confidence: 0.817047; Fan coordinates: (113.91114503402713E, 41.16737199045176N), confidence: 0.781014;

[0144] Fan coordinates: (113.90949278984067E, 41.17186442189882N), confidence: 0.731324; Fan coordinates: (113.94750774421692E, 41.1424773285917N), confidence: 0.799589; Fan coordinates: (113.93998951148984E, 41.14580201215505N), confidence: 0.79616; Fan coordinates: (113.90582621612549E, 41.16113658743154N), confidence: 0.744473;

[0145] Fan coordinates: (113.93279582176206E, 41.14655740817474N), confidence: 0.797951; Fan coordinates: (113.93198042678836E, 41.154547180548846N), confidence: 0.742836; Fan coordinates: (113.93895417709354E, 41.169994798649824N), confidence: 0.717601;

[0146] Fan coordinates: (113.94544780769348E, 41.167711205064236N), confidence: 0.701669; Fan coordinates: (113.94500255241394E, 41.13461133606578N), confidence: 0.83494; Fan coordinates: (113.9417383049011E, 41.14021499139993N), confidence: 0.793926; Fan coordinates: (113.9486047623825E, 41.17379455620248N), confidence: 0.751074; Fan coordinates: (113.95204871978763E, 41.17430332758506N), confidence: 0.715185;

[0147] Fan coordinates: (113.92001510009766E, 41.14226927721766N), confidence: 0.801251; Fan coordinates: (113.92689764671326E, 41.14757941074007N), confidence: 0.790204; Fan coordinates: (113.91946792945862E, 41.14837721021825N), confidence: 0.773909;

[0148] Fan coordinates: (113.92727047462466E, 41.14234603516638N), confidence: 0.663316; Fan coordinates: (113.93829435161592E, 41.15506417429399N), confidence: 0.800463; Fan coordinates: (113.90321642074582E, 41.15511466677445N), confidence: 0.807291; Fan coordinates: (113.92444878387454E, 41.1327830763661N), confidence: 0.737501;

[0149] Fan coordinates: (113.89897316780093E, 41.143923563308924N), confidence: 0.771235; Fan coordinates: (113.90032232322693E, 41.14767838292103N), confidence: 0.740479; Fan coordinates: (113.92837017791749E, 41.13676074458807N), confidence: 0.827413;

[0150] Fan coordinates: (113.93437028274536E, 41.13620925771664N), confidence: 0.795647; Fan coordinates: (113.93671184349063E, 41.139362554504864N), confidence: 0.778031; Fan coordinates: (113.90641898345945E, 41.17131929313413N), confidence: 0.75222; Fan coordinates: (113.93166392784116E, 41.16087003507087N), confidence: 0.695306;

[0151] The fan coordinates are: (113.91287773933414E, 41.13447800997315N), and the confidence is: 0.784006; the fan coordinates are: (113.91527027320859E, 41.14108761542199N), and the confidence is: 0.773354; the fan coordinates are: (113.9098361091614E, 41.13951607711643N), and the confidence is: 0.739539; the fan coordinates are: (113.91421347427371E, 41.15135421605612N), and the confidence is: 0.76855;

[0152] The fan coordinates are: (113.94857525791167E, 41.155534719897176N), and the confidence is: 0.826686; the fan coordinates are: (113.95458877601624E, 41.15383024106572N), and the confidence is: 0.820716; the fan coordinates are: (113.95371437416077E, 41.15001518741647N), and the confidence is: 0.771986.

[0153] In this embodiment, the wind turbine data set of multiple ground surfaces and multiple resolutions is obtained by image processing on the wind turbine spliced image data, so as to train the wind turbine detection network model constructed by wind turbine data of different ground surface types and resolutions, so as to obtain the optimal wind turbine detection network model, and facilitate the diversity detection and recognition accuracy of wind turbines of different ground surface types and resolutions; and the output detection graph of the diversity detection of the wind turbine features in the wind power plant remote sensing image to be detected is greatly improved. The embodiment solves the problem that with the rapid development of the wind power industry, due to the influence of factors such as regional geology, climate environment and total wind energy reserves, wind power installations show regionalization, centralization and other trends, resulting in significant differences in wind power development in different regions, and wind turbines in the region cannot be efficiently recognized. The embodiment can detect and obtain wind turbine data of multiple ground surfaces and multiple resolutions in real time according to a small amount of wind turbine data in the early stage, greatly reducing the waste of human resource costs, and significantly improving the efficiency and accuracy of wind turbine recognition while ensuring efficient recognition of regional wind turbines.

[0154] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-terrain multi-resolution fan detection method, characterized in that, The method comprises the following steps: S1: acquiring wind turbine splicing image data; The wind turbine splicing image data comprises wind farm tile coordinates and wind turbine contour features; S2: performing image processing on the wind turbine splicing image data to acquire a wind turbine dataset of multiple terrains and multiple resolutions, and dividing the wind turbine dataset into a data training set and a data verification set according to a preset ratio; S3: constructing a wind turbine detection network model; The wind turbine detection network model comprises a YOLOv5 backbone network for extracting wind turbine features of different scales, an improved YOLOv5 neck network for performing multi-scale feature fusion on the wind turbine features extracted by the backbone network, and a prediction end for wind turbine detection of multiple terrains and multiple resolutions, and the prediction end is provided with an ASFF-4 module for adaptively fusing the output features of the improved YOLOv5 neck network; The improved YOLOv5 neck network is a neck end network layer for further sampling operation on the output feature maps of the original YOLOv5 neck network added to the upper layer of the original YOLOv5 neck network; The neck end network layer is used for fusing the output features of the YOLOv5 backbone network; The prediction end comprises a plurality of ASFF-4 modules and a Key Detect module connected in sequence for adaptive feature fusion; The ASFF-4 module is used for weighting fusion on the output feature maps of the improved YOLOv5 neck network to obtain adaptive fusion feature maps of different scales; The output feature maps comprise four feature maps of different scales output by four C3 modules connected with the prediction end in the improved YOLOv5 neck network; The Key Detect module is used for realizing wind turbine detection and positioning of wind turbine key points according to the output of the ASFF-4 module; S4: model training and model verification are performed on the wind turbine detection network model based on the data training set and the data verification set to obtain an optimal wind turbine detection network model; The model training comprises: extracting wind turbine features of different scales from the wind turbine data of multiple terrains and multiple resolutions in the data training set based on the YOLOv5 backbone network to obtain multi-scale wind turbine feature maps; Performing feature fusion training on the multi-scale wind turbine feature maps based on the improved YOLOv5 neck network to obtain wind turbine fusion feature maps of different scales; Performing adaptive feature fusion on the wind turbine fusion feature maps of different scales based on the ASFF-4 module in the prediction end to obtain adaptive fusion feature maps for realizing wind turbine detection training of multiple terrains and multiple resolutions; The model verification comprises: verifying the wind turbine detection network model after model training through the data verification set to obtain optimal weight parameters of the wind turbine detection network model and reconstruct the wind turbine detection network model to obtain the optimal wind turbine detection network model; S5: acquiring a remote sensing image of a wind farm to be detected, performing segmentation processing to obtain a plurality of remote sensing sub-images, and performing wind turbine feature detection on each remote sensing sub-image based on the optimal wind turbine detection network model to obtain an output detection image.

2. The multi-terrain multi-resolution fan detection method of claim 1, wherein, The S1 specifically comprises the following steps: S11: acquiring wind farm coordinate information of multiple terrains and multiple resolutions; The wind farm coordinate information includes coordinate information of multiple types of ground surfaces and resolutions; The coordinate information is converted into Mercator coordinates to obtain wind farm tile coordinate information of Google Maps, and a map level of the wind farm tile coordinate information is set; S12: calling a Google API to obtain wind farm tile data based on the wind farm tile coordinate information and the map level; S13: based on an image stitching strategy, the wind farm tile data is stitched to obtain wind turbine stitching image data; The image stitching strategy is Setting a stitching range of image stitching wherein represents a number of rows of the image stitching range; represents a number of columns of the image stitching range; The acquired quantity of wind farm tile data graphs is The wind farm tile data graphs are divided into groups, and each group includes a quantity of wind farm tile data graphs. The quantity of wind farm tile data graphs is Longitudinal splicing is performed on the wind farm tile data graphs to obtain Longitudinal splicing graphs The grouping method of the wind farm tile data is to arrange the wind farm tile data in each group in descending order of image length while keeping the image width the same; if the image lengths are the same, the wind farm tile data with the same image length is randomly sorted; Again to The longitudinal spliced images are transversely spliced to obtain a fan spliced image.

3. The multi-terrain multi-resolution fan detection method of claim 2, wherein, S2: the wind turbine stitching image data is processed to obtain a wind turbine dataset with multiple ground surfaces and resolutions, which includes the following steps: S21: based on a labelme labeling tool, the wind turbine stitching image is manually identified and labeled according to a labeling rule to obtain a labeled image; The labeling rule is to label the wind turbine body in the wind turbine stitching image with a rectangular frame, and to label the center point of the wind turbine wing and the wind turbine tower base as key points; S22: the labeled image is sequentially cut, filtered, and augmented to obtain a wind turbine data graph with multiple ground surfaces and resolutions; Data cutting: the center point of the rectangular frame of the rectangular label is cut to obtain a plurality of cutting subgraphs with a preset resolution size; Data filtering: based on a pre-created data filter, cutting subgraphs that are confirmed to have no wind turbines or incomplete wind turbine images are deleted, and cutting subgraphs in which the same wind turbine is close to the center of the image are retained; The cutting subgraphs after data filtering are divided into multiple ground surface types according to the ground environment to obtain multiple ground surface subgraphs; The ground environment includes at least forest land, mountainous land, grassland, and wasteland; Data augmentation: a mean resampling image processing algorithm is used to augment the data of each ground surface subgraph, i.e., every other pixel is extracted and every other two pixels are extracted to obtain different resolution subgraphs of the same image. Thus, the wind turbine data graph with multiple ground surfaces and resolutions is obtained.

4. The multi-surface multi-resolution fan detection method of claim 3, wherein, S4: obtaining an optimal wind turbine detection network model, which includes the following steps: S41: inputting the data training set into the wind turbine detection network model for model training to obtain a trained wind turbine detection network model; S42: verifying the trained wind turbine detection network model through the data verification set; That is, whether the output of the trained wind turbine detection network model converges is determined; If the output of the trained wind turbine detection network model converges, the trained wind turbine detection network model at this time is confirmed to be the optimal wind turbine detection network model. Otherwise, based on the back propagation algorithm, the weight parameters of the trained fan detection network model are adaptively adjusted, and step S41 is repeatedly executed until the weight parameters of the trained fan detection network model that is confirmed to be converged are optimal weight parameters and the fan detection network model is reconstructed to obtain an optimal fan detection network model.

5. The multi-terrain multi-resolution fan detection method of claim 4, wherein, The loss function for judging whether the output of the trained fan detection network model converges in S42 For In the formula: , , , represents a weight coefficient; represents a fan positioning loss; represents a fan detection confidence loss; represents a fan multi-surface multi-resolution classification loss; represents a fan key point detection loss; represents a fan key point coordinate predicted by a model; represents a real coordinate of a fan key point.

Images