Multi-surface multi-resolution fan detection method

Abstract

The invention discloses a multi-surface multi-resolution fan detection method. The method comprises the following steps: S1, acquiring fan spliced image data; s2, performing image processing on the fan spliced image data to obtain a multi-surface multi-resolution fan data set, and dividing the fan data set into a data training set and a data verification set according to a preset proportion; s3, constructing a fan detection network model; s4, performing model training and model verification on the fan detection network model based on the data training set and the data verification set to obtain an optimal fan detection network model; and S5, obtaining a to-be-detected wind power plant remote sensing image, carrying out segmentation processing on the to-be-detected wind power plant remote sensing image to obtain a plurality of remote sensing sub-images, and carrying out fan feature detection on each remote sensing sub-image based on the optimal fan detection network model to obtain an output detection image. The problems that along with rapid development of the wind power industry, due to the influence of factors such as geology, climate environment and total wind energy reserves of all regions, wind power installation tends to be regionalized and centralized, wind power development differences of all the regions are remarkable, and regional fans cannot be efficiently recognized are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of wind turbine detection, and in particular to a multi-surface and multi-resolution wind turbine detection method. Background Art

[0003] There are currently two main ways to obtain wind turbine data: the first is to obtain the number and distribution information of wind turbines based on the data provided by the wind farm; the second is to obtain the number and location information of wind turbines by manual identification based on remote sensing images. Both methods have great limitations and greatly waste human resources; the first method requires irregular and real-time communication with major wind farms to obtain wind turbine data, and cannot detect and obtain multi-surface and multi-resolution wind turbine data in real time based on a small amount of wind turbine data in the early stage. The second method requires human recognition of images, which is time-consuming and labor-intensive, and partial errors may be generated through manual identification. Therefore, it is increasingly important to develop a multi-surface and multi-resolution wind turbine detection technology to address the above problems. Summary of the invention

[0004] The present invention provides a multi-surface and multi-resolution wind turbine detection method to overcome the above technical problems.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A multi-surface and multi-resolution wind turbine detection method comprises the following steps:

[0007] S1: Obtain wind turbine stitching image data;

[0008] The wind turbine splicing image data includes wind farm tile coordinates and wind turbine contour features;

[0009] S2: performing image processing on the wind turbine stitching image data to obtain a wind turbine data set with multiple surfaces and multiple resolutions, and dividing the wind turbine data set into a data training set and a data verification set according to a preset ratio;

[0010] S3: Build a wind turbine detection network model;

[0011] The wind turbine detection network model includes a YOLOv5 backbone network for extracting wind turbine features of multiple surfaces and multiple resolutions at different scales, an improved YOLOv5 neck network for performing multi-scale feature fusion on wind turbine features extracted by the backbone network, and a prediction end for wind turbine detection of multiple surfaces and multiple resolutions, and the prediction end is provided with an ASFF-4 module for adaptive feature fusion of output features of the improved YOLOv5 neck network;

[0012] S4: Perform model training and model verification on the wind turbine detection network model based on the data training set and the data verification set to obtain the optimal wind turbine detection network model;

[0013] The model training includes: extracting wind turbine features of different scales from wind turbine data of multiple surfaces and multiple resolutions in a data training set based on a YOLOv5 backbone network to obtain a multi-scale wind turbine feature map;

[0014] Based on the improved YOLOv5 neck network, feature fusion training is performed on multi-scale fan feature maps to obtain fan fusion feature maps of different scales;

[0015] Based on the ASFF-4 module in the prediction end, adaptive feature fusion is performed on wind turbine fusion feature maps of different scales to obtain adaptive fusion feature maps for wind turbine detection training on multiple surfaces and multiple resolutions.

[0016] The model verification includes: verifying the wind turbine detection network model after model training through a data verification set to obtain the optimal weight parameters of the wind turbine detection network model and reconstruct the wind turbine detection network model to obtain the optimal wind turbine detection network model;

[0017] S5: Obtain a remote sensing image of the wind power plant to be detected, and perform segmentation processing on it to obtain a number of remote sensing sub-images, and perform wind turbine feature detection on each remote sensing sub-image based on the optimal wind turbine detection network model to obtain an output detection image.

[0018] Furthermore, the S1 specifically includes the following steps:

[0019] S11: Acquire wind farm coordinate information with multiple surfaces and multiple resolutions;

[0020] The wind farm coordinate information includes coordinate information of various types of surfaces and resolutions;

[0021] And perform Mercator coordinate conversion on it to obtain Google map wind farm tile coordinate information, and set the map level of Google map tile coordinate information;

[0022] S12: calling Google API to obtain a wind farm tile data map according to wind farm tile coordinate information and map level;

[0023] S13: based on the image stitching strategy, stitching the wind farm tile data map to obtain wind turbine stitching image data;

[0024] And the image stitching strategy is specifically:

[0025] Set the image stitching range m×n, where m represents the number of rows in the image stitching range; n represents the number of columns in the image stitching range;

[0026] The obtained wind farm tile data graphs are m×n in number, and the wind farm tile data graphs are divided into n groups, and each group includes m wind farm tile data graphs;

[0027] Vertically splice m wind farm tile data images to obtain n groups of vertically spliced ​​images;

[0028] The wind farm tile data graphs are grouped in a manner that wind farm tile data graphs in each group are arranged in descending order according to image lengths with the same image width; if the image lengths are the same, the wind farm tile data graphs with the same image lengths are randomly sorted;

[0029] Then, the n groups of longitudinal mosaic images are horizontally mosaicked to obtain the wind turbine mosaic image.

[0030] Furthermore, the wind turbine mosaic image data is processed as described in S2 to obtain a wind turbine dataset with multiple surfaces and multiple resolutions, specifically:

[0031] S21: Manually identify and annotate the wind turbine splicing image based on the labelme annotation tool to obtain an annotated image;

[0032] The marking rule is to use a rectangular frame to mark the wind turbine body in the wind turbine splicing image, and mark the center point of the wind turbine wing and the wind turbine tower base as key points;

[0033] S22: performing data cutting, data screening and data expansion on the annotated image in sequence to obtain a wind turbine data map with multiple surfaces and multiple resolutions;

[0034] Data cutting: Cutting is performed at the center point of the rectangular frame marked with a rectangle to obtain several cut sub-images of preset resolution size;

[0035] Data filtering: Based on the pre-created data filter, the cut sub-graphs that are confirmed to not exist or have incomplete fan images will be deleted, and the cut sub-graphs where the fan center is close to the image center will be retained when the same fan exists in different cut sub-graphs;

[0036] The cut sub-maps after data screening are divided into surface types according to the surface environment to obtain multiple types of surface sub-maps;

[0037] The surface environment at least includes forest land, mountain land, grassland and wasteland;

[0038] Data expansion: The mean resampling image processing algorithm is used to expand the data of each type of surface sub-image, that is, extracting every one pixel and every two pixels, respectively, to obtain sub-images of different resolutions of the same image;

[0039] Then, wind turbine data maps with multiple surfaces and multiple resolutions are obtained.

[0040] Furthermore, the improved YOLOv5 neck network described in S3 is to add a neck-end network layer for further sampling the output feature map of the original YOLOv5 neck network to the upper layer of the original YOLOv5 neck network;

[0041] The neck end network layer is used to fuse the output features of the YOLOv5 backbone network;

[0042] The prediction end includes a plurality of ASFF-4 modules and a KeyDetect module connected in sequence for adaptive feature fusion;

[0043] The ASFF-4 module is used to perform weighted fusion on the output feature map of the improved YOLOv5 neck network to obtain adaptive fusion feature maps of different scales;

[0044] The output feature map includes four feature maps of different scales output by four C3 modules connected to the prediction end in the improved YOLOv5 neck network;

[0045] The Key Detect module is used to detect the fan and locate the key points of the fan according to the output of the ASFF-4 module.

[0046] Furthermore, obtaining the optimal wind turbine detection network model described in S4 specifically includes the following steps:

[0047] S41: inputting the data training set into the wind turbine detection network model for model training, and obtaining the trained wind turbine detection network model;

[0048] S42: verifying the trained wind turbine detection network model through the data test set;

[0049] That is, to determine whether the output of the trained wind turbine detection network model converges;

[0050] If the output of the trained fan detection network model converges, it is confirmed that the trained fan detection network model is the optimal fan detection network model;

[0051] Otherwise, based on the back propagation algorithm, the weight parameters of the trained wind turbine detection network model are adaptively adjusted, and step S41 is repeated until it is confirmed that the weight parameters of the trained wind turbine detection network model with converged output are the optimal weight parameters and the wind turbine detection network model is reconstructed to obtain the optimal wind turbine detection network model.

[0052] Furthermore, in S42, the loss function loss for judging whether the output of the trained wind turbine detection network model converges is & for

[0053] loss & =λ1loss Obj +λ2loss Conf +λ3loss Class +λ4loss key

[0054]

[0055] Where: λ1,λ2,λ3,λ4 represent weight coefficients; loss Obj Indicates the fan positioning loss; loss Conf Indicates the confidence loss of fan detection; loss Class Indicates the classification loss of wind turbines with multiple surfaces and multiple resolutions; loss key Indicates the loss of key point detection of the fan; Represents the coordinates of the key points of the wind turbine predicted by the model; Indicates the real coordinates of the key points of the fan.

[0056] Beneficial effects: The present invention provides a multi-surface multi-resolution wind turbine detection method, which obtains a multi-surface multi-resolution wind turbine data set by performing image processing on wind turbine splicing image data, so as to train the constructed wind turbine detection network model through wind turbine data of different surface types and resolutions, so as to obtain the optimal wind turbine detection network model, and facilitate the realization of the diversity detection and identification accuracy of wind turbines of different surface types and resolutions; and greatly improves the diversity detection of wind turbine features in the remote sensing image of the wind power plant to be detected to obtain the output detection map. The present invention solves the problem that with the rapid development of the wind power industry, due to the influence of factors such as the geology, climate environment and total wind energy reserves in various regions, wind power installation has shown a trend of regionalization and centralization, resulting in significant differences in wind power development in various regions, and it is impossible to efficiently identify regional wind turbines. The present invention can detect and obtain multi-surface multi-resolution wind turbine data in real time based on a small amount of wind turbine data in the early stage, which greatly reduces the waste of human resource costs and significantly improves the efficiency and accuracy of wind turbine identification while ensuring efficient identification of regional wind turbines. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0058] Figure 1 It is a flow chart of the wind turbine detection method with multiple surfaces and multiple resolutions of the present invention;

[0059] Figure 2 This is a core flow chart of the multi-surface and multi-resolution wind turbine detection method in this embodiment;

[0060] Figure 3 It is a structural schematic diagram of the wind turbine detection network model constructed in this embodiment;

[0061] Figure 4 Graph showing experimental results of wind turbine detection at multiple surfaces and multiple resolutions in this embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] This embodiment provides a multi-surface and multi-resolution wind turbine detection method, such as Figure 1 As shown, the following steps are included:

[0064] S1: Obtain wind turbine stitching image data;

[0065] The wind turbine splicing image data includes wind farm tile coordinates and wind turbine contour features;

[0066] like Figure 2 As shown, the specific steps include:

[0067] S11: Acquire wind farm coordinate information with multiple surfaces and multiple resolutions;

[0068] The wind farm coordinate information includes coordinate information of various types of surfaces and resolutions;

[0069] And perform Mercator coordinate conversion on it to obtain Google map wind farm tile coordinate information, and set the map level of Google map tile coordinate information, and the name of the wind farm tile data is "x_y_z";

[0070] Among them, the formula for Mercator coordinate transformation is

[0071] n=2^zoom

[0072] x = n * ((lon + 180) / 360)

[0073] y=n*(1-(log(tan(lat)+sec(lat)) / π)) / 2

[0074] Where: x, y represent the horizontal and vertical coordinates of the Google map tile; z represents the tile level; lon, lat represent the longitude and latitude of the wind farm coordinates respectively;

[0075] S12: calling Google API to obtain a wind farm tile data map according to wind farm tile coordinate information and map level;

[0076] S13: Based on the image stitching strategy, stitching the wind farm tile data map to obtain wind turbine stitching image data; and the image stitching strategy is specifically:

[0077] Set the image stitching range m×n, where m represents the number of rows in the image stitching range; n represents the number of columns in the image stitching range;

[0078] The obtained wind farm tile data graphs are m×n in number, and the wind farm tile data graphs are divided into n groups, and each group includes m wind farm tile data graphs;

[0079] Vertically splice m wind farm tile data images to obtain n groups of vertically spliced ​​images;

[0080] The wind farm tile data graphs are grouped in a manner that wind farm tile data graphs in each group are arranged in descending order according to image lengths with the same image width; if the image lengths are the same, the wind farm tile data graphs with the same image lengths are randomly sorted;

[0081] Then, the n groups of longitudinal mosaic images are horizontally mosaicked to obtain a fan mosaic image;

[0082] The longitudinal stitching and the transverse stitching are to stitch pictures according to the preset transverse and longitudinal directions, which are well-known technical means and will not be described in detail here;

[0083] S2: performing image processing on the wind turbine stitching image data to obtain a wind turbine data set with multiple surfaces and multiple resolutions, and dividing the wind turbine data set into a data training set and a data verification set according to a preset ratio;

[0084] Specifically, image processing is performed on the wind turbine stitching image data to obtain a wind turbine data set with multiple surfaces and multiple resolutions, which specifically includes the following steps:

[0085] S21: Manually identify and annotate the wind turbine splicing image based on the labelme annotation tool to obtain an annotated image;

[0086] The marking rule is to use a rectangular frame to mark the wind turbine body in the wind turbine splicing image, and mark the center point of the wind turbine wing and the wind turbine tower base as key points;

[0087] The reasons for selecting these two points as key points are as follows: the center point of the wind turbine wing and the base of the wind turbine tower have obvious image features in the remote sensing image and are less affected by remote sensing imaging and the surface environment. Therefore, the center point of the wind turbine wing and the base of the wind turbine tower are marked as key points respectively;

[0088] S22: performing data cutting, data screening and data expansion on the annotated image in sequence to obtain a wind turbine data map with multiple surfaces and multiple resolutions;

[0089] Data cutting: cutting is performed at the center point of the rectangular marked rectangle to obtain several cutting sub-images of preset resolution size; the preferred preset resolution is 1024×1024;

[0090] Data filtering: Based on the pre-created data filter, the cut sub-graphs that are confirmed to not exist or have incomplete fan images will be deleted, and the cut sub-graphs where the fan center is close to the image center will be retained when the same fan exists in different cut sub-graphs;

[0091] The pre-created data filter is a unit module set according to the existing data screening technology, which is only used to implement data screening according to the existing data screening function. The data screening technology itself is not the invention point of the present application and will not be described in detail here.

[0092] The cut sub-maps after data screening are divided into surface types according to the surface environment to obtain multiple types of surface sub-maps;

[0093] The surface environment at least includes forest, mountain, grassland and wasteland; and the classification of the multi-class surface sub-map is forest surface class: mountain surface class: grassland surface class: wasteland surface class in a ratio of 1:1:1:1;

[0094] Data expansion: The mean resampling image processing algorithm is used to expand the data of each type of surface sub-image, that is, extracting every one pixel and every two pixels, respectively, to obtain sub-images of different resolutions of the same image;

[0095] Then, wind turbine data maps with multiple surfaces and multiple resolutions are obtained;

[0096] S3: Build a wind turbine detection network model;

[0097] like Figure 3As shown, the wind turbine detection network model includes a YOLOv5 backbone network for extracting wind turbine features of multiple surfaces and multiple resolutions at different scales, an improved YOLOv5 neck network for performing multi-scale feature fusion on wind turbine features extracted by the backbone network, and a prediction end for wind turbine detection of multiple surfaces and multiple resolutions, and the prediction end is provided with an ASFF-4 module for adaptive feature fusion of output features of the improved YOLOv5 neck network;

[0098] Specifically, the improved YOLOv5 neck network is to add a neck-end network layer for further sampling the output feature map of the original YOLOv5 neck network to the upper layer of the original YOLOv5 neck network;

[0099] The neck end network layer is used to fuse the output features of the YOLOv5 backbone network, so that the model can better capture the characteristic information of the wind turbine;

[0100] The prediction end includes a plurality of ASFF-4 modules and a KeyDetect module connected in sequence for adaptive feature fusion;

[0101] The ASFF-4 module is used to perform weighted fusion on the output feature map of the improved YOLOv5 neck network to obtain adaptive fusion feature maps of different scales;

[0102] The output feature map includes four feature maps of different scales output by four C3 modules connected to the prediction end in the improved YOLOv5 neck network; the C3 module is the C3 module in the original YOLOv5 neck network, and in this embodiment, it is only used to increase the number of C3 modules to further implement the sampling operation of the neck end network layer;

[0103] Specifically, the original YOLOv5 model did not fully fuse the features of different levels, which would lead to serious loss of image features of wind turbines in the surface environment. Therefore, the ASFF-4 adaptive feature fusion module was added to solve the problem of wind turbine feature loss.

[0104] The ASFF-4 modules are explained as follows:

[0105] Y n =F ASFF-4 (P 2→n ,P 3→n ,P 4→n ,P 5→n )

[0106] Among them, Y n is the feature map after the ASFF-4 model; the value range of n is 2-5, P 2→n ,P 3→n ,P 4→n,P 5→n Represents the feature map after scale unification; the so-called scale unification refers to the four feature maps of different scales output by the four feature layers P2, P3, P4, and P5, i.e., the four C3 modules connected to the neck network layer, before being output to each ASFF-4 module; due to the different sizes of different feature layers, it is necessary to unify the scales first, and the feature map at level n is represented as P n , for different levels x, P x Adjust to and P n The same size, get P x→n ;

[0107] After obtaining four feature maps of the same size, they are input into the ASFF-4 module for weighted fusion. The fusion formula is as follows:

[0108] F ASFF-4 =α n ×P 2→n +β n ×P 3→n +γ n ×P 4→n +δ n ×P 5→n

[0109] Among them, α n ,β n ,γ n ,δ n The learning parameters are obtained by learning the network model and are shared in the channel. This embodiment defines four learning parameters that satisfy:

[0110] α n +β n +γ n +δ n =1

[0111] Each parameter has a value range between 0 and 1 and satisfies:

[0112]

[0113] This formula is a softmax function, and the parameter weights are obtained through the softmax function. This process can be learned through the standard back propagation algorithm (BP algorithm), and β n , γ n , δ n With α n The calculation method is the same. Since the BP algorithm is satisfied, the weight can obtain appropriate parameters through network training and adjust the parameters adaptively.

[0114] The KeyDetect module is used to realize fan detection and positioning of key points of the fan according to the output of the ASFF-4 module;

[0115] Specifically, for the detection head end of the prediction end, the channel dimension of each output feature layer is expanded, and the output length is increased to 10. This is mainly achieved by adding two sets of coordinates of the key points of the fan on the basis of the original channel dimension; since each feature layer corresponds to three anchor boxes, the total output length on the entire channel dimension reaches 30; this improvement method can synchronously and accurately locate the target bounding box and the position of the key points, thereby realizing the synchronous processing of fan detection and key point positioning; due to the introduction of the key point information of the fan, the key point loss is added to the YOLO5 loss, and the KeyDetect module is set;

[0116] S4: Perform model training and model verification on the wind turbine detection network model based on the data training set and the data verification set to obtain the optimal wind turbine detection network model;

[0117] The model training includes: extracting wind turbine features of different scales from wind turbine data of multiple surfaces and multiple resolutions in a data training set based on a YOLOv5 backbone network to obtain a multi-scale wind turbine feature map;

[0118] Based on the improved YOLOv5 neck network, feature fusion training is performed on multi-scale fan feature maps to obtain fan fusion feature maps of different scales;

[0119] Based on the ASFF-4 module in the prediction end, adaptive feature fusion is performed on wind turbine fusion feature maps of different scales to obtain adaptive fusion feature maps for wind turbine detection training on multiple surfaces and multiple resolutions.

[0120] The model verification includes: verifying the wind turbine detection network model after model training through a data verification set to obtain the optimal weight parameters of the wind turbine detection network model and reconstruct the wind turbine detection network model to obtain the optimal wind turbine detection network model;

[0121] The obtaining of the optimal wind turbine detection network model specifically includes the following steps:

[0122] S41: inputting the data training set into the wind turbine detection network model for model training, and obtaining the trained wind turbine detection network model;

[0123] S42: verifying the trained wind turbine detection network model through the data test set;

[0124] That is, to determine whether the output of the trained wind turbine detection network model converges;

[0125] If the output of the trained fan detection network model converges, it is confirmed that the trained fan detection network model is the optimal fan detection network model;

[0126] Otherwise, based on the back propagation algorithm, the weight parameters of the trained wind turbine detection network model are adaptively adjusted, and step S41 is repeatedly performed until it is confirmed that the weight parameters of the trained wind turbine detection network model that have been output converged are the optimal weight parameters and the wind turbine detection network model is reconstructed to obtain the optimal wind turbine detection network model;

[0127] The loss function in S42 that determines whether the output of the trained wind turbine detection network model converges & for

[0128] loss & =λ1loss Obj +λ2loss Conf +λ3loss Class +λ4loss key

[0129]

[0130] Where: λ1,λ2,λ3,λ4 represent weight coefficients; loss Obj Indicates the fan positioning loss; loss Conf Indicates the confidence loss of fan detection; loss Class Indicates the classification loss of wind turbines with multiple surfaces and multiple resolutions; loss key Indicates the loss of key point detection of the fan; Represents the coordinates of the key points of the wind turbine predicted by the model; represents the real coordinates of the key points of the wind turbine; the λ1 parameter is set to 0.05, the λ2 parameter is set to 0.4225, the λ3 parameter is set to 0.125, and the λ4 parameter is set to 0.025, because the key point loss of the wind turbine only works in the prediction box, so the weight is small; the loss loss is based on the wing loss function, which is very sensitive to small errors and helps to improve the network's ability to cope with small errors during training, thereby improving the accuracy of key point positioning;

[0131] S5: Obtain the remote sensing image of the wind farm to be detected, and perform segmentation processing on it to obtain several remote sensing sub-images. Based on the optimal wind turbine detection network model, perform wind turbine feature detection on each remote sensing sub-image to obtain an output detection map, that is, cut the wind farm image to be detected into 1024×1024 small pixels. Figure 1 The wind turbine detection images are input into the optimal wind turbine detection network model for identification and detection. The model outputs the detection images in turn and splices them to generate wind farm wind turbine detection images and wind turbine coordinates to achieve wind turbine detection on multiple surfaces and with multiple resolutions.

[0132] like Figure 4 There are 74 wind turbines in the remote sensing image of the wind farm shown.

[0133] The coordinates of the fan are: (113.94886493648528E, 41.16947185963589N), the confidence level is: 0.734785; the coordinates of the fan are: (113.95386189651487E, 41.16941330707751N), the confidence level is: 0.692275; the coordinates of the fan are: (113.94509106788632E, 41.16213009122498N), the confidence level is: 0.739686;

[0134] The coordinates of the fan are: (113.94401013679504E, 41.15897788924774N), the confidence level is: 0.671113; the coordinates of the fan are: (113.93262147903442E, 41.17717418163292N), the confidence level is: 0.746163; the coordinates of the fan are: (113.93672526397705E, 41.17996415492976N), the confidence level is: 0.740697;

[0135] The coordinates of the fan are: (113.92947256736755E, 41.176719939629095N), the confidence level is: 0.71142; the coordinates of the fan are: (113.95129233169558E, 41.14485673689699N), the confidence level is: 0.845698; the coordinates of the fan are: (113.95771622314453E, 41 .14452143721771N), the confidence level is 0.822277; the wind turbine coordinates are (113.87741625137329E, 41.1613829467599N), the confidence level is 0.730238; the wind turbine coordinates are (113.88051956634519E, 41.1607852266059N), the confidence level is 0.722215;

[0136] The coordinates of the fan are: (113.87910604820252E, 41.16374551756207N), the confidence level is: 0.720683; the coordinates of the fan are: (113.91804635490416E, 41.124627001175N), the confidence level is: 0.797804; the coordinates of the fan are: (113.92097532920837E, 41.16779197277381N), the confidence level is: 0.755559; the coordinates of the fan are: (113.89520734634402E, 41.12904357167062N), the confidence level is: 0.711692;

[0137] The coordinates of the fan are: (113.92084389495852E, 41.160375295507585N), the confidence level is: 0.819443; the coordinates of the fan are: (113.9259991024017E, 41.16071657020764N), the confidence level is: 0.800595; the coordinates of the fan are: (113.92083317298892E, 41.15389890467738N), the confidence level is: 0.75565;

[0138] The coordinates of the fan are: (113.92723292198184E, 41.154213954492135N), the confidence level is: 0.707532; the coordinates of the fan are: (113.95440637893677E, 41.16295194495096N), the confidence level is: 0.836703; the coordinates of the fan are: (113.95039379768372E, 41.162273464694714N), the confidence level is: 0.712345; the coordinates of the fan are: (113.9349147651672E, 41.16855115491449N), the confidence level is: 0.730213;

[0139] The coordinates of the fan are: (113.92933577728269E, 41.168666245650556N), the confidence level is: 0.712321; the coordinates of the fan are: (113.94238740119931E, 41.17342105153266N), the confidence level is: 0.7786; the coordinates of the fan are: (113.93974274139401E, 41.17848035210261N), the confidence level is: 0.732996; the coordinates of the fan are: (113.91650408248898E, 41.16037731938633N), the confidence level is: 0.822986;

[0140] The coordinates of the fan are: (113.91090631828308E, 41.16110428328173N), the confidence level is: 0.717739; the coordinates of the fan are: (113.91590059318543E, 41.173853104845286N), the confidence level is: 0.766677; the coordinates of the fan are: (113.9115527263641E, 41.12829402525912N), the confidence level is: 0.694032;

[0141] The coordinates of the fan are: (113.90873104591373E, 41.12930822859406N), the confidence level is: 0.682637; the coordinates of the fan are: (113.89872104187015E, 41.12199833531127N), the confidence level is: 0.759567; the coordinates of the fan are: (113.90038669624329E, 41.12501896985209N), the confidence level is: 0.712079;

[0142] The coordinates of the fan are: (113.94146472473147E, 41.12445929984121N), the confidence level is: 0.79548; the coordinates of the fan are: (113.93873691558838E, 41.119909065092614N), the confidence level is: 0.705838; the coordinates of the fan are: (113.89296233482361E, 41.12423301292081N), the confidence level is: 0.760336;

[0143] The coordinates of the fan are: (113.93853843246461E, 41.12952844679393N), the confidence level is: 0.691041; the coordinates of the fan are: (113.92723559989929E, 41.179166742232574N), the confidence level is: 0.77211; the coordinates of the fan are: (113.95293116569519E, 41.1198686556321N), the confidence level is: 0.743587;

[0144] The coordinates of the fan are: (113.94892930950164E, 41.12478257549202N), the confidence level is: 0.708825; the coordinates of the fan are: (113.91689032058713E, 41.16769707051098N), the confidence level is: 0.817047; the coordinates of the fan are: (113.91114503402713E, 41.16737199045176N), the confidence level is: 0.781014;

[0145] The coordinates of the fan are: (113.90949278984067E, 41.17186442189882N), the confidence level is: 0.731324; the coordinates of the fan are: (113.94750774421692E, 41.1424773285917N), the confidence level is: 0.799589; the coordinates of the fan are: (113.93998951148984E, 41.14580201215505N), the confidence level is: 0.79616; the coordinates of the fan are: (113.90582621612549E, 41.16113658743154N), the confidence level is: 0.744473;

[0146] The coordinates of the fan are: (113.93279582176206E, 41.14655740817474N), the confidence level is: 0.797951; the coordinates of the fan are: (113.93198042678836E, 41.154547180548846N), the confidence level is: 0.742836; the coordinates of the fan are: (113.93895417709354E, 41.169994798649824N), the confidence level is: 0.717601;

[0147] The coordinates of the wind turbine are: (113.94544780769348E, 41.167711205064236N), the confidence level is: 0.701669; the coordinates of the wind turbine are: (113.94500255241394E, 41.13461133606578N), the confidence level is: 0.83494; the coordinates of the wind turbine are: (113.9417383049011E, 41.167711205064236N), the confidence level is: 0.701669; 14021499139993N), the confidence level is 0.793926; the wind turbine coordinates are (113.9486047623825E, 41.17379455620248N), the confidence level is 0.751074; the wind turbine coordinates are (113.95204871978763E, 41.17430332758506N), the confidence level is 0.715185;

[0148] The coordinates of the fan are: (113.92001510009766E, 41.14226927721766N), the confidence level is: 0.801251; the coordinates of the fan are: (113.92689764671326E, 41.14757941074007N), the confidence level is: 0.790204; the coordinates of the fan are: (113.91946792945862E, 41.14837721021825N), the confidence level is: 0.773909;

[0149] The coordinates of the fan are: (113.92727047462466E, 41.14234603516638N), the confidence level is: 0.663316; the coordinates of the fan are: (113.93829435161592E, 41.15506417429399N), the confidence level is: 0.800463; the coordinates of the fan are: (113.90321642074582E, 41.15511466677445N), the confidence level is: 0.807291; the coordinates of the fan are: (113.92444878387454E, 41.1327830763661N), the confidence level is: 0.737501;

[0150] The coordinates of the fan are: (113.89897316780093E, 41.143923563308924N), the confidence level is: 0.771235; the coordinates of the fan are: (113.90032232322693E, 41.14767838292103N), the confidence level is: 0.740479; the coordinates of the fan are: (113.92837017791749E, 41.13676074458807N), the confidence level is: 0.827413;

[0151] The coordinates of the fan are: (113.93437028274536E, 41.13620925771664N), the confidence level is: 0.795647; the coordinates of the fan are: (113.93671184349063E, 41.139362554504864N), the confidence level is: 0.778031; the coordinates of the fan are: (113.90641898345945E, 41.17131929313413N), the confidence level is: 0.75222; the coordinates of the fan are: (113.93166392784116E, 41.16087003507087N), the confidence level is: 0.695306;

[0152] The coordinates of the fan are: (113.91287773933414E, 41.13447800997315N), the confidence level is: 0.784006; the coordinates of the fan are: (113.91527027320859E, 41.14108761542199N), the confidence level is: 0.773354; the coordinates of the fan are: (113.9098361091614E, 41.13951607711643N), the confidence level is: 0.739539; the coordinates of the fan are: (113.91421347427371E, 41.15135421605612N), the confidence level is: 0.76855;

[0153] The coordinates of the fan are: (113.94857525791167E, 41.155534719897176N), and the confidence level is: 0.826686; the coordinates of the fan are: (113.95458877601624E, 41.15383024106572N), and the confidence level is: 0.820716; the coordinates of the fan are: (113.95371437416077E, 41.15001518741647N), and the confidence level is: 0.771986.

[0154] In this embodiment, a wind turbine data set with multiple surfaces and multiple resolutions is obtained by image processing the wind turbine splicing image data, so as to train the constructed wind turbine detection network model through wind turbine data of different surface types and resolutions to obtain the optimal wind turbine detection network model, which is convenient for realizing the diversity detection and identification accuracy of wind turbines of different surface types and resolutions; and the diversity detection of wind turbine features in the remote sensing image of the wind power plant to be detected is greatly improved to obtain the output detection map. This embodiment solves the problem that with the rapid development of the wind power industry, due to the influence of factors such as the geology, climate environment and total wind energy reserves in various regions, wind power installation has shown a trend of regionalization and centralization, resulting in significant differences in wind power development in various regions and the inability to efficiently identify regional wind turbines. This embodiment can detect and obtain wind turbine data with multiple surfaces and multiple resolutions in real time based on a small amount of wind turbine data in the early stage, which greatly reduces the waste of human resource costs and significantly improves the efficiency and accuracy of wind turbine identification while ensuring efficient identification of regional wind turbines.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-surface and multi-resolution wind turbine detection method, characterized in that: The following steps are involved: S1: Obtain wind turbine stitching image data; The wind turbine splicing image data includes wind farm tile coordinates and wind turbine contour features; S2: performing image processing on the wind turbine stitching image data to obtain a wind turbine data set with multiple surfaces and multiple resolutions, and dividing the wind turbine data set into a data training set and a data verification set according to a preset ratio; S3: Build a wind turbine detection network model; The wind turbine detection network model includes a YOLOv5 backbone network for extracting wind turbine features of multiple surfaces and multiple resolutions at different scales, an improved YOLOv5 neck network for performing multi-scale feature fusion on wind turbine features extracted by the backbone network, and a prediction end for wind turbine detection of multiple surfaces and multiple resolutions, and the prediction end is provided with an ASFF-4 module for adaptive feature fusion of output features of the improved YOLOv5 neck network; S4: Perform model training and model verification on the wind turbine detection network model based on the data training set and the data verification set to obtain the optimal wind turbine detection network model; The model training includes: extracting wind turbine features of different scales from wind turbine data of multiple surfaces and multiple resolutions in a data training set based on a YOLOv5 backbone network to obtain a multi-scale wind turbine feature map; Based on the improved YOLOv5 neck network, feature fusion training is performed on multi-scale fan feature maps to obtain fan fusion feature maps of different scales; Based on the ASFF-4 module in the prediction end, adaptive feature fusion is performed on wind turbine fusion feature maps of different scales to obtain adaptive fusion feature maps for wind turbine detection training on multiple surfaces and multiple resolutions. The model verification includes: verifying the wind turbine detection network model after model training through a data verification set to obtain the optimal weight parameters of the wind turbine detection network model and reconstruct the wind turbine detection network model to obtain the optimal wind turbine detection network model; S5: Obtain a remote sensing image of the wind power plant to be detected, and perform segmentation processing on it to obtain a number of remote sensing sub-images, and perform wind turbine feature detection on each remote sensing sub-image based on the optimal wind turbine detection network model to obtain an output detection image.

2. The multi-surface and multi-resolution wind turbine detection method according to claim 1, characterized in that: The S1 specifically includes the following steps: S11: Acquire wind farm coordinate information with multiple surfaces and multiple resolutions; The wind farm coordinate information includes coordinate information of various types of surfaces and resolutions; And perform Mercator coordinate conversion on it to obtain Google map wind farm tile coordinate information, and set the map level of Google map tile coordinate information; S12: calling Google API to obtain a wind farm tile data map according to wind farm tile coordinate information and map level; S13: based on the image stitching strategy, stitching the wind farm tile data map to obtain wind turbine stitching image data; And the image stitching strategy is specifically: Set the image stitching range m×n, where m represents the number of rows in the image stitching range; n represents the number of columns in the image stitching range; The obtained wind farm tile data graphs are m×n in number, and the wind farm tile data graphs are divided into n groups, and each group includes m wind farm tile data graphs; Vertically splice m wind farm tile data images to obtain n groups of vertically spliced ​​images; The wind farm tile data graphs are grouped in a manner that wind farm tile data graphs in each group are arranged in descending order according to image lengths with the same image width; if the image lengths are the same, the wind farm tile data graphs with the same image lengths are randomly sorted; Then, the n groups of longitudinal mosaic images are horizontally mosaicked to obtain the wind turbine mosaic image.

3. The multi-surface and multi-resolution wind turbine detection method according to claim 2, characterized in that: The wind turbine mosaic image data is processed as described in S2 to obtain a wind turbine dataset with multiple surfaces and multiple resolutions, specifically: S21: Manually identify and annotate the wind turbine splicing image based on the labelme annotation tool to obtain an annotated image; The marking rule is to use a rectangular frame to mark the wind turbine body in the wind turbine splicing image, and mark the center point of the wind turbine wing and the wind turbine tower base as key points; S22: performing data cutting, data screening and data expansion on the annotated image in sequence to obtain a wind turbine data map with multiple surfaces and multiple resolutions; Data cutting: Cutting is performed at the center point of the rectangular frame marked with a rectangle to obtain several cut sub-images of preset resolution size; Data filtering: Based on the pre-created data filter, the cut sub-graphs that are confirmed to not exist or have incomplete fan images will be deleted, and the cut sub-graphs where the fan center is close to the image center will be retained when the same fan exists in different cut sub-graphs; The cut sub-maps after data screening are divided into surface types according to the surface environment to obtain multiple types of surface sub-maps; The surface environment at least includes forest land, mountain land, grassland and wasteland; Data expansion: The mean resampling image processing algorithm is used to expand the data of each type of surface sub-image, that is, extracting every one pixel and every two pixels, respectively, to obtain sub-images of different resolutions of the same image; Then, wind turbine data maps with multiple surfaces and multiple resolutions are obtained.

4. The multi-surface and multi-resolution wind turbine detection method according to claim 2, characterized in that: The improved YOLOv5 neck network described in S3 is to add a neck-end network layer for further sampling the output feature map of the original YOLOv5 neck network to the upper layer of the original YOLOv5 neck network; The neck end network layer is used to fuse the output features of the YOLOv5 backbone network; The prediction end includes a plurality of ASFF-4 modules and a Key Detect module connected in sequence for adaptive feature fusion; The ASFF-4 module is used to perform weighted fusion on the output feature maps of the improved YOLOv5 neck network to obtain adaptive fusion feature maps of different scales; The output feature map includes four feature maps of different scales output by four C3 modules connected to the prediction end in the improved YOLOv5 neck network; The Key Detect module is used to detect the fan and locate the key points of the fan according to the output of the ASFF-4 module.

5. The multi-surface and multi-resolution wind turbine detection method according to claim 4, characterized in that: The obtaining of the optimal wind turbine detection network model in S4 specifically includes the following steps: S41: inputting the data training set into the wind turbine detection network model for model training, and obtaining the trained wind turbine detection network model; S42: verifying the trained wind turbine detection network model through the data test set; That is, to determine whether the output of the trained wind turbine detection network model converges; If the output of the trained fan detection network model converges, it is confirmed that the trained fan detection network model is the optimal fan detection network model; Otherwise, based on the back propagation algorithm, the weight parameters of the trained wind turbine detection network model are adaptively adjusted, and step S41 is repeated until it is confirmed that the weight parameters of the trained wind turbine detection network model with converged output are the optimal weight parameters and the wind turbine detection network model is reconstructed to obtain the optimal wind turbine detection network model.

6. The multi-surface and multi-resolution wind turbine detection method according to claim 5, characterized in that: The loss function in S42 that determines whether the output of the trained wind turbine detection network model converges & loss & =λ1loss Obj +λ2loss Conf +λ3loss Class +λ4loss key Where: λ1,λ2,λ3,λ4 represent weight coefficients; loss Obj Indicates the fan positioning loss; loss Conf Indicates the confidence loss of fan detection; loss Class Indicates the classification loss of wind turbines with multiple surfaces and multiple resolutions; loss key Indicates the loss of key point detection of the fan; Represents the coordinates of the key points of the wind turbine predicted by the model; Indicates the real coordinates of the key points of the fan.

Images