Monitoring method and device, medium, equipment and product

Abstract

The invention relates to a monitoring method and device, a medium, equipment and a product, and the method comprises the steps: obtaining a target image which comprises wind power generation equipment and the surrounding environment of the wind power generation equipment, inputting the target image into a pre-trained monitoring model, so as to obtain monitoring information outputted by the monitoring model, and determining a monitoring result based on the monitoring information. Therefore, the monitoring result is determined by combining various monitoring information, and the surrounding environment risk of the wind power generation equipment can be finely and automatically judged, so that the monitoring accuracy and the monitoring efficiency of the wind power generation equipment can be effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of wind power generation equipment technology, and more specifically, to a monitoring method, device, medium, equipment, and product. Background Technology

[0002] In existing technologies, monitoring is usually carried out independently for a single monitoring information, and the information obtained is limited in dimensions, which makes it difficult for the monitoring results to support the safety assessment of wind farms. There are problems such as insufficient information support, low analysis accuracy, and poor processing efficiency. Summary of the Invention

[0003] To overcome the problems of insufficient information support, low analytical accuracy, and poor processing efficiency in related technologies, this disclosure provides a monitoring method, device, medium, equipment, and product.

[0004] According to a first aspect of the present disclosure, a monitoring method is provided, the method comprising: Acquire a target image, the target image including the wind power generation equipment and the surrounding environment of the wind power generation equipment; The target image is input into a pre-trained monitoring model to obtain monitoring information output by the monitoring model. The monitoring information includes environmental information of the surrounding environment and the location information of the wind power generation equipment. Based on the monitoring information, the monitoring results are determined, and the monitoring results are used to conduct a risk assessment of the surrounding environment of the wind power generation equipment.

[0005] Optionally, inputting the target image into a pre-trained monitoring model to obtain monitoring information output by the monitoring model includes: The target image is subjected to multi-scale fusion processing to determine the fusion feature map corresponding to the target image; The monitoring information is determined based on the fused feature map.

[0006] Optionally, determining the monitoring information based on the fused feature map includes: The fused feature map is optimized for orientation awareness to obtain an enhanced target fused feature map; Based on the target fusion feature map, the rotation angle of the wind power generation equipment is determined by a preset rotating frame detection head; The orientation information of the wind power generation equipment is determined based on the rotation angle.

[0007] Optionally, the step of performing orientation-aware optimization on the fused feature map to obtain an enhanced target fused feature map includes: Determine the offset of each preset sampling point in the fused feature map; The fused feature map is resampled based on the offset to obtain the target fused feature map.

[0008] Optionally, the environmental information includes multiple roads and multiple buildings.

[0009] Optionally, the monitoring model is trained in the following manner: Acquire multiple preset sample data, including original image sample data of the wind power generation equipment and the area surrounding the wind power generation equipment, road labeling data, building labeling data, and orientation labeling data of the wind power generation equipment; The monitoring model is obtained by training the preset initial model using the multiple preset sample data as training data.

[0010] Optionally, the building labeling data is obtained in the following ways: Obtain building patches from the original image sample data, as well as point of interest data of the surrounding environment of the wind power generation equipment; The building category of the building patch is determined based on the association rules between target keywords and building types in the point of interest data; The building category corresponding to the building patch is used as the building labeling data.

[0011] Optionally, determining the monitoring result based on the monitoring information includes: Based on the location information, the surrounding environment of the wind power generation equipment is divided into multiple different types of buffer zones; The target building in each buffer zone is determined based on the environmental information; The monitoring results are determined based on the category attribute of the target building and the type of the target buffer zone in which the target building is located.

[0012] Optionally, the monitoring results include the maintenance level of the wind power generation equipment, and determining the monitoring results based on the monitoring information further includes: Based on the location information and the environmental information, determine the road distance between the wind power generation equipment and each road in the surrounding environment; Determine the minimum target road distance among the multiple road distances; The maintenance level of the wind power generation equipment is determined based on a preset distance threshold and the distance to the target road.

[0013] According to a second aspect of the present disclosure, a monitoring device is provided, the device comprising: The acquisition module is configured to acquire a target image, the target image including the wind power generation equipment and the surrounding environment of the wind power generation equipment; The monitoring module is configured to input the target image into a pre-trained monitoring model to obtain monitoring information output by the monitoring model. The monitoring information includes environmental information of the surrounding environment and location information of the wind power generation equipment. An assessment module is configured to determine monitoring results based on the monitoring information, the monitoring results being used to conduct a risk assessment of the surrounding environment of the wind power generation equipment.

[0014] Optionally, the monitoring module further includes: The fusion module is configured to perform multi-scale fusion processing on the target image to determine the fusion feature map corresponding to the target image; The determination module is configured to determine the monitoring information based on the fused feature map.

[0015] Optionally, the fusion module is further configured to: The fused feature map is optimized for orientation awareness to obtain an enhanced target fused feature map; Based on the target fusion feature map, the rotation angle of the wind power generation equipment is determined by a preset rotating frame detection head; The orientation information of the wind power generation equipment is determined based on the rotation angle.

[0016] Optionally, the fusion module is further configured to: Determine the offset of each preset sampling point in the fused feature map; The fused feature map is resampled based on the offset to obtain the target fused feature map.

[0017] Optionally, the environmental information includes multiple roads and multiple buildings.

[0018] Optionally, the evaluation module is further configured to: Based on the location information, the surrounding environment of the wind power generation equipment is divided into multiple different types of buffer zones; The target building in each buffer zone is determined based on the environmental information; The monitoring results are determined based on the category attribute of the target building and the type of the target buffer zone in which the target building is located.

[0019] Optionally, the monitoring results include the maintenance level of the wind power generation equipment, and the evaluation module is further configured to: Based on the location information and the environmental information, determine the road distance between the wind power generation equipment and each road in the surrounding environment; Determine the minimum target road distance among the multiple road distances; The maintenance level of the wind power generation equipment is determined based on a preset distance threshold and the distance to the target road.

[0020] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in any of the first aspects.

[0021] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method of any one of the first aspects.

[0022] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0023] The above technical solution acquires a target image, including the wind power generation equipment and its surrounding environment, inputs the target image into a pre-trained monitoring model to obtain monitoring information output by the model, and determines the monitoring result based on this information. In this way, by combining multiple monitoring information sources to determine the monitoring result, it is possible to achieve refined and automated identification of environmental risks surrounding the wind power generation equipment, thereby effectively improving the accuracy and efficiency of wind power generation equipment monitoring.

[0024] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0025] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a monitoring method according to an exemplary embodiment; Figure 2 This is a network structure diagram of a monitoring model according to an exemplary embodiment; Figure 3 It is based on Figure 1 The illustrated embodiment shows a flowchart of a monitoring method; Figure 4 It is based on Figure 3 The illustrated embodiment shows a flowchart of a monitoring method; Figure 5 It is based on Figure 1The illustrated embodiment shows a flowchart of another monitoring method; Figure 6 It is based on Figure 1 The illustrated embodiment shows a flowchart of yet another monitoring method; Figure 7 This is a block diagram illustrating a monitoring device according to an exemplary embodiment; Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0026] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0027] Before detailing the specific implementation methods of this disclosure, the application scenarios of this disclosure are explained below. This disclosure can be applied to the application scenario of monitoring wind power generation equipment.

[0028] Current wind farm environmental monitoring technologies largely focus on the independent extraction of single monitoring information, such as interpreting and analyzing only the location of wind turbine equipment, road networks, or building distribution. This "island-like" monitoring model severs the spatial connections and functional coupling between various elements, failing to support management tasks requiring comprehensive judgment, such as safety assessments, noise impact predictions, and operational accessibility analyses. Furthermore, the limited dimensions of monitoring information for each element lead to problems such as insufficient information support, low analytical accuracy, and poor processing efficiency.

[0029] To address the aforementioned technical problems, this disclosure provides a monitoring method, apparatus, medium, equipment, and product. By acquiring a target image, including a wind power generation device and its surrounding environment, the target image is input into a pre-trained monitoring model to obtain monitoring information output by the model. The monitoring result is then determined based on this information. In this way, by combining multiple monitoring information sources to determine the monitoring result, refined and automated assessment of environmental risks surrounding wind power generation devices can be achieved, thereby effectively improving the accuracy and efficiency of wind power generation device monitoring.

[0030] The specific embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0031] Figure 1 This is a flowchart illustrating a monitoring method according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following steps: Step 101: Obtain the target image.

[0032] The target image includes the wind power generation equipment and the surrounding environment of the wind power generation equipment.

[0033] In this step, a drone with a high-resolution camera can be used to take low-altitude aerial photos of a single wind farm or key area to obtain centimeter-level (5–10 cm) ultra-high resolution images, and these ultra-high resolution images can be used as target images.

[0034] Step 102: Input the target image into the pre-trained monitoring model to obtain the monitoring information output by the monitoring model.

[0035] The monitoring information includes environmental information of the surrounding environment and location information of the wind power generation equipment. The environmental information includes road information of multiple roads and building information of multiple buildings. The location information includes the equipment location and orientation of the wind power generation equipment.

[0036] For example, Figure 2 This is a network structure diagram of a monitoring model illustrated according to an exemplary embodiment, such as... Figure 2 As shown, the monitoring model adopts an improved U-Net architecture (Convolutional Networks for Biomedical Image Segmentation), including an encoder 201, a multi-task decoder 202, a directed wind turbine 203, multiple building types 204, and roads 205. The multi-task decoder 202 consists of a multi-scale feature fusion network 2021 and multiple parallel task-specific output heads, including a directed wind turbine detection head 2022, a semantic segmentation head for multiple building types 2023, and a semantic segmentation head for roads 2024. In specific implementation, the image (target image) is input into the encoder 201 to obtain its high-level semantic feature representation. The encoded image information is then fed into the multi-scale feature fusion network 2021, where cross-level feature aggregation generates an enhanced fusion feature map. This fusion feature map serves as a shared representation and is input to each task head. The directed wind turbine detection head 2022 outputs the wind turbine's position and rotation direction (i.e., the equipment's orientation). The multi-type building semantic segmentation head 2023 is used to achieve pixel-level classification of buildings and identify the type of buildings (such as residential, industrial, public facilities, etc.). The road semantic segmentation head 2024 is used to extract the spatial distribution and topology of the road network.

[0037] It should be noted that, considering the monitoring model needs to handle target elements with large scale differences and diverse feature attributes, such as wind turbines, buildings, and roads, the encoder adopts a ConvNeXt network (A ConvNet for the 2020s) to ensure the quality of feature extraction. This network borrows the "global perception" and "dynamic weights" characteristics of the Transformer (Attention Is All You Need, neural network architecture) while avoiding the high computational cost of self-attention mechanisms, thus controlling the amount of computation while ensuring excellent feature extraction capabilities. The ConvNeXt network is assembled from ConvNeXt modules, which consist of, in sequence: large kernel depthwise convolutions (using 7×7 depthwise separable convolutions to extract spatial features and simulate the local-global relationship of self-attention), pointwise convolutional dimensionality increase (using 1×1 convolutions to expand the number of channels from C to 4C, forming an inverted bottleneck structure), normalization and activation, pointwise convolutional dimensionality reduction (using 1×1 convolutions to reduce the number of channels from 4C back to C, completing the inverted bottleneck), and residual connections with Droppath (Stochastic Depth) (adding the module input to the DropPath-regularized output). When constructing the encoder, the number of ConvNeXt modules is set to 3, 3, 9, and 3 in the four stages, respectively.

[0038] Traditional wind turbine detection often uses frame-by-frame detection, while this solution innovatively employs rotated frame detection. Traditional building semantic segmentation only distinguishes between "buildings / non-buildings," lacking category information and limiting its practical application value. This solution introduces building type classification, dividing buildings into four categories: residential buildings, commercial buildings, industrial buildings, and public facilities, achieving the dual functions of "pixel-level segmentation + type recognition." Channel attention feature refinement: Through average pooling → fully connected dimensionality reduction → ReLU activation → fully connected dimensionality increase → Sigmoid activation, channel attention weights are generated. These weights are multiplied by the input features to enhance the building features of key channels. First, the minimum and maximum values ​​of the feature map in the channel dimension are calculated. After concatenating the two types of extreme features, spatial attention weights are generated through 1×1 convolution. These weights are multiplied by the input features to highlight the spatial details of the building area. Bilinear interpolation restores the feature map resolution to match the input image. Using 1×1 convolution, the output channel number is "number of building categories + background category" (a total of 5 categories), achieving pixel-level category prediction. The semantic segmentation head for multi-type buildings takes high-resolution fusion features as input. After processing by the above modules, it outputs pixel-level classification results with the same resolution as the input image, directly distinguishing between "background, residential buildings, commercial buildings, industrial buildings, and public facilities".

[0039] The road semantic segmentation head and the multi-type building semantic segmentation head employ a similar attention feature refinement architecture. The core difference lies in the classification target and the number of output channels. It only distinguishes between "roads" and "non-roads." The 1×1 convolution outputs 2 channels (corresponding to roads and non-roads). Its workflow is consistent with the building semantic segmentation head: high-resolution fused features are input, refined through channel attention and spatial attention, and then resolution is restored through bilinear interpolation. The final output is a pixel-level road segmentation result with the same resolution as the input image, accurately extracting the spatial distribution and topology of the road network.

[0040] Step 103: Determine the monitoring result based on the monitoring information.

[0041] The monitoring results are used to conduct a risk assessment of the surrounding environment of the wind power generation equipment.

[0042] In one implementation, determining the monitoring result based on the monitoring information may include: dividing the surrounding environment of the wind power generation equipment into multiple different types of buffer zones according to the location information, determining the target building in each buffer zone according to the environmental information, and determining the monitoring result according to the category attribute of the target building and the type of the target buffer zone in which the target building is located.

[0043] In another embodiment, the monitoring results include the maintenance level of the wind power generation equipment. Determining the monitoring results based on the monitoring information may include: determining the road distance between the wind power generation equipment and each road in the surrounding environment according to the location information and the environmental information, determining the minimum target road distance among multiple road distances, and determining the maintenance level of the wind power generation equipment according to a preset distance threshold and the target road distance.

[0044] The above technical solutions, by combining multiple monitoring information to determine the monitoring results, can achieve refined and automated identification of the environmental risks surrounding wind power generation equipment, thereby effectively improving the accuracy and efficiency of monitoring wind power generation equipment.

[0045] Figure 3 It is based on Figure 1 The illustrated embodiment shows a flowchart of a monitoring method, such as Figure 3 As shown, Figure 1 Step 102, which involves inputting the target image into a pre-trained monitoring model to obtain the monitoring information output by the monitoring model, may include: Step 301: Perform multi-scale fusion processing on the target image to determine the fusion feature map corresponding to the target image.

[0046] For example, the target image can be multi-scale fusion processed using a three-layer fusion architecture of "Feature Pyramid Network (FPN) + Path Aggregation Network (PAN) + Adaptive Feature Fusion Network" to determine the corresponding fused feature map of the target image. The specific process is as follows: The Feature Pyramid Network (FPN) transmits semantic information from top to bottom, starting with the deep features output by the encoder and transmitting high-level abstract semantic information along the "deep → shallow" direction. The resolution of the deep features is amplified to the resolution of adjacent shallow features through transposed convolution. The upsampled deep features are then concatenated with adjacent shallow features along the channel dimension. First, the number of channels in the concatenated features is adjusted using a 1×1 convolution, and then the aliasing effect caused by upsampling is eliminated using a 3×3 convolution, finally outputting an intermediate feature map with enhanced semantic information. The Path Aggregation Network (PAN) starts with the shallow features output by the FPN and transmits high-resolution detail information along the "shallow → deep" direction. The resolution of the shallow features output by the FPN is reduced to the resolution of adjacent deep features through stride convolution. The downsampled shallow features are then concatenated with adjacent deep features. The number of channels is adjusted using 1×1 convolutions, and feature consistency is optimized using 3×3 convolutions, ultimately outputting a feature map that retains detailed information. The features fused from FPN and PAN can be directly used for directed wind turbine detection tasks, but segmentation tasks (buildings, roads) require higher resolution features. An adaptive feature fusion network further fuses the aforementioned features to a high-resolution scale, adaptively learning the weights of features at different levels. Through bilinear interpolation upsampling, the resolution of features at each level is unified to a high-resolution scale. For each level of feature map, weight coefficients for spatial and channel dimensions are learned separately. After normalizing the weight coefficients using the Softmax function, the features at each level are weighted and summed to finally obtain a fused feature map at a high-resolution scale.

[0047] Step 302: Determine the monitoring information based on the fused feature map.

[0048] In this step, the fused feature map can be optimized for orientation awareness to obtain an enhanced target fused feature map. Based on the target fused feature map, the rotation angle of the wind power generation equipment is determined by a preset rotating frame detection head, and the orientation information of the wind power generation equipment is determined according to the rotation angle.

[0049] The above technical solution can perform multi-scale fusion processing on the target image, determine the fusion feature map corresponding to the target image, and determine the monitoring information based on the fusion feature map, thereby providing data support for subsequent determination of monitoring results.

[0050] Figure 4 It is based on Figure 3 The illustrated embodiment shows a flowchart of a monitoring method, such as Figure 4 As shown, Figure 3Step 302, determining the monitoring information based on the fused feature map, may include: Step 401: Perform orientation-aware optimization on the fused feature map to obtain an enhanced target fused feature map.

[0051] In this step, the offset of each preset sampling point in the fused feature map can be determined, and the fused feature map is resampled based on the offset to obtain the target fused feature map.

[0052] Step 402: Based on the target fusion feature map, determine the rotation angle of the wind power generation equipment using a preset rotating frame detection head.

[0053] Step 403: Determine the orientation information of the wind power generation equipment based on the rotation angle.

[0054] For example, the wind turbine (including rotation direction) can be accurately located using a parameterized representation of "center point + width and height + angle". Each rotation box is defined by 5 parameters: center coordinates (x, y), the offset of the bounding box center point relative to the grid, width and height (w, h), the width and height of the bounding box (relative proportion), and angle (theta), the rotation angle of the bounding box (in radians, ranging from -π / 2 to π / 2). 3×3 convolution, BatchNorm, SiLU activation, 3×3 convolution, BatchNorm, SiLU activation are performed sequentially to optimize feature representation. To accurately capture the directional features of the wind turbine, an offset prediction branch is introduced in the first 3×3 convolution of feature refinement—predicting the offset of each sampling point through a 1×1 convolution, and then updating the feature map output by the 3×3 convolution using bilinear interpolation based on the offset, improving the discriminative power of direction-related features. A 1×1 convolution is used, with the number of input channels matching the number of feature layer channels, and the number of output channels being 5 (corresponding to the 5 parameters x, y, w, h, and theta). A 1×1 convolution is used, with the number of input channels matching the number of feature layer channels, and the number of output channels equal to the number of classes (only 1 class for wind turbine detection). Alternatively, a 1×1 convolution is used, with the number of input channels matching the number of feature layer channels, and the number of output channels is 1 (outputting the confidence score of the detection box). Multi-scale features are input to the directed wind turbine detection head to predict rotation box detection results at different scales. Non-maximum suppression (NMS) is performed on the multi-scale results to filter redundant detection boxes, ultimately outputting the accurate wind turbine position and rotation direction.

[0055] Optionally, the environmental information includes multiple roads and multiple buildings, and the monitoring model is trained in the following way: Obtain multiple preset sample data.

[0056] The preset sample data includes original image sample data of the wind power generation equipment and its surroundings, road labeling data, building labeling data, and orientation labeling data of the wind power generation equipment.

[0057] In this step, for a single original image sample data, location annotations for wind turbines, building annotations, and road annotations are simultaneously created. The location annotations for wind turbines are stored as XML format tag files, and the building and road annotations are rasterized and stored as PNG format tag files. The dataset, composed of multiple pre-defined sample data, is divided into training, validation, and test sets in a 7:1:2 ratio.

[0058] The monitoring model is obtained by training the preset initial model using the multiple preset sample data as training data.

[0059] The road labeling data and the location labeling data of the wind power generation equipment can be labeled manually by visually inspecting the original image sample data. The building labeling data is obtained by acquiring building patches in the original image sample data and point of interest data of the surrounding environment of the wind power generation equipment. Based on the association rules between target keywords and building types in the point of interest data, the building category of the building patches is determined, and the building category corresponding to the building patches is used as the building labeling data.

[0060] For example, building features can be manually sketched from target images using visual interpretation. Download publicly available map POI (Point of Interest) data for the area surrounding the wind power equipment, and extract the names and spatial coordinates of each POI. Based on building functional characteristics, establish association rules between building types and POI name keywords. Specifically, the association rules are as follows: Residential buildings include keywords such as "building number," "community," "garden," "apartment," "garden," "new village," "residence," "house," "courtyard," and "mansion." Commercial buildings include keywords such as "shopping mall," "building," "plaza," "center," "hotel," "office building," "market," and "market." Industrial buildings include keywords such as "factory," "park," "industrial park," "science park," "development zone," "base," and "group." Public facilities include keywords such as "school," "hospital," "government," "park," "library," "gymnasium," "subway station," and "bus stop." Match the POI data with building features that intersect with their spatial locations, and assign preliminary building category attributes to the relevant buildings according to the above mapping rules. For building features without a category, retrieve their nearest POI based on the principle of spatial proximity, and assign the corresponding building type according to the category to which that POI belongs. The building category results obtained by automatic assignment are manually checked to correct a few unreasonable labels caused by matching errors or semantic ambiguity, and finally accurate and reliable building label data is formed.

[0061] In this step, an adaptive optimizer (AdamW) and a cosine annealing learning rate scheduling strategy can be used. The batch size is set to 16, the encoder learning rate to 1e-4, the multi-task decoder learning rate to 3e-4, and the number of training iterations to 40,000 (including 1,000 warm-up iterations).

[0062] It's important to note that the monitoring model uses a combination of loss functions, including cross-entropy loss, generalized intersection-over-union (GIoU) ​​loss, L1 loss, and angle loss. Cross-entropy loss is used for classification tasks to measure the difference between the model's predicted probability distribution and the actual label distribution. GIoU loss + L1 loss: These two loss functions work together to regress the bounding box. GIoU loss improves upon traditional IoU loss by considering the bounding rectangles of the predicted and target boxes and introducing an error metric relative to these bounding rectangles. This effectively addresses the vanishing gradient problem that can occur when the target and predicted boxes do not overlap, thus improving the accuracy of bounding box prediction. L1 loss, or mean absolute error (MAE), optimizes the coordinates and size of the predicted bounding box to be as close as possible to the position and size of the true bounding box. This combination makes the optimization process more stable. Angle loss is used to regress the angle parameters of the rotated box, i.e., the direction of the wind turbine. The rotated box is typically defined by five parameters: center point coordinates ((x, y)), width (w), height (h), and rotation angle (theta). When calculating the angle loss, considering the periodicity and continuity of angles, a cosine value is used instead of a direct angle value, and the result is Gaussian filtered to smooth the angle estimate. The building segmentation task employs a combination of boundary loss and cross-entropy loss. Boundary loss: inspired by the method of calculating the gradient flow of curve evolution in graph-based discrete optimization methods, it uses an integral method to evaluate the boundary variation, avoiding the problem of directly calculating local differentiation at contour points. The final boundary loss is a linear combination of the softmax probabilities of the network output region, thus it is compatible with other region losses (such as cross-entropy loss). Cross-entropy loss is used to distinguish different types of building pixels. For the specific task of road segmentation, a combination of Dice loss and Focal loss is used. Dice loss focuses on minimizing the difference between the predicted and actual results, and is particularly effective for small-area targets or detailed parts. The Dice Loss value ranges from 0 to 1, with lower values ​​indicating a closer match between the predicted and actual results. Focal loss aims to address class imbalance, especially by giving higher weights to samples that are difficult to classify. By introducing a modulation factor (1-pt)^γ (where γ≥0 is the focusing parameter), Focal loss can significantly reduce the loss impact of easy-to-classify samples while maintaining the loss contribution of difficult samples, thereby better focusing training attention on difficult samples.

[0063] The above technical solutions and monitoring models not only achieve high-precision detection of the location, size, and type of wind turbines, but also accurately determine the building and road information in the surrounding environment of the wind power generation equipment, as well as the orientation information of the wind power generation equipment. This provides data support for subsequent determination of monitoring results by combining multiple monitoring information, thereby improving the accuracy and efficiency of monitoring wind power generation equipment.

[0064] Figure 5 It is based on Figure 1 The illustrated embodiment shows a flowchart of another monitoring method, such as... Figure 5 As shown, Figure 1 Step 103, which involves determining the monitoring result based on the monitoring information, may include: Step 501: Divide the surrounding environment of the wind power generation equipment into multiple different types of buffer zones based on the location information.

[0065] The location information includes the location and orientation of the wind turbine equipment. Equipment orientation (i.e., the direction the wind turbine nacelle points or the normal to the blade rotation plane, usually expressed as 0° north and clockwise, e.g., 270° west) is primarily used to optimize visual impact analysis. The buffer zones are divided into safety buffer zones, noise buffer zones, and visual landscape buffer zones. The safety buffer zone is used to prevent hazards to personnel and facilities from extreme accidents such as turbine collapse, blade breakage, or icing. Its range is set at 1.3 times the total height of the turbine (tower height + blade radius) as the minimum safe distance. The noise buffer zone is used to control the noise generated during turbine operation from interfering with residents' lives. According to relevant acoustic environment standards, the radius of this buffer zone is set at 500 meters. The visual landscape buffer zone is used to assess and mitigate the visual impact of the wind turbine on natural and cultural landscapes. Considering the human eye's viewing distance and landscape harmony, the radius of this buffer zone is set at 3000 meters.

[0066] In this step, different types of buffer zones are divided based on the location of the wind power generation equipment, according to its operating characteristics and potential impact.

[0067] For example, a circular area with a radius of 1.3 × (tower height + blade length) centered on the equipment location serves as a safety buffer zone. This omnidirectional circular buffer zone is designed to cover the maximum potential impact range of a wind turbine collapsing or component detaching in any direction, ensuring the safety of personnel and facilities. For instance, if a wind turbine tower is 80 meters high and its blades are 50 meters long, the radius of the safety buffer zone is 1.3 × (80 + 50) = 169 meters. A circular area with a radius of 500 meters centered on the equipment location serves as a noise buffer zone. Noise generated by wind power equipment attenuates with distance, and 500 meters is a common radius for noise impact assessment. This omnidirectional circular noise buffer zone is used to assess the general impact of noise on the surrounding environment. A circular area with a radius of 3000 meters centered on the equipment location serves as a visual landscape buffer zone. Extending 60° to each side of the equipment's orientation (a total of 120° fan-shaped areas) represents the area with the strongest visual impact. For example, if the equipment is oriented at 135° (southeast), the main visual impact sector is a fan-shaped area between 75° and 195°. This division allows for more precise identification of sensitive landscape targets located in the "frontal" view of the wind turbine.

[0068] Step 502: Determine the target building in each buffer zone based on the environmental information.

[0069] The environmental information includes multiple buildings.

[0070] In this step, the segmentation results for each type of building are binarized to generate 0 / 1 mask images. Morphological operations (such as opening and closing operations) are used to eliminate small patches and holes. Gaussian filtering is applied to smooth the edges, reducing jagged edges and improving contour quality. Using the `findContours` function in OpenCV, closed polygon contours are extracted from the binary mask based on a boundary tracing algorithm, achieving raster-to-vector conversion. The extracted polygons are simplified to reduce redundant nodes, and false detections due to excessively small size are filtered out based on a minimum area threshold, optimizing the geometric quality and topological structure of the vector data. The affine transformation matrix of the original raster data is read, and the contour vertices in the pixel coordinate system are transformed to the real geographic coordinate system and saved as a standard geospatial vector format. Spatial intersection analysis is performed between the building vector layers and three types of buffers: security, noise, and visual landscape. All buildings located within each buffer are extracted as "target buildings."

[0071] Step 503: Determine the monitoring result based on the category attribute of the target building and the type of the target buffer zone in which the target building is located.

[0072] For example, check if there are buildings within the safety buffer zone. If so, especially target buildings such as residences, schools, and public facilities, a direct safety threat is identified, requiring immediate warning or protective measures. Collect data on buildings within the noise buffer zone (within 500 meters). If residential buildings such as residential communities and villages are present, they are considered potentially subject to noise interference, requiring further assessment using acoustic models. If landscape-sensitive targets such as nature reserves, scenic spots, and historical buildings are located within the visual landscape buffer zone (within 3000 meters), they are considered potentially subject to visual impact. If such target buildings fall further into the main visual impact sector (e.g., facing 135°±60°), they are considered to have a significant visual impact, with a higher impact level. The monitoring results will integrate the above information to output a detailed assessment report, clearly listing: the specific names, categories, and distances of affected buildings within the buffer zone; the type and severity of impact faced by each target (safety, noise, visual); and providing management recommendations or risk levels (e.g., "high risk," "requires attention," etc.).

[0073] The above technical solution, by accurately dividing the buffer zone and associating it with attributes, determines the monitoring results based on the category attribute of the target building and the type of the target buffer zone in which the target building is located. This enables refined and automated identification of the environmental risks surrounding wind power generation equipment, thereby effectively improving the accuracy and efficiency of monitoring wind power generation equipment.

[0074] Figure 6 It is based on Figure 1 The illustrated embodiment shows a flowchart of another monitoring method, whereby the monitoring results include the maintenance level of the wind power generation equipment, such as... Figure 6 As shown, Figure 1 Step 103, which involves determining the monitoring result based on the monitoring information, may further include: Step 601: Based on the location information and the environmental information, determine the road distance between the wind power generation equipment and each road in the surrounding environment.

[0075] In this step, the acquired road segmentation results are processed to extract the centerline (skeleton line) of the road surface area, convert it into linear vector data, and decompose the centerline into several continuous road segments based on road intersections. Taking the location of a single wind turbine as the starting point of the analysis, nearest neighbor spatial analysis is performed to search for the road segment closest to it. On this closest road segment, the shortest perpendicular distance to the wind turbine location is calculated, and this shortest perpendicular distance is taken as the road distance. The geographic coordinates of the perpendicular foot point (i.e., the point on the road closest to the wind turbine) are then determined.

[0076] Step 602: Determine the minimum target road distance among the plurality of road distances.

[0077] In this step, the minimum value among all calculated road distances is selected as the target road distance, which is the straight-line distance from the wind turbine to the nearest passable road.

[0078] Step 603: Determine the maintenance level of the wind power generation equipment based on the preset distance threshold and the target road distance.

[0079] For example, if the target road distance is ≤50 meters, it is classified as "Easy Accessible," meaning that maintenance vehicles can quickly reach the site and repairs are convenient. If the target road distance is ≤200 meters and 50 meters < the target road distance, it is classified as "Medium Accessible," which may require the temporary construction of a simple access road. If the target road distance is >200 meters, it is classified as "Difficult," indicating poor traffic conditions, difficulty in accessing large maintenance equipment, and the need to plan a dedicated construction access road or use special transportation methods in advance.

[0080] Optionally, to address common topological defects such as breaks, burrs, and holes in the road binarization results output by the monitoring model, skeleton extraction and path repair algorithms can be used for optimization to generate a road network with good connectivity and a complete topological structure. The road probability map output by the model is converted into a binary mask (0 / 1) and morphological closing operations (with a circular kernel of 10 pixels as the structuring element) are performed to fill small holes inside the road and bridge minor breaks between adjacent areas, improving the integrity of the main road structure. The Zhang-Suen thinning algorithm is used to skeletonize the optimized binary image, extracting the road centerline (skeleton line). A point is defined as an endpoint or breakpoint when there is only one connected pixel in its 8-neighborhood. Based on the breakpoint distribution, if there are two matching breakpoints within a 100-pixel search range, a straight line segment is directly generated between the two points for connection. If there is only one isolated breakpoint, the road segment is extended along its direction. The first 10 skeleton points before the breakpoint are selected, and a straight line is fitted using univariate linear regression to determine the average extension direction of the road. An extension line is generated along this direction, and each point on the extension line is checked to see if it falls into the point set of an existing road skeleton or road surface. If there are overlapping points, the extension line is considered to intersect with the existing road, and the intersection point is the overlapping position. The average width of the original road surface to which the connected line segment belongs is calculated, and the newly connected centerline is expanded into a planar road according to this width. The newly generated road surface is then merged with the original road surface to form an optimized complete road surface layer. Based on the optimized road surface and skeleton lines, the average width of each road skeleton line segment is recalculated, and the geometric and attribute information of the road network is updated to ensure that it can be used for subsequent path analysis and topology modeling.

[0081] It should be noted that, in order to avoid invalid connections, the following filtering conditions are set: only breakpoint pairs with a spacing of less than 50 pixels are attempted to be connected, and the direction of the line connecting the two breakpoints to be connected should be basically consistent with the local direction of their respective roads. Through the above constraints, valid breakpoint pairs that need to be repaired are selected.

[0082] The above technical solutions can accurately identify the nearest road and its contact point to each wind turbine, providing spatial decision support for operation and maintenance scheduling, emergency response and infrastructure construction, thereby effectively improving the accuracy and efficiency of monitoring wind power generation equipment.

[0083] Figure 7 This is a block diagram illustrating a monitoring device according to an exemplary embodiment, such as... Figure 7 As shown, the monitoring device 700 includes: The acquisition module 701 is configured to acquire a target image, the target image including the wind power generation equipment and the surrounding environment of the wind power generation equipment; The monitoring module 702 is configured to input the target image into a pre-trained monitoring model to obtain monitoring information output by the monitoring model. The monitoring information includes environmental information of the surrounding environment and location information of the wind power generation equipment. The assessment module 703 is configured to determine monitoring results based on the monitoring information, the monitoring results being used to conduct a risk assessment of the surrounding environment of the wind power generation equipment.

[0084] The above technical solution acquires a target image, including the wind power generation equipment and its surrounding environment, inputs the target image into a pre-trained monitoring model to obtain monitoring information output by the model, and determines the monitoring result based on this information. In this way, by combining multiple monitoring information sources to determine the monitoring result, it is possible to achieve refined and automated identification of environmental risks surrounding wind power generation equipment, thereby effectively improving the accuracy and efficiency of wind power generation equipment monitoring.

[0085] Optionally, the monitoring module 702 further includes: The fusion module is configured to perform multi-scale fusion processing on the target image to determine the fusion feature map corresponding to the target image; The determination module is configured to determine the monitoring information based on the fused feature map.

[0086] Optionally, the fusion module is further configured to: The fused feature map is optimized for orientation awareness to obtain an enhanced target fused feature map; Based on the target fusion feature map, the rotation angle of the wind power generation equipment is determined by a preset rotating frame detection head; The orientation information of the wind power generation equipment is determined based on the rotation angle.

[0087] Optionally, the fusion module is further configured to: Determine the offset of each preset sampling point in the fused feature map; The fused feature map is resampled based on the offset to obtain the target fused feature map.

[0088] Optionally, the environmental information includes multiple roads and multiple buildings.

[0089] Optionally, the evaluation module 703 is further configured to: Based on the location information, the surrounding environment of the wind power generation equipment is divided into multiple different types of buffer zones; The target building in each buffer zone is determined based on the environmental information; The monitoring results are determined based on the category attribute of the target building and the type of the target buffer zone in which the target building is located.

[0090] Optionally, the monitoring results include the maintenance level of the wind power generation equipment, and the evaluation module 703 is further configured to: Based on the location information and the environmental information, determine the road distance between the wind power generation equipment and each road in the surrounding environment; Determine the minimum target road distance among the multiple road distances; The maintenance level of the wind power generation equipment is determined based on a preset distance threshold and the distance to the target road.

[0091] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0092] Figure 8 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment. For example... Figure 8 As shown, the electronic device 800 may include a processor 801 and a memory 802. The electronic device 800 may also include one or more of a multimedia component 803, an input / output (I / O) interface 804, and a communication component 805.

[0093] The processor 801 controls the overall operation of the electronic device 800 to complete all or part of the steps in the monitoring method described above. The memory 802 stores various types of data to support the operation of the electronic device 800. This data may include, for example, instructions for any application or method operating on the electronic device 800, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 803 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 802 or transmitted via communication component 805. The audio component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 805 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0094] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the monitoring method described above.

[0095] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the monitoring method described above. For example, the computer-readable storage medium may be the memory 802 including the program instructions described above, which may be executed by the processor 801 of the electronic device 800 to complete the monitoring method described above.

[0096] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the monitoring method described above when executed by the programmable device.

[0097] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0098] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0099] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A monitoring method, characterized in that, The method includes: Acquire a target image, the target image including the wind power generation equipment and the surrounding environment of the wind power generation equipment; The target image is input into a pre-trained monitoring model to obtain monitoring information output by the monitoring model. The monitoring information includes environmental information of the surrounding environment and the location information of the wind power generation equipment. Based on the monitoring information, the monitoring results are determined, and the monitoring results are used to conduct a risk assessment of the surrounding environment of the wind power generation equipment.

2. The monitoring method according to claim 1, characterized in that, The step of inputting the target image into a pre-trained monitoring model to obtain monitoring information output by the monitoring model includes: The target image is subjected to multi-scale fusion processing to determine the fusion feature map corresponding to the target image; The monitoring information is determined based on the fused feature map.

3. The monitoring method according to claim 2, characterized in that, The determination of the monitoring information based on the fused feature map includes: The fused feature map is optimized for orientation awareness to obtain an enhanced target fused feature map; Based on the target fusion feature map, the rotation angle of the wind power generation equipment is determined by a preset rotating frame detection head; The orientation information of the wind power generation equipment is determined based on the rotation angle.

4. The monitoring method according to claim 3, characterized in that, The step of performing orientation-aware optimization on the fused feature map to obtain an enhanced target fused feature map includes: Determine the offset of each preset sampling point in the fused feature map; The fused feature map is resampled based on the offset to obtain the target fused feature map.

5. The monitoring method according to claim 1, characterized in that, The environmental information includes multiple roads and multiple buildings.

6. The monitoring method according to claim 5, characterized in that, The monitoring model was trained in the following way: Acquire multiple preset sample data, including original image sample data of the wind power generation equipment and the area surrounding the wind power generation equipment, road labeling data, building labeling data, and orientation labeling data of the wind power generation equipment; The monitoring model is obtained by training the preset initial model using the multiple preset sample data as training data.

7. The monitoring method according to claim 6, characterized in that, The building annotation data was obtained through the following methods: Obtain building patches from the original image sample data, as well as point of interest data of the surrounding environment of the wind power generation equipment; The building category of the building patch is determined based on the association rules between target keywords and building types in the point of interest data; The building category corresponding to the building patch is used as the building labeling data.

8. The monitoring method according to claim 5, characterized in that, Determining the monitoring result based on the monitoring information includes: Based on the location information, the surrounding environment of the wind power generation equipment is divided into multiple different types of buffer zones; The target building in each buffer zone is determined based on the environmental information; The monitoring results are determined based on the category attribute of the target building and the type of the target buffer zone in which the target building is located.

9. The monitoring method according to claim 5, characterized in that, The monitoring results include the maintenance level of the wind power generation equipment, and the determination of the monitoring results based on the monitoring information further includes: Based on the location information and the environmental information, determine the road distance between the wind power generation equipment and each road in the surrounding environment; Determine the minimum target road distance among the multiple road distances; The maintenance level of the wind power generation equipment is determined based on a preset distance threshold and the distance to the target road.

10. A monitoring device, characterized in that, The device includes: The acquisition module is configured to acquire a target image, the target image including the wind power generation equipment and the surrounding environment of the wind power generation equipment; The monitoring module is configured to input the target image into a pre-trained monitoring model to obtain monitoring information output by the monitoring model. The monitoring information includes environmental information of the surrounding environment and location information of the wind power generation equipment. An assessment module is configured to determine monitoring results based on the monitoring information, the monitoring results being used to conduct a risk assessment of the surrounding environment of the wind power generation equipment.

11. The monitoring device according to claim 10, characterized in that, The monitoring module also includes: The fusion module is configured to perform multi-scale fusion processing on the target image to determine the fusion feature map corresponding to the target image; The determination module is configured to determine the monitoring information based on the fused feature map.

12. The monitoring device according to claim 11, characterized in that, The fusion module is also configured to: The fused feature map is optimized for orientation awareness to obtain an enhanced target fused feature map; Based on the target fusion feature map, the rotation angle of the wind power generation equipment is determined by a preset rotating frame detection head; The orientation information of the wind power generation equipment is determined based on the rotation angle.

13. The monitoring device according to claim 11, characterized in that, The fusion module is also configured to: Determine the offset of each preset sampling point in the fused feature map; The fused feature map is resampled based on the offset to obtain the target fused feature map.

14. The monitoring device according to claim 10, characterized in that, The environmental information includes multiple roads and multiple buildings, and the assessment module is further configured to: Based on the location information, the surrounding environment of the wind power generation equipment is divided into multiple different types of buffer zones; The target building in each buffer zone is determined based on the environmental information; The monitoring results are determined based on the category attribute of the target building and the type of the target buffer zone in which the target building is located.

15. The monitoring device according to claim 10, characterized in that, The environmental information includes multiple roads and multiple buildings, the monitoring results include the maintenance level of the wind power generation equipment, and the assessment module is further configured to: Based on the location information and the environmental information, determine the road distance between the wind power generation equipment and each road in the surrounding environment; Determine the minimum target road distance among the multiple road distances; The maintenance level of the wind power generation equipment is determined based on a preset distance threshold and the distance to the target road.

16. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-9.

17. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-9.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-9.

Images