A method and system for tracking blades in wind turbines without stopping

Abstract

The present invention relates to the field of fan control algorithms, and in particular to a blade tracking algorithm and system for non-stop inspection of fan blades, comprising step S1, tracking the fan in the fan image in subsequent frames through a long video object segmentation model XMem, performing semantic segmentation to generate a mask image; step S2, calculating and screening the longest intersection segment of the perpendicular line and the contour of adjacent points in the boundary point set and the corresponding perpendicular line equation, determining two perpendicular lines that meet preset conditions and determining the intersection point, which is the center point of the fan; step S3, performing position offset and change analysis on each blade key point to achieve accurate tracking of the blade key point. The present invention can accurately segment and track fan blades, achieve fine positioning of the blades by accurately extracting the edges, key points and morphological information of the blades, improve the accuracy and stability of tracking, avoid shutdown and manual intervention, reduce the workload and maintenance cost of manual inspections, and reduce the misjudgment and risks caused by human factors.

Description

Technical Field

[0001] The present invention relates to the field of fan control algorithms, and in particular to a fan blade tracking method and system for non-stop inspection. Background Art

[0002] As the importance of renewable energy becomes increasingly prominent, the wind energy industry has entered a stage of rapid development. As the core component of wind energy conversion equipment, wind turbines have gradually increased in height and size in order to obtain more powerful wind energy resources and improve power generation efficiency. Therefore, ensuring that the blades of wind turbines operate within a safe and stable range is an important measure to ensure the safe and effective operation of wind power generation systems. Traditional wind turbine inspections require shutdown or manual inspections, which are not only time-consuming and labor-intensive, but also unable to monitor the operating status of wind turbine blades in real time. It also leads to a decrease in power generation capacity and production losses. Therefore, the present invention proposes the concept of non-stop inspections. During the operation of the wind turbine, the status of the wind turbine blades is tracked in real time through image processing and computer vision technology. The goal is to promptly detect potential faults and abnormal conditions and improve the operating efficiency and reliability of the wind turbine. Summary of the Invention

[0003] The purpose of the present invention is to solve the above technical problems and realize the tracking of fan blades during the inspection of the fan without stopping the fan, including the following steps:

[0004] Step S1: Frame sampling is performed on the pre-acquired video of the fan blade rotation process. The first frame fan image and the corresponding mask image are used as initial information. In the mask image, the fan is separated from the fan background, and the position of the fan in the fan image is determined. The long video object segmentation model XMem is used to track the fan in the fan image in subsequent frames, and semantic segmentation is performed to generate a mask image.

[0005] Step S2: Obtain the point set of the wind turbine's outline in the mask image through the target detection algorithm and perform the convex hull algorithm to solve it to obtain the boundary point set of the outline point set. Calculate and filter out the longest intersection line segment of the perpendicular line between adjacent points in the boundary point set and the outline and the corresponding perpendicular line equation. Determine two perpendicular lines that meet the preset conditions and determine the intersection point. The intersection point is the center point of the wind turbine.

[0006] In step S3, the edge point set of the fan in the mask image is obtained through edge detection, the maximum contour formed by the edge point set is extracted through the convex hull algorithm, and the minimum circumscribed polygon of the maximum contour is calculated. The minimum distance between the center of gravity of the minimum circumscribed polygon and the center point of the fan is determined, and the blade key points of the fan are determined. The position offset and change analysis of each blade key point is then performed to achieve accurate tracking of the blade key points.

[0007] Furthermore, the long video object segmentation model XMem is used to track the wind turbine in the wind turbine image in subsequent frames, and semantic segmentation is performed to generate a mask map, including:

[0008] Initialize the feature storage pool, which includes the long-term memory pool, the working memory pool, and the sensitive memory pool;

[0009] Feature information reading and segmentation: feature information of the wind turbine image in subsequent frames is read through the long-term memory pool, working memory pool, and sensitive memory pool. The feature information is used to generate a mask image corresponding to the wind turbine image;

[0010] Feature storage pool update. During the process of feature information reading and segmentation, the feature storage pool updates the long-term memory pool, working memory pool, and sensitive memory pool in the feature storage pool according to different frequencies.

[0011] Furthermore, the feature storage pool update includes:

[0012] The sensitive memory pool is updated for each frame; the sensitive memory pool is used to store the feature information of each frame, ensuring that the feature information on each frame is stored and serves as the data source for semantic segmentation;

[0013] Updating the working memory pool every preset number of frames; when the working memory pool is updated according to the preset number of frames, merging the feature information of the current frame with the feature information of the stored working memory pool;

[0014] When the working memory pool reaches the set number of frames, the feature information in the working memory pool is integrated into the long-term memory pool. When the storage space of the long-term memory pool is full, the outdated features are deleted to ensure CPU utilization.

[0015] Furthermore, step S2 includes,

[0016] Calculate the intersection length of the perpendicular line between the adjacent points in the boundary point set and the contour, and select the longest intersection segment and the corresponding perpendicular line equation;

[0017] Sort the lengths of the longest intersection segments corresponding to all perpendicular line equations in the current frame;

[0018] Determine whether the angle formed by the first two perpendicular line equations after sorting is greater than 45° and less than 135°;

[0019] When the conditions are met, the intersection of the vertical lines corresponding to the first two vertical line equations is the center point of the fan;

[0020] When the condition is not met, the first perpendicular line equation and the third perpendicular line equation are used in sequence to determine whether the formed angle is greater than 45° and less than 135°. When the angle is greater than 45° and less than 135°, the intersection of the perpendicular lines corresponding to the two perpendicular line equations is the center point of the fan.

[0021] Furthermore, step S3 includes:

[0022] Obtain the edge point set of the fan after segmenting the fan from the background through edge detection;

[0023] Extract the maximum contour and use the convex hull algorithm to find the maximum contour formed by the edge points. The maximum contour corresponds to the boundary of the wind turbine.

[0024] Calculate the minimum circumscribed polygon and calculate the minimum circumscribed polygon of the maximum contour;

[0025] Calculate the key points of the leaf and take the minimum circumscribed polygon Calculate the center of gravity of the triangle and find the minimum distance with the center point of the fan. The three points of the triangle with the minimum distance are recorded as the key points of the blade.

[0026] Furthermore, in step S3, tracking the key points of the leaf further includes:

[0027] Initialize the pixel coordinates of the leaf key points. By default, the pixel coordinates of the three leaf key points in the first frame are used as the initial positions.

[0028] Calculate the distance and sort. For subsequent frames, find the position coordinates of the leaf key point closest to the pixel coordinates of the three points in the previous frame according to the pixel coordinates of the leaf key point. Calculate the distance between the position coordinates of the leaf key point and the pixel coordinates of the three points, and sort them according to the distance size.

[0029] Assign tracking numbers to blade key points. Select the two points closest to each other and assign corresponding tracking numbers to the blade key points. The third blade key point is assigned a tracking number other than the first and second points by default. The tracking number is used to track the blade key points. According to the assigned tracking number, the blade position of the wind turbine in the current frame is tracked.

[0030] Based on the same inventive concept, the present invention also provides a fan non-stop inspection blade tracking system, which executes the above-mentioned fan non-stop inspection blade tracking method, including:

[0031] The wind turbine identification and tracking module samples frames of the pre-acquired wind turbine blade rotation process video and identifies and tracks the wind turbine in the video frame. It then performs semantic segmentation using the long video object segmentation model XMem to generate the corresponding mask image.

[0032] The fan geometric feature calculation module uses the convex hull algorithm and perpendicular line calculation to find two perpendicular lines that meet the conditions, find the intersection of the two perpendicular lines, and determine the center point of the fan;

[0033] Wind turbine blade key point tracking module: extracts the key points of the wind turbine blades based on the center point of the wind turbine to achieve continuous tracking of the blade key points.

[0034] Furthermore, the wind turbine identification and tracking module includes:

[0035] The video frame processing unit samples the frames of the video of the fan blade rotation process and operates based on the first fan image and the corresponding mask image as initial information;

[0036] The feature memory management unit processes and updates the feature memory pool in the long video object segmentation model XMem. The feature memory pool is used to store feature information of the wind turbine image, including the sensitive memory pool, the working memory pool, and the long-term memory pool.

[0037] The wind turbine tracking and segmentation unit, the long video object segmentation model XMem performs semantic segmentation on the wind turbine images in subsequent frames based on feature information to generate corresponding mask images.

[0038] Furthermore, the fan geometric characteristics calculation module includes:

[0039] The contour extraction unit obtains the point set of the wind turbine contour in the mask image and extracts the boundary point set using the convex hull algorithm;

[0040] The center point calculation unit calculates and filters out the longest intersection segment of the perpendicular line and the contour of the adjacent points in the boundary point set and the corresponding perpendicular line equation based on the boundary point set, determines two perpendicular lines that meet the preset conditions and determines the intersection point, which is the center point of the fan.

[0041] Furthermore, the wind turbine blade key point tracking module includes:

[0042] The edge detection unit obtains the edge point set of the fan in the mask image;

[0043] Convex hull construction unit, which extracts the maximum contour formed by the edge point set through the convex hull algorithm and obtains the minimum circumscribed polygon of the maximum contour;

[0044] A centroid calculation unit, which determines the centroid of the minimum circumscribed polygon;

[0045] The key point determination unit calculates the minimum distance between the center of gravity and the center point of the fan, and determines the key point of the fan blade through the minimum distance;

[0046] The key point tracking unit performs position offset and change analysis on each blade key point to achieve accurate tracking of the blade key points.

[0047] Compared with the prior art, the present invention has at least one of the following technical effects:

[0048] (1) The blade tracking method for wind turbine inspection without stopping the machine provided by the present invention can track the blades without stopping the machine. Traditional maintenance methods usually require shutdown or manual inspection. The present invention uses algorithms of computer vision and image processing technology to achieve automatic tracking of blades, avoiding shutdown and manual intervention, improving the efficiency of wind turbine maintenance and the utilization rate of the wind turbine, reducing the workload and maintenance cost of manual inspection, and reducing the misjudgment and risks caused by human factors.

[0049] (2) The present invention can achieve accurate segmentation and tracking of wind turbine blades. By accurately extracting the edges, key points and morphological information of the blades, the present invention can achieve fine positioning of the blades and improve the accuracy and stability of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments:

[0051] Figure 1 This is a flowchart of the steps of the non-stop inspection blade tracking method for a wind turbine according to the present invention;

[0052] Figure 2 This is a semantic segmentation effect diagram of the video of the fan blade rotation process of the present invention ( Figure 2 a, 2b, and 2c are semantic segmentation renderings with different frame numbers);

[0053] Figure 3 The effect diagram of the pixel coordinates of the center point of the fan of the present invention ( Figure 3 a, 3b, and 3c are the center point pixel coordinate renderings for different frame numbers);

[0054] Figure 4 Schematic diagram of the maximum outline and minimum circumscribed polygon of the fan of the present invention ( Figure 4 4a, 4b, and 4c are schematic diagrams of the maximum contour and minimum circumscribed polygon at different frame numbers);

[0055] Figure 5 This is a system block diagram of the fan blade tracking system without shutdown inspection of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0057] Example 1

[0058] The present invention provides a fan blade tracking method for non-stop inspection of the fan, which is used to track the fan blades during the non-stop inspection of the fan. Figure 1 , Figure 1 This is a flowchart of the method for non-stop inspection and blade tracking of a wind turbine according to the present invention, comprising the following steps:

[0059] Step S1: Frame sampling is performed on the pre-acquired video of the fan blade rotation process. The first frame of the fan image and the corresponding mask image are used as initial information. In the mask image, the fan is separated from the fan background, and the position of the fan in the fan image is determined. The fan in the fan image in subsequent frames is tracked using the long video object segmentation model XMem, and semantic segmentation is performed to generate a mask image. Each subsequent frame of the fan image is read using different initialized feature storage pools.

[0060] Preferably, the first frame wind turbine image and the corresponding mask image are used as initial information and as input to the long video object segmentation model XMem, which can help the model track and segment the wind turbine in subsequent frames. The acquisition methods can be manual creation, trained model recognition, and image processing technology. The specific operation methods of the three solutions are as follows;

[0061] (1) Manual creation is the most direct method, which generates the initial mask by manually drawing the boundary of the wind turbine blade in the first frame image;

[0062] (2) Using the trained model, the wind blades can be identified and segmented. Running this model on the first frame image can directly generate the initial mask.

[0063] (3) Customized according to specific scenarios, using image processing techniques (such as threshold processing, edge detection, etc.) to generate the initial mask, which can handle various image noise and abnormal situations.

[0064] Preferably, the feature storage pool is initialized, and the feature storage pool includes a long-term memory pool, a working memory pool, and a sensitive memory pool;

[0065] Feature information reading and segmentation: feature information of the wind turbine image in subsequent frames is read through the long-term memory pool, working memory pool, and sensitive memory pool. The feature information is used to generate a mask image corresponding to the wind turbine image;

[0066] Feature storage pool update: During the feature information reading and segmentation process, the feature storage pool updates the long-term memory pool, working memory pool, and sensitive memory pool in the feature storage pool at different frequencies, and keeps updating the feature information of the video frames in long videos to adapt to the changes and movements of the wind turbine in the video, thereby improving the accurate segmentation and recognition capabilities of wind turbine blades, towers, and hubs; providing high-quality semantic segmentation effects in longer video sequences.

[0067] The sensitive memory pool is updated for each frame. The sensitive memory pool is used to store the feature information of each frame, ensuring that the feature information on each frame is stored and serving as the data source for semantic segmentation to timely capture the wind turbine operation information in the video.

[0068] The working memory pool is updated every preset number of frames (r frames); when the working memory pool is updated according to the preset number of frames (the set maximum number of frames), the feature information of the current frame is merged with the feature information of the stored working memory pool;

[0069] When the working memory pool reaches the set number of frames, the feature information in the working memory pool is integrated into the long-term memory pool in a highly compact form to retain the historical feature information of the wind turbine for a long time and extract more comprehensive feature information. When the storage space of the long-term memory pool is full (which is only possible when processing thousands of frames), outdated features are deleted to ensure CPU utilization.

[0070] "Obsolete features" in this context are features that are no longer useful or effective. This can be due to a variety of reasons.

[0071] Temporal changes: When processing time series data (such as video streams and time series analysis), if certain features are only meaningful within a specific time period, then after this time period, these features can be considered outdated. The specific definition and handling of outdated features need to be determined according to the specific task and requirements. For example:

[0072] (1) Environmental changes: If the environment changes during task execution, features that were effective in the previous environment may become invalid in the new environment. These are outdated features.

[0073] (2) Model update or optimization: In machine learning, through repeated training and optimization, the model may discover new and better features to describe the data, so the original ones may no longer be needed.

[0074] (3) Memory limitations: When processing big data tasks, due to memory limitations, it may be necessary to regularly delete or forget some old features, the so-called outdated features.

[0075] When these feature memories work together, they provide high-quality features even for very long videos, while using less GPU memory to complete blade segmentation. A semi-supervised approach is used to segment wind turbines in long videos; this means that we don't need to provide labeled mask images for every frame in the video; only a subset of frames needs to be labeled for the model to effectively learn. During model training, the feature memory pool is updated at different frequencies to ensure that the model can adapt to changes in the wind turbine blades in the video and accurately segment them.

[0076] See also Figure 2 , Figure 2 This is the semantic segmentation effect diagram of the video of the fan blade rotation process, and Figure 2 a, 2b, and 2c are semantic segmentation renderings at different frame numbers. As shown in the figure, in semantic segmentation, the mask image can represent the segmentation result of the wind turbine or the environmental background in which the wind turbine is located in the image. Each pixel point of the mask image can be marked as belonging to the wind turbine (foreground) or the area that does not belong to the wind turbine (background). By using the mask, the objects in the image can be located, extracted, or analyzed for subsequent processing or analysis.

[0077] Step S2, see Figure 3 , Figure 3 is the effect diagram of the pixel coordinates of the center point of the fan of the present invention, and Figure 3 a, 3b, and 3c are the pixel coordinate renderings of the center point at different frame numbers; the point set of the fan outline in the mask image is obtained by the Yolov5 target detection algorithm and the convex hull algorithm is executed to solve the boundary point set of the outline point set. The longest intersection segment of the perpendicular line between the adjacent points in the boundary point set and the outline and the corresponding perpendicular line equation are calculated and screened out. The two perpendicular lines that meet the preset conditions are determined and the intersection point is determined. The intersection point is the center point of the fan, and then the pixel coordinates of the center point of the fan are determined;

[0078] Preferably, the length of the intersection of the perpendicular line of the adjacent points in the boundary point set and the contour is calculated, and the longest intersection segment and the corresponding perpendicular line equation are selected;

[0079] Sort the lengths of the longest intersection segments corresponding to all perpendicular line equations in the current frame;

[0080] Determine whether the angle formed by the first two perpendicular line equations after sorting is greater than 45° and less than 135°;

[0081] When the conditions are met, the intersection of the vertical lines corresponding to the first two vertical line equations is the center point of the fan;

[0082] When the condition is not met, the first perpendicular line equation and the third perpendicular line equation are used in sequence to determine whether the formed angle is greater than 45° and less than 135°. When the angle is greater than 45° and less than 135°, the intersection of the perpendicular lines corresponding to the two perpendicular line equations is the center point of the fan.

[0083] Step S3, obtain the edge point set of the wind turbine in the mask image through edge detection, extract the maximum contour formed by the edge point set through the convex hull algorithm, and then calculate the minimum circumscribed polygon of the maximum contour, see Figure 4 , Figure 4 Schematic diagram of the maximum outline and minimum circumscribed polygon of the wind turbine. Figure 4a is a schematic diagram of the maximum contour and minimum circumscribed polygon at the 30th frame of the video. Figure 4 b is the schematic diagram of the maximum contour and minimum circumscribed polygon at the 25th frame. Figure 4 c is a schematic diagram of the maximum contour and minimum circumscribed polygon at frame 461. The dashed straight line in the figure depicts the maximum contour, and the dotted line depicts the minimum circumscribed polygon. The minimum distance between the center of gravity of the minimum circumscribed polygon and the center point of the wind turbine is determined, thereby identifying the key points of the wind turbine blades. Position offset and change analysis is then performed on each blade key point to achieve accurate tracking of the blade key points.

[0084] Preferably, step S3 includes:

[0085] Obtain the edge point set of the fan after segmenting the fan from the background through edge detection;

[0086] Extract the maximum contour and use the convex hull algorithm to find the maximum contour formed by the edge points. The maximum contour corresponds to the boundary of the wind turbine. This can better describe the shape of the wind turbine. The maximum contour corresponds to the boundary of the wind turbine.

[0087] Calculate the minimum circumscribed polygon, calculate the minimum circumscribed polygon of the maximum contour, and the minimum circumscribed polygon surrounds the fan blade;

[0088] Calculate the key points of the blade, take advantage of the fact that the center of the triangle is equal to the center of gravity, and select the smallest circumscribed polygon. The center of gravity of the triangle is calculated, and the minimum distance between the triangle and the center point of the fan is found. The three points of the triangle with the minimum distance are recorded as the key points of the blade. These three key points of the blade can represent the position of the blade.

[0089] Furthermore, in step S3, tracking the key points of the leaf further includes:

[0090] Initialize the pixel coordinates of the leaf key points. By default, the pixel coordinates of the three leaf key points in the first frame are used as the initial positions.

[0091] Calculate the distance and sort. For subsequent frames, find the position coordinates of the leaf key point closest to the pixel coordinates of the three points in the previous frame according to the pixel coordinates of the leaf key point. Calculate the distance between the position coordinates of the leaf key point and the pixel coordinates of the three points, and sort them according to the distance size.

[0092] Assign tracking numbers to blade key points. Select the two points closest to each other and assign corresponding tracking numbers to the blade key points. The third blade key point is assigned a tracking number other than the first and second points by default. The tracking number is used to track the blade key points. According to the assigned tracking number, the blade position of the wind turbine in the current frame is tracked.

[0093] Preferably, the tracking number is used to track the blades, further including updating the positions of key points of the blades and updating the blade positions in the current frame according to the tracking numbers assigned to the blades. By tracking blade numbers 1, 2, and 3, the motion trajectory and attitude changes of the wind turbine blades can be estimated through feature matching, trajectory prediction, etc., and the position and attitude of the wind turbine blades can be monitored in real time to identify potential accidents or abnormal situations, providing effective assistance for wind turbine maintenance.

[0094] During the wind turbine blade tracking process, if the position of blade 123 is lost or the tracking fails, the position can be detected through a detection algorithm and corresponding improvement measures can be taken, such as reinitialization or analysis and correction through historical feature information in long-term memory.

[0095] Based on the same inventive concept, the present invention also provides a fan non-stop inspection blade tracking system, which executes the above-mentioned fan non-stop inspection blade tracking method. Figure 5 , Figure 5 This is a system block diagram of the fan blade tracking system without stopping the machine, including:

[0096] The wind turbine identification and tracking module samples frames of the pre-acquired wind turbine blade rotation process video and identifies and tracks the wind turbine in the video frame. It then performs semantic segmentation using the long video object segmentation model XMem to generate the corresponding mask image.

[0097] Preferably, the wind turbine identification and tracking module includes:

[0098] The video frame processing unit is responsible for sampling frames of the video of the wind turbine blade rotation process and performing operations based on the first wind turbine image and the corresponding mask image as initial information;

[0099] The feature memory management unit processes and updates the feature memory pool in the long video object segmentation model XMem. The feature memory pool is used to store feature information of the wind turbine image, including the sensitive memory pool, the working memory pool, and the long-term memory pool.

[0100] The wind turbine tracking and segmentation unit uses the long video object segmentation model XMem to perform semantic segmentation on the wind turbine images in subsequent frames based on feature information to generate corresponding mask images;

[0101] Preferably, the fan geometric characteristics calculation module includes:

[0102] The contour extraction unit obtains the point set of the wind turbine contour in the mask image and extracts the boundary point set using the convex hull algorithm;

[0103] The center point calculation unit calculates and filters out the longest intersection segment of the perpendicular line and the contour of the adjacent points in the boundary point set and the corresponding perpendicular line equation based on the boundary point set, determines two perpendicular lines that meet the preset conditions and determines the intersection point, which is the center point of the fan.

[0104] The fan geometric feature calculation module uses the convex hull algorithm and perpendicular line calculation to find two perpendicular lines that meet the conditions, find the intersection of the two perpendicular lines, and determine the center point of the fan;

[0105] Wind turbine blade key point tracking module: extracts the key points of the wind turbine blades based on the center point of the wind turbine to achieve continuous tracking of the blade key points.

[0106] Preferably, the wind turbine blade key point tracking module includes:

[0107] The edge detection unit obtains the edge point set of the fan in the mask image;

[0108] The convex hull construction unit extracts the maximum contour formed by the edge point set through the convex hull algorithm, which helps us understand the overall shape of the wind turbine and obtain its minimum circumscribed polygon;

[0109] A centroid calculation unit, which determines the centroid of the minimum circumscribed polygon;

[0110] The key point determination unit calculates the minimum distance between the center of gravity and the center point of the fan, and determines the key point of the fan blade through the minimum distance;

[0111] The key point tracking unit performs position offset and change analysis on each blade key point to achieve accurate tracking of the blade key points, thereby performing real-time monitoring and analysis of the wind turbine's movement status.

[0112] Although the present invention has been disclosed above in terms of preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solutions of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the scope of protection of the technical solutions of the present invention.

Claims

1. A method for tracking blades of a wind turbine without stopping the machine, characterized in that: The following steps are involved: Step S1, performing frame sampling on a pre-acquired video of a fan blade rotation process, using a first frame fan image and a corresponding mask image as initial information, segmenting the fan from its background in the mask image, determining the position of the fan in the fan image, tracking the fan in the fan image in subsequent frames using a long video object segmentation model XMem, and performing semantic segmentation to generate the mask image; Step S2, obtaining the point set of the outline of the wind turbine in the mask image through the target detection algorithm and executing the convex hull algorithm to solve and obtain the boundary point set of the point set of the outline, calculating and screening the longest intersection segment of the perpendicular lines of adjacent points in the boundary point set with the outline and the corresponding perpendicular equations, determining two perpendicular lines that meet the preset conditions and determining the intersection point, the intersection point is the center point of the wind turbine; wherein the preset conditions in step S2 are: sorting the lengths of the longest intersection segments corresponding to all the perpendicular line equations in the current frame; judging whether the angle formed by the first two perpendicular line equations after sorting is greater than 45° and less than 135°; Step S3, obtaining the edge point set of the fan in the mask image by edge detection, extracting the maximum contour formed by the edge point set by the convex hull algorithm, and then calculating the minimum circumscribed polygon of the maximum contour, and taking the minimum circumscribed polygon from the minimum circumscribed polygon. The center of gravity of the group triangle is calculated to determine the minimum distance between the center of gravity of the minimum circumscribed polygon and the center point of the fan. The three points of the triangle with the minimum distance are the key points of the fan blades, and then the position offset and change analysis is performed on each of the key points of the blades to achieve accurate tracking of the key points of the blades.

2. The method for tracking blades of a wind turbine without stopping the machine according to claim 1, characterized in that: Tracking the wind turbine in the wind turbine image in subsequent frames by using a long video object segmentation model XMem, performing semantic segmentation to generate the mask image, further comprising: Initializing a feature storage pool, wherein the feature storage pool includes a long-term memory pool, a working memory pool, and a sensitive memory pool; Feature information reading and segmentation: reading feature information of the wind turbine image in the subsequent frame through the long-term memory pool, the working memory pool, and the sensitive memory pool, wherein the feature information is used to generate the mask image corresponding to the wind turbine image; The feature storage pool is updated. During the process of reading and segmenting the feature information, the feature storage pool updates the long-term memory pool, the working memory pool and the sensitive memory pool in the feature storage pool according to different frequencies.

3. The method for non-stop inspection and blade tracking of a wind turbine according to claim 2 is characterized in that: The feature storage pool update further includes: Updating the sensitive memory pool for each frame; The sensitive memory pool is used to store the feature information of each frame, to ensure that the feature information on each frame is stored, and to serve as a data source for the semantic segmentation; Updating the working memory pool every preset number of frames; When the working memory pool is updated according to a preset number of frames, the feature information of the current frame is merged with the feature information of the stored working memory pool; When the working memory pool reaches the set number of frames, the feature information in the working memory pool is integrated into the long-term memory pool. When the storage space of the long-term memory pool is full, outdated features are deleted to ensure CPU usage.

4. The method for non-stop inspection and blade tracking of a wind turbine according to claim 3 is characterized in that: Said step S2 further comprises, When the preset condition is met, the intersection of the vertical lines corresponding to the first two vertical line equations is the center point of the fan; When the preset condition is not met, the first vertical line equation and the third vertical line equation are used in sequence to determine whether the formed angle is greater than 45° and less than 135°. When the angle is greater than 45° and less than 135°, the intersection of the vertical lines corresponding to the two vertical line equations is the center point of the fan.

5. The method for tracking blades of a wind turbine without stopping the machine according to claim 1, characterized in that: The step S3 further includes: Acquire the edge point set of the fan after segmenting the fan from the background through the edge detection; Extracting a maximum contour, and calculating the maximum contour formed by the edge points using the convex hull algorithm, wherein the maximum contour corresponds to a boundary of the wind turbine; Calculating a minimum circumscribed polygon, calculating the minimum circumscribed polygon of the maximum contour; Calculate the key points of the leaf and take The center of gravity of the triangle is calculated, and the minimum distance between the triangle and the center point of the fan is calculated. The three points of the triangle with the minimum distance are recorded as the key points of the blade.

6. The method for tracking blades of a wind turbine without stopping the machine according to claim 5, characterized in that: In step S3, tracking the key points of the leaf further includes: Initializing the pixel coordinates of the leaf key points, and by default taking the pixel coordinates of the three leaf key points in the first frame as the initial positions; Calculate the distance and sort. For the subsequent frames, find the position coordinates of the leaf key point closest to the pixel coordinates of the three points in the previous frame according to the pixel coordinates of the three points, calculate the distance between the position coordinates of the leaf key point and the pixel coordinates of the three points, and sort the distances according to the distances. The blade key point tracking number is assigned, and the two points with the closest distance are selected and the corresponding tracking numbers of the blade key points are assigned. The third blade key point is assigned the tracking number other than the first and second points by default. The tracking number is used to track the blade key point; according to the assigned tracking number, the blade position of the fan in the current frame is tracked.

7. A wind turbine blade tracking system for non-stop inspection, which implements the wind turbine blade tracking method for non-stop inspection according to any one of claims 1 to 6, characterized in that: include, The wind turbine identification and tracking module samples frames of the pre-acquired wind turbine blade rotation process video and identifies and tracks the wind turbine in the video frame. It then performs semantic segmentation using the long video object segmentation model XMem to generate the corresponding mask image. The fan geometric feature calculation module uses the convex hull algorithm and perpendicular line calculation to find two perpendicular lines that meet the conditions, find the intersection of the two perpendicular lines, and determine the center point of the fan; A fan blade key point tracking module is used to extract the fan blade key points according to the center point of the fan to achieve continuous tracking of the blade key points.

8. The wind turbine blade tracking system for non-stop inspection according to claim 7 is characterized in that: The wind turbine identification and tracking module further includes: a video frame processing unit for sampling frames of the video of the fan blade rotation process and performing operations based on the first fan image and the corresponding mask image as initial information; A feature memory management unit processes and updates a feature memory pool in the long video object segmentation model XMem, wherein the feature memory pool is used to store feature information of the wind turbine image, including a sensitive memory pool, a working memory pool, and a long-term memory pool; The wind turbine tracking and segmentation unit, the long video object segmentation model XMem performs semantic segmentation on the wind turbine image in subsequent frames according to the feature information to generate the corresponding mask image.

9. The wind turbine blade tracking system for non-stop inspection according to claim 8 is characterized in that: The fan geometric feature calculation module further includes: A contour extraction unit, which obtains a point set of the contour of the wind turbine in the mask image and extracts the boundary point set using the convex hull algorithm; The center point calculation unit calculates and filters out the longest intersection segment of the perpendicular line of the adjacent points in the boundary point set and the contour and the corresponding perpendicular line equation based on the boundary point set, determines the two perpendicular lines that meet the preset conditions and determines the intersection point, and the intersection point is the center point of the fan.

10. The wind turbine blade tracking system for non-stop inspection according to claim 9, characterized in that: The fan blade key point tracking module further includes: An edge detection unit, which obtains an edge point set of the fan in the mask image; a convex hull construction unit, which extracts the maximum contour formed by the edge point set and obtains the minimum circumscribed polygon of the maximum contour by using the convex hull algorithm; a center of gravity calculation unit, for determining the center of gravity of the minimum circumscribed polygon; a key point determination unit, calculating a minimum distance between the center of gravity and the center point of the wind turbine, and determining the key point of the blade of the wind turbine according to the minimum distance; The key point tracking unit performs position offset and change analysis on each of the blade key points to achieve accurate tracking of the blade key points.

Images