Target detection and tracking method based on fan blades

Abstract

The invention discloses a target detection and tracking method based on fan blades. The method comprises the following steps: shooting a fan by a camera to obtain a fan global surface image; modeling the global surface image of the fan to obtain a Gaussian mixture model of the fan, further obtaining a segmented image, and then sequentially performing morphological operation and contour extraction operation on the segmented image to obtain a blade contour of the fan; carrying out minimum enclosing rectangle fitting processing on the fan blade contours, calculating to obtain fan blade rotation centers, and further determining the positions of the fan blades; and calculating and processing according to the positions of the fan blades in the two adjacent fan global surface images to obtain the instantaneous angular velocity of the fan blades. The rotation center and the real-time angle of the fan are obtained according to the layout characteristics of the blade on the fan, the instantaneous angular velocity is obtained by using the angle difference and the time difference of the adjacent image frames, the accurate measurement of the real-time information of the blade without shutdown is realized, the blade angle and the fan blade rotation speed are efficiently obtained at low cost, and the error is small.

Description

Technical Field

[0001] The present invention relates to a wind turbine blade status monitoring technology, and in particular to a target detection and tracking method based on wind turbine blades. Background Art

[0002] Wind energy is a clean, renewable energy source. Using wind power to generate electricity can reduce dependence on fossil fuels and lower greenhouse gas emissions, becoming an increasingly economically viable new energy development direction. The safe operation of wind turbines is crucial for the viable development of wind power technology. Continuous monitoring and effective maintenance without downtime can promptly detect abnormalities in wind turbine blades, reduce safety hazards and overall costs, and effectively extend the lifespan of wind turbines. Blade speed is one of the most critical operating parameters of wind turbines, affecting not only the turbine's power output but also the operating status of its mechanical components.

[0003] To monitor the blade status of wind turbines without shutting down, a related method uses visual algorithms to inspect the wind turbine based on a global image. The key is to obtain the center coordinates of the wind turbine hub, the angles of each blade, and the wind turbine speed in real time to obtain clearer blade images and improve inspection accuracy. However, in existing technical solutions, visual algorithms are prone to errors in key point positioning, resulting in low algorithm efficiency and reduced subsequent inspection accuracy. Therefore, how to cost-effectively and efficiently obtain blade angles and wind turbine blade speed has become an urgent problem to be solved. Summary of the Invention

[0004] The present invention aims to solve the technical difficulties in visual image acquisition of wind turbine blades. The present invention proposes a target detection and tracking method based on wind turbine blades.

[0005] The technical solution adopted in the present invention is:

[0006] The target detection and tracking method of the wind turbine blade includes the following steps:

[0007] S1. The camera shoots the wind turbine to obtain the global surface image of the wind turbine;

[0008] S2. Modeling the global surface image of the wind turbine to obtain a Gaussian mixture model of the wind turbine, and then obtaining a segmented image. Subsequently, morphological operations and contour extraction operations are performed on the segmented image in sequence to obtain the outline of the wind turbine blades.

[0009] S3, performing minimum circumscribed rectangle fitting on the fan blade contour to calculate the fan blade rotation center, and then determining the position of each fan blade;

[0010] S4. Calculate and process the positions of the fan blades in the two adjacent global surface images of the fan to obtain the instantaneous angular velocity of the fan blades. Finally, detect and track the target of the fan blades based on the positions and instantaneous angular velocity of each fan blade.

[0011] The Gaussian mixture model of the fan in S2 is obtained according to the following formula:

[0012]

[0013] Among them, P(X t ) is the pixel value X in the global surface image of the wind turbine t The probability of K being the number of Gaussian models of the preset wind turbine global surface image, η() represents the Gaussian function, ω i,t 、μ i,t ,Σ i,t Respectively represent the weight, mean, and covariance matrix of the i-th Gaussian function at time t, represents the variance of the i-th Gaussian function at time t, n represents the dimension of the normal distribution, T represents the transpose, exp() represents the exponential function with the natural constant e as the base, Represents the inverse matrix of the covariance matrix of the i-th Gaussian function at time t.

[0014] The S2 is specifically:

[0015] S21, using a Gaussian mixture model to model the global surface image of the wind turbine, and then obtaining a segmented image;

[0016] S22, performing closing and opening operations on the segmented image in sequence to obtain a morphologically processed image;

[0017] S23. Performing a contour extraction operation on the morphologically processed image to obtain a fan blade contour.

[0018] The S21 is specifically as follows:

[0019] First, the Gaussian mixture model is trained using several global surface images of the wind turbine. Then, a global surface image of the wind turbine is input into the wind turbine Gaussian mixture model. All pixels are traversed and the wind turbine Gaussian mixture model is used to judge each pixel in the following way. The image that distinguishes the wind turbine blade area and the wind turbine background is obtained as the segmentation image:

[0020] If |X t -μ i,t-1 |>D*σ i,t-1 , then the pixel is the fan blade area, otherwise it is the fan background area, where X t Represents the grayscale value of the pixel, μ i,t-1 and σ i,t-1They represent the mean and standard deviation of the i-th Gaussian distribution at time t-1, respectively, and D is a control parameter.

[0021] The step S23 specifically includes: performing a contour extraction operation on the morphologically processed image using an eight-connectedness analysis method to extract the contour of the fan blade.

[0022] The S3 is specifically:

[0023] S31. Create a minimum circumscribed rectangle for each fan blade outline, and calculate and process each minimum circumscribed rectangle according to the following method to obtain the fan blade rotation center:

[0024] Connect the midpoints of the two short sides of the minimum circumscribed rectangle of each fan blade to obtain the short side connection line of each fan blade, and then calculate the rotation center of the fan blade based on the intersection of all the short side connection lines of the fan blades;

[0025] S32. The midpoint of the short side of the minimum circumscribed rectangle of each fan blade that is away from the rotation center of the fan blade is used as the tip of the fan blade, and the line connecting the tip of the fan blade and the rotation center of the fan blade is used as the position of each fan blade.

[0026] The instantaneous angular velocity of the fan blade is calculated according to the following formula:

[0027] Δθ0=[(θ1+θ2+θ3)-(θ 10 +θ 20 +θ 30 )] / 3

[0028]

[0029] ω=Δθ / Δt

[0030] Among them, θ 10 ,θ 20 ,θ 30 where θ1, θ2, and θ3 represent the angles between the horizontal line pointing to the right from the center of rotation of the fan blade in the previous global surface image and the three fan blades in the clockwise direction. Δt represents the time interval between two adjacent global surface images of the fan. Δθ0 represents the initial difference in the blade angles of the two global images of the fan. Δθ represents the angle rotated by the fan blade within Δt. ω represents the instantaneous angular velocity of the fan blade.

[0031] The method adopts a galvanometer-based wind turbine blade surface image acquisition system, which includes a wide-angle camera, a high-magnification camera, an image processor and a galvanometer control module. The wide-angle camera is directly opposite the wind turbine blade, the galvanometer control module is arranged directly below the wide-angle camera, and the high-magnification camera is arranged on the side of the galvanometer control module facing the galvanometer control module. The optical axis of the high-magnification camera is perpendicular to the optical axis of the wide-angle camera. The wide-angle camera and the high-magnification camera are respectively used to capture the global surface image and the local surface image of the wind turbine. The wide-angle camera, the high-magnification camera and the galvanometer control module are all electrically connected to the image processor.

[0032] The galvanometer control module includes a motion controller, a galvanometer motor and a galvanometer. The image processor is electrically connected to the motion controller, the motion controller is electrically connected to the galvanometer motor, and the galvanometer motor is connected to the rotating shaft of the galvanometer. The galvanometer is arranged directly below the wide-angle camera, and the high-magnification camera is arranged on the side of the galvanometer facing the galvanometer. The imaging light path of the high-magnification camera is that the fan blade is reflected by the galvanometer and then enters the high-magnification camera. The high-magnification camera has a magnification greater than 10 times.

[0033] The galvanometer includes a second lens and a first lens. The rotation axis of the first lens is parallel to the optical axis of the high-magnification camera, and the rotation axis of the second lens is parallel to the optical axis of the wide-angle camera. The imaging light path of the high-magnification camera is that the wind blade passes through the first lens and the second lens in sequence and then enters the high-magnification camera to obtain a local surface image of the wind blade.

[0034] The beneficial effects of the present invention are:

[0035] The present invention provides an image-based method for measuring the real-time angle and speed of wind turbine blades. The method extracts the wind turbine blade target and blade boundary contour through an online target detection and tracking algorithm, obtains the wind turbine rotation center and real-time angle based on the layout characteristics of the blades in the wind turbine, and obtains the speed using the angle difference and the time difference between adjacent image frames, thereby realizing accurate measurement of the real-time information of the blades without stopping the machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flowchart of the target online detection and tracking algorithm.

[0037] Figure 2 Schematic diagram of the eight-neighborhood boundary tracking algorithm.

[0038] Figure 3 This is a correspondence diagram of vertex order, center coordinates, width, height, and rotation angle, where (a) represents the correspondence diagram of vertex order, center coordinates, width, height, and rotation angle when the direction of the fan blade is to the left, and (b) represents the correspondence diagram of vertex order, center coordinates, width, height, and rotation angle when the direction of the fan blade is to the right.

[0039] Figure 4 Schematic diagram of fan blade angle.

[0040] Figure 5 Schematic diagram for calculating the instantaneous angular velocity of fan blades.

[0041] Figure 6 Figure 1 is the result of the target online detection and tracking algorithm, where (a) represents the original wide-angle image, (b) represents the foreground image determined by the blade target detection algorithm based on the mixed Gaussian background model, (c) represents the blade contour extraction image extracted by the blade target boundary tracking algorithm based on the eight-connected region, (d) represents the minimum circumscribed rotated rectangle image of the blade target contour, (e) represents the approximate line segment of the wind turbine blade and the image marked with the coordinates of the wind turbine rotation center, and (f) represents the real-time angle calculation result of the wind turbine blade. DETAILED DESCRIPTION

[0042] The present invention will be further described below with reference to the accompanying drawings and examples. The embodiments of the present invention include but are not limited to the following examples.

[0043] The present invention proposes a target detection and tracking method based on wind turbine blades. Figure 1 As shown in the figure, for a wind turbine in a real-world scenario, a wide-angle camera captures a global image and converts the coordinates of each point on the wind turbine into the image coordinate system. This creates a one-to-one correspondence between the pixel coordinates in the wide-angle camera and the actual target locations of the wind turbine. The online target detection and tracking algorithm in the image processing module then determines the pixel coordinates of the blade targets in the wide-angle camera.

[0044] The method comprises the following steps:

[0045] S1. The wide-angle camera continuously photographs the wind turbine to obtain several global surface images of the wind turbine;

[0046] S2. Modeling the global surface image of the wind turbine to obtain a Gaussian mixture model of the wind turbine, then inputting the global surface image of the wind turbine into the Gaussian mixture model for processing to obtain a segmented image, and then performing morphological operations and contour extraction operations on the segmented image in sequence to obtain the outline of the wind turbine blade;

[0047] S21, using a Gaussian mixture model to model the global surface image of the wind turbine, and then obtaining a segmented image;

[0048] That is, the background of multiple frames of wind turbine images is trained and modeled, the foreground image is determined based on the current image and background modeling results, and the foreground target of the wind turbine blade is extracted.

[0049] The fan Gaussian mixture model is obtained according to the following formula:

[0050]

[0051] Among them, P(X t ) is the pixel value X in the global surface image of the wind turbine t The probability of K being the number of preset Gaussian models of the wind turbine global surface image, η() represents the i-th Gaussian function at time t, ω i,t 、μ i,t ,Σ i,t Respectively represent the weight, mean, and covariance matrix of the i-th Gaussian function at time t, represents the variance of the i-th Gaussian function at time t, n represents the dimension of the normal distribution, T represents the transpose, exp() represents the exponential function with the natural constant e as the base, Represents the inverse matrix of the covariance matrix of the i-th Gaussian function at time t, and * represents the multiplication symbol.

[0052] Specifically, we first use several global surface images of the wind turbine to train the Gaussian mixture model to obtain a trained wind turbine Gaussian mixture model. Then, we input a global surface image of the wind turbine into the trained wind turbine Gaussian mixture model, traverse all pixels, and use the trained wind turbine Gaussian mixture model to judge each pixel in the following way, obtaining an image that distinguishes the wind turbine blade area from the wind turbine background as a segmented image:

[0053] If |X t -μ i,t-1 |>D*σ i,t-1 , then the pixel is the fan blade area, otherwise it is the fan background area, where X t Represents the grayscale value of the pixel, μ i,t-1 and σ i,t-1 They represent the mean and standard deviation of the i-th Gaussian distribution at time t-1, respectively. D is a control parameter used to control the judgment of X t With μ i,t-1 Whether the degree of deviation is significant, * indicates the multiplication symbol.

[0054] Specifically, for the newly input wide-angle image pixel, the pixel gray value X t Match with the existing K Gaussian distribution models, if |X t -μ i,t-1 | <D*σ i,t-1 Then its pixels obey the existing Gaussian distribution model and are the wind turbine background, otherwise they are the blade foreground; where D is generally taken as 2.5-3.

[0055] S22, performing closing and opening operations on the segmented image in sequence to obtain a morphologically processed image;

[0056] The closing operation selects a 2×2 structuring element, and morphological operations are used to further remove noise and fill the interior of the contour, resulting in better contour extraction results. Specifically, after the wind turbine background is modeled and the blade foreground is extracted, morphological operations are performed on the blade foreground image to erode the background noise and expand the target area. First, a closing operation is used to select a 2×2 structuring element to fill the holes in the connected domain and connect connected domains with similar distances, expanding the blade target boundary and making its contour clearer. Then, an opening operation is used to erode the fine background noise in the image, further eliminating small targets and small areas, and highlighting the blade edge information details.

[0057] S23. Performing a contour extraction operation on the morphologically processed image to obtain a fan blade contour.

[0058] The contour of the wind turbine blade is extracted by using the eight-connectivity analysis method on the morphologically processed image. The eight-connectivity method extracts the image contour by checking the connectivity of a pixel with its eight neighboring pixels, tracing and connecting edge pixels.

[0059] After obtaining the processed blade foreground image, the target tracking algorithm will use the blade target boundary tracking algorithm of the eight-connected region to determine the outline of the wind turbine blade. In the blade target boundary tracking algorithm of the eight-connected region, if there are 8 adjacent points around a pixel, they are numbered 0, 1, 2, 3, 4, 5, 6, and 7 respectively. Since the image of the dynamic blade target detection is a binary image, the fan background pixel value is set to 0 and the blade foreground pixel value is set to 1. The algorithm is shown in the figure below. Figure 2 As shown, the specific steps include: (1) Scan the target area from left to right and from top to bottom, and use the first pixel value of 1 as the starting point O for tracking the blade target contour, and store the coordinates of this point in the blade boundary point sequence. (2) After determining the starting position of the search, start searching from the right direction with point O as the starting point. If the value of the pixel point on the right is 1, it is the required new blade contour point and is written into the blade boundary point sequence. If it is not 1, rotate 45° clockwise and continue searching until the blade foreground point marked as 1 is found. (3) Mark the found point as the new tracking starting point and rotate 45 degrees counterclockwise. (4) Repeat the above steps to search for the next pixel point with a value of 1 until returning to the starting boundary point O, and the boundary tracking process of the target area is completed. (5) By continuously searching and judging, recording the coordinate information of the boundary points, and finally obtaining a coordinate linked list. (6) Traverse the retrieved contour and set the area threshold of the enclosing shape. When the detected contour area exceeds the set area threshold, the contour can be filtered out to find the most accurate blade contour.

[0060] S3, performing minimum circumscribed rectangle fitting on the fan blade contour to calculate the fan blade rotation center, and then determining the position of each fan blade;

[0061] S31. Create a minimum circumscribed rectangle for each fan blade outline, and calculate and process each minimum circumscribed rectangle according to the following method to obtain the fan blade rotation center:

[0062] Connect the midpoints of the two short sides of the minimum circumscribed rectangle of each fan blade to obtain the short side connection line of each fan blade, and then calculate the rotation center of the fan blade based on the intersection of the straight lines where all the short side connection lines of the fan blades are located. That is, the coordinates obtained by taking the average of the coordinates of all the intersection points are used as the coordinates of the rotation center of the fan blade;

[0063] S32. The midpoint of the short side of the minimum circumscribed rectangle of each fan blade that is away from the rotation center of the fan blade is used as the tip of the fan blade, and the line connecting the tip of the fan blade and the rotation center of the fan blade is used as the position of each fan blade.

[0064] After determining the required wind turbine blade contour, the shape of the blade can be fitted by creating a minimum enclosing rectangle. By fitting the contour, the width, height, angle and other information of the minimum enclosing rectangle can be obtained, and the coordinates of the four vertices of the minimum enclosing rectangle can be calculated. The line connecting the midpoints of the short sides of the minimum enclosing rectangle is approximated as the center line of the wind turbine blade. The intersection of two adjacent center lines is determined, and the center point of the three intersections is the rotation center of the wind turbine blade. First, the corresponding relationship between the vertex order, center point coordinates, width, height and rotation angle of the minimum enclosing rectangle is given as follows: Figure 3 As shown, the coordinates of the rectangle vertices 1, 2, 3, and 4 are box[0], box[1], box[2], and box[3], respectively, and the width and height of the rectangle are w and h, respectively.

[0065] Take the line connecting the midpoints of the short sides as the centerline of the fan blade, where the midpoint 1 and midpoint 2 of the short sides are Point1 and Point2 respectively. Figure 3 (a), the coordinates of the midpoint of the short side are:

[0066] Point1=(x1,y1)=(box[2]+box[3]) / 2,Point2=(x2,y2)=(box[0]+box[1]) / 2

[0067] If the blade direction is Figure 3 (b), the coordinates of the midpoint of the short side are:

[0068] Point1=(x1,y1)=(box[0]+box[3]) / 2, Point2=(x2,y2)=(box[1]+box[2]) / 2

[0069] According to the obtained coordinates of the center line endpoints, the parameters of the general formula of the straight line are calculated as follows:

[0070]

[0071] Then the coordinates of the intersection of the two fan blade center lines line1 and line2 (intersection.x, intersection.y) are:

[0072]

[0073] D=line1.A*line2.B-line2.A*line1.B

[0074] Because a wind turbine usually has three blades, the intersection points between each two blades are obtained to obtain three straight line intersections, namely intersection1, intersection2, and intersection3. The center point of the three intersections is taken as the center of rotation of the wind turbine. The coordinates of the wind turbine rotation center are:

[0075] center=(intersection1+intersection2+intersection3) / 3

[0076] By performing the above processing on multiple wide-angle images of the wind turbine, the rotation center coordinates of multiple blade intersections can be obtained; all center coordinates that meet the requirements are averaged to obtain the wind turbine rotation center coordinates with higher accuracy.

[0077] like Figure 4 As shown in the figure, when processing the global image, the angle of the line 1 connecting the midpoints of the short sides of the fan blades is first considered as the blade angle θ, using the formula θ = arctan(Point1-Point2), where Point1 and Point2 are short side midpoints 1 and 2, respectively. When the accuracy of the fan rotation center coordinates is sufficient and local image acquisition of the blade surface begins, the angle of the line 2 connecting the fan rotation center and the midpoint of the short side of the rectangle at the blade tip can be considered as the real-time blade angle θ, using the formula θ = arctan(Point-center), where Point is the midpoint of the short side of the rectangle at the blade tip and center is the fan rotation center, to eliminate the rotated rectangle outline error.

[0078] S4. Calculate and process the positions of the fan blades in the two adjacent global surface images of the fan to obtain the instantaneous angular velocity of the fan blades. Finally, detect and track the target of the fan blades based on the positions and instantaneous angular velocity of each fan blade.

[0079] like Figure 5As shown, the instantaneous angular velocity of the fan blade is calculated according to the following formula:

[0080] Δθ0=[(θ1+θ2+θ3)-(θ 10 +θ 20 +θ 30 )] / 3

[0081]

[0082] ω=Δθ / Δt

[0083] Among them, θ 10 ,θ 20 ,θ 30 where θ1, θ2, and θ3 represent the angles between the horizontal line pointing to the right from the center of rotation of the fan blade in the previous global surface image and the three fan blades in the clockwise direction. Δt represents the time interval between two adjacent global surface images of the fan. Δθ0 represents the initial difference in the blade angles of the two global images of the fan. Δθ represents the angle rotated by the fan blade within Δt. ω represents the instantaneous angular velocity of the fan blade.

[0084] This formula holds true when the angle Δθ of the wind blade rotation in two adjacent global surface images of the wind turbine is less than 120 degrees. Therefore, the interval between the wide-angle camera shots needs to be less than the time required for the wind blade to rotate 120 degrees. In actual implementation, the interval between the wide-angle camera shots is much less than the time required for the wind blade to rotate 120 degrees.

[0085] The method adopts a galvanometer-based wind blade surface image acquisition system, which includes a wide-angle camera, a high-magnification camera, an image processor and a galvanometer control module. The wide-angle camera and its lens are facing the wind blade, the galvanometer control module is arranged directly below the wide-angle camera, the high-magnification camera and its lens are arranged on the side of the galvanometer control module facing the galvanometer control module, and the high-magnification camera is connected to the galvanometer control module through a bracket. The optical axis of the high-magnification camera is perpendicular to the optical axis of the wide-angle camera. The wide-angle camera and the high-magnification camera are used to capture the global surface image and the local surface image of the wind turbine, respectively. The wide-angle camera, the high-magnification camera and the galvanometer control module are all electrically connected to the image processor. The image processor is used to control the camera to capture and process the global surface image of the wind turbine to obtain blade information.

[0086] The galvanometer control module includes a motion controller, a galvanometer motor and a galvanometer. The image processor is electrically connected to the motion controller, the motion controller is electrically connected to the galvanometer motor, and the galvanometer motor is connected to the rotating shaft of the galvanometer. The motion controller receives the control signal output by the image processor and drives the galvanometer motor to rotate the galvanometer. The galvanometer is set directly below the wide-angle camera, and the high-magnification camera and lens are set on the side of the galvanometer facing the galvanometer. The imaging light path of the high-magnification camera is from the fan blades reflected by the galvanometer to enter the high-magnification camera. The high-magnification camera is a camera with a magnification greater than 10 times.

[0087] The galvanometer includes a second lens and a first lens. Both lenses are located directly below the wide-angle camera and on the same side of the high-magnification camera. The rotation axis of the first lens is parallel to the optical axis of the high-magnification camera, and the rotation axis of the second lens is parallel to the optical axis of the wide-angle camera. The rotation axes of the second lens and the first lens are perpendicular to each other. The imaging light path of the high-magnification camera is that the wind blade passes through the first lens and the second lens in sequence and then enters the high-magnification camera to obtain a local surface image of the wind blade.

[0088] The system is set up on the ground 80-150m away from the wind turbine.

[0089] The wide-angle camera is a visible light camera. The wide-angle camera and its lens are positioned directly against the wind turbine blades, capturing a global image of the wind turbine and transmitting it to the image processing module, displaying the turbine's overall operating posture in real time. The hardware for the image processing module is a host computer, to which the image acquisition module is connected via a USB serial port. The image processing module processes the global wind turbine image information captured by the multi-frame wide-angle camera, analyzing it using an online target detection and tracking algorithm to obtain blade information, including blade position, real-time angle, and rotational speed. It then determines motion control instructions for the galvanometer deflection angle and image acquisition instructions for the camera's capture rhythm, sending these instructions to the galvanometer and camera devices, respectively.

[0090] The wide-angle camera placement must meet global capture requirements, ensuring that the operating posture of all three wind turbine blades is observed as much as possible, while allowing for incomplete images of blades above the turbine center. In a real-world test, the wind turbine hub center height was 100 meters, the blade diameter was 50 meters, and the width ranged from 8 to 12 meters. The system was positioned directly opposite the wind turbine, 80 meters from the turbine, and placed on flat ground. The system's elevation angle was 30°, and the wide-angle lens' field of view was 59.8° × 46.6°. The actual range of altitudes that the wide-angle camera can capture is from 0 to 162.2 meters.

[0091] like Figure 6 The result image of the blades is extracted from the global image of the wind turbine through the target online detection and tracking algorithm. Figure 6 (a) is the original wide-angle image taken by the system. Figure 6(b) is the foreground image determined by the leaf target detection algorithm based on the mixed Gaussian background model. Figure 6 (c) is the leaf contour extraction based on the leaf target boundary tracking algorithm of the eight connected regions. Figure 6 (d) is the minimum circumscribed rotation rectangle of the blade target contour, Figure 6 (e) is the approximate line segment of the fan blade and the coordinate mark of the fan rotation center, Figure 6 (f) is the real-time angle calculation result of the fan blade.

[0092] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A target detection and tracking method based on wind turbine blades, characterized in that: The method comprises the following steps: S1. The camera shoots the wind turbine to obtain the global surface image of the wind turbine; S2. Modeling the global surface image of the wind turbine to obtain a Gaussian mixture model of the wind turbine, and then obtaining a segmented image. Subsequently, morphological operations and contour extraction operations are performed on the segmented image in sequence to obtain the outline of the wind turbine blades. S3, performing minimum circumscribed rectangle fitting on the fan blade contour to calculate the fan blade rotation center, and then determining the position of each fan blade; S4. Calculate and process the positions of the fan blades in the two adjacent global surface images of the fan to obtain the instantaneous angular velocity of the fan blades. Finally, detect and track the target of the fan blades based on the positions and instantaneous angular velocity of each fan blade.

2. The target detection and tracking method based on wind turbine blades according to claim 1, characterized in that: The Gaussian mixture model of the fan in S2 is obtained according to the following formula: Among them, P(X t ) is the pixel value X in the global surface image of the wind turbine t The probability of K being the number of Gaussian models of the preset wind turbine global surface image, η() represents the Gaussian function, ω i,t 、μ i,t ,Σ i,t Respectively represent the weight, mean, and covariance matrix of the i-th Gaussian function at time t, represents the variance of the i-th Gaussian function at time t, n represents the dimension of the normal distribution, T represents the transpose, exp() represents the exponential function with the natural constant e as the base, Represents the inverse matrix of the covariance matrix of the i-th Gaussian function at time t.

3. The target detection and tracking method based on wind turbine blades according to claim 1, characterized in that: The S2 is specifically: S21, using a Gaussian mixture model to model the global surface image of the wind turbine, and then obtaining a segmented image; S22, performing closing and opening operations on the segmented image in sequence to obtain a morphologically processed image; S23. Performing a contour extraction operation on the morphologically processed image to obtain a fan blade contour.

4. The target detection and tracking method based on wind turbine blades according to claim 3 is characterized in that: The S21 is specifically: First, the Gaussian mixture model is trained using several global surface images of the wind turbine. Then, a global surface image of the wind turbine is input into the wind turbine Gaussian mixture model. All pixels are traversed and the wind turbine Gaussian mixture model is used to judge each pixel in the following way. The image that distinguishes the wind turbine blade area and the wind turbine background is obtained as the segmentation image: If |X t -μ i,t-1 |>D*σ i,t-1 , then the pixel is the fan blade area, otherwise it is the fan background area, where X t Represents the grayscale value of the pixel, μ i,t-1 and σ i,t-1 They represent the mean and standard deviation of the i-th Gaussian distribution at time t-1, respectively, and D is a control parameter.

5. The target detection and tracking method based on wind turbine blades according to claim 3 is characterized in that: The step S23 specifically includes: performing a contour extraction operation on the morphologically processed image using an eight-connectedness analysis method to extract the contour of the fan blade.

6. The target detection and tracking method based on wind turbine blades according to claim 1, characterized in that: The S3 is specifically: S31. Create a minimum circumscribed rectangle for each fan blade outline, and calculate and process each minimum circumscribed rectangle according to the following method to obtain the fan blade rotation center: Connect the midpoints of the two short sides of the minimum circumscribed rectangle of each fan blade to obtain the short side connection line of each fan blade, and then calculate the rotation center of the fan blade based on the intersection of all the short side connection lines of the fan blades; S32. The midpoint of the short side of the minimum circumscribed rectangle of each fan blade that is away from the rotation center of the fan blade is used as the tip of the fan blade, and the line connecting the tip of the fan blade and the rotation center of the fan blade is used as the position of each fan blade.

7. The target detection and tracking method based on wind turbine blades according to claim 1, characterized in that: The instantaneous angular velocity of the fan blade is calculated according to the following formula: Δθ0=[(θ1+θ2+θ3)-(θ 10 +θ 20 +θ 30 )] / 3 ω=Δθ / Δt Among them, θ 10 ,θ 20 ,θ 30 where θ1, θ2, and θ3 represent the angles between the horizontal line pointing to the right from the center of rotation of the fan blade in the previous global surface image and the three fan blades in the clockwise direction. Δt represents the time interval between two adjacent global surface images of the fan. Δθ0 represents the initial difference in the blade angles of the two global images of the fan. Δθ represents the angle rotated by the fan blade within Δt. ω represents the instantaneous angular velocity of the fan blade.

8. The target detection and tracking method based on wind turbine blades according to claim 1, characterized in that: The method adopts a galvanometer-based wind turbine blade surface image acquisition system, which includes a wide-angle camera, a high-magnification camera, an image processor and a galvanometer control module. The wide-angle camera is directly opposite the wind turbine blade, the galvanometer control module is arranged directly below the wide-angle camera, and the high-magnification camera is arranged on the side of the galvanometer control module facing the galvanometer control module. The optical axis of the high-magnification camera is perpendicular to the optical axis of the wide-angle camera. The wide-angle camera and the high-magnification camera are respectively used to capture the global surface image and the local surface image of the wind turbine. The wide-angle camera, the high-magnification camera and the galvanometer control module are all electrically connected to the image processor.

9. The target detection and tracking method based on wind turbine blades according to claim 8, characterized in that: The galvanometer control module includes a motion controller, a galvanometer motor and a galvanometer. The image processor is electrically connected to the motion controller, the motion controller is electrically connected to the galvanometer motor, and the galvanometer motor is connected to the rotating shaft of the galvanometer. The galvanometer is arranged directly below the wide-angle camera, and the high-magnification camera is arranged on the side of the galvanometer facing the galvanometer. The imaging light path of the high-magnification camera is that the fan blade is reflected by the galvanometer and then enters the high-magnification camera. The high-magnification camera has a magnification greater than 10 times.

10. The target detection and tracking method based on wind turbine blades according to claim 9, characterized in that: The galvanometer includes a second lens and a first lens. The rotation axis of the first lens is parallel to the optical axis of the high-magnification camera, and the rotation axis of the second lens is parallel to the optical axis of the wide-angle camera. The imaging light path of the high-magnification camera is that the wind blade passes through the first lens and the second lens in sequence and then enters the high-magnification camera to obtain a local surface image of the wind blade.

Images