Event camera based wind turbine blade tower scanning monitoring method, device and equipment

Abstract

The application relates to an event camera-based wind power blade tower scanning monitoring method, device and equipment. The method comprises the following steps: installing an event camera at a hub of a wind driven generator and pre-calibrating internal parameters of the event camera; monitoring rotating wind driven generator blades by using the calibrated event camera to obtain an event stream; when the blades rotate to a position with the minimum clearance distance, candidate blade tip events are extracted according to the polarity of blade tip events and pixel coordinates of the blade tips; actual blade tip pixel coordinates are extracted from the candidate blade tip events, and the distance between the actual blade tip and an optical axis is calculated according to the actual blade tip pixel coordinates; the distance between the optical axis of the camera and the central axis of the tower is used to calculate the minimum blade clearance distance together with the distance between the actual blade tip and the optical axis; and when the minimum blade clearance distance is smaller than a safety distance, a monitoring warning is sent. The method can be used for real-time monitoring and early warning of wind power blade tower scanning.

Description

Technical Field

[0001] This application relates to the field of wind turbine health monitoring technology, and in particular to a method, device and equipment for wind turbine blade sweep monitoring based on event camera. Background Technology

[0002] Wind energy is an important clean energy source with widespread applications worldwide. my country has become the country with the largest installed wind power capacity and manufacturing scale. As the single-unit capacity of wind turbines continues to increase, blade length and flexibility also increase. Under complex unsteady aerodynamic loads, serious accidents such as blade strikes to the tower, known as blade sweep, may occur. Timely warnings of blade sweep can allow for preventative measures to avoid such serious accidents and reduce losses. Blade sweep warnings are achieved by measuring the minimum blade clearance distance. The blade clearance distance refers to the geometric distance from the blade tip to the tower surface during wind turbine blade rotation, which is minimized when the blade rotates to be perpendicular to the ground. Computer vision methods are commonly used to measure the minimum blade clearance distance. Patent application number CN201811454363 proposes a method of determining the blade clearance distance by installing an optical camera on the ground to photograph the tip of the wind turbine blade and the tower, and then analyzing the positions of the blade tip and tower in the images. However, wind turbines rotate their rotor heads according to wind direction, requiring the camera's ground mounting position to be adjusted accordingly, which is insufficient for the practical needs of long-term wind turbine blade monitoring. Patent application CN202210830550 proposes a method for installing a traditional optical camera at the wind turbine hub to capture real-time images of the blade tips and tower, and then analyzing the video stream information to calculate the blade clearance distance. However, traditional optical cameras are highly sensitive to ambient lighting conditions. If the environment contains high-brightness areas such as direct sunlight or low-brightness areas caused by backlighting, the monitored images suffer from overexposure in high-brightness areas and underexposure in low-brightness areas, affecting the wind turbine blade tower monitoring task. Furthermore, low-frame-rate ordinary cameras struggle to capture images of the blades rotating to their minimum clearance position, inevitably leading to errors in blade clearance distance measurement. Summary of the Invention

[0003] Therefore, it is necessary to provide a method, device, and equipment for monitoring wind turbine blades based on an event camera, which can improve the accuracy of wind turbine blade monitoring, in order to address the above-mentioned technical problems.

[0004] A method for monitoring wind turbine blade sweeping based on event cameras, the method comprising:

[0005] An event camera is installed at the hub of the wind turbine and its internal parameters are pre-calibrated. The calibrated event camera is used to monitor the rotating wind turbine blades to obtain an event flow. When the blade rotates to the position with the minimum clearance distance, the blade tip search area is determined in the event flow. The polarity of the blade tip event is determined according to the rotation direction of the blade in the blade tip search area. Candidate blade tip events are extracted according to the polarity of the blade tip event and the pixel coordinates of the blade tip.

[0006] Extract the actual leaf tip pixel coordinates from the candidate leaf tip events. If the event camera is a monocular event camera, calculate the distance from the leaf tip to the camera's principal point based on the actual leaf tip pixel coordinates. Calculate the actual distance from the leaf tip to the optical axis using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. If the event camera is a binocular event camera, calculate the coordinate points in the left and right camera pixel coordinate systems based on the actual leaf tip pixel coordinates, and calculate the actual distance from the leaf tip to the optical axis based on these coordinate points.

[0007] The distance from the camera's optical axis to the tower's central axis is obtained by adding the distance from the camera's installation location to the tower surface and the tower's radius at the camera's installation location. The minimum blade clearance distance is calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis. When the minimum blade clearance distance is less than the safe distance, a monitoring warning is issued.

[0008] In one embodiment, determining the polarity of the tip-generating event in the region where the leaf tip appears, based on the leaf's rotation direction, includes:

[0009] In the region where the light intensity decreases in the leaf tip area, a negative polarity event occurs on the leaf edge on the side with reduced light intensity, while a positive polarity event occurs on the leaf edge on the other side.

[0010] In one embodiment, candidate leaf tip events are extracted based on the polarity of the leaf tip event and the pixel coordinates of the leaf tip, including:

[0011] Candidate leaf tip events are extracted based on the polarity of the leaf tip event and the pixel coordinates of the leaf tip.

[0012] e b =(x b ,y b ,p,t),t k ≤t≤t k +Δt

[0013] Where, x b y b Here are the pixel coordinates of the candidate leaf tip event; p is the polarity of the event, representing an increase or decrease in light intensity, and t represents the timestamp of the event. k Δt is the moment when the leaf tip rotates into the search area, and Δt is the time within the leaf tip search area.

[0014] In one embodiment, extracting the actual leaf tip pixel coordinates from candidate leaf tip events includes:

[0015] Step 4.1: Using candidate leaf tip events e b Centered on the candidate leaf tip event, search for events with the same timestamp. If the pixel coordinates of the searched event are adjacent to the candidate leaf tip event, denoted as (x... i ,y i ,p,t);

[0016] Step 4.2: Calculate the Euclidean distance d between the searched events and the candidate leaf tip events. i ;

[0017] Step 4.3: Using the searched event as the center, repeat steps 4.1-4.2 to calculate the farthest Euclidean distance d between the searched event and the candidate leaf tip event. i_max ;

[0018] Step 4.4: Repeat steps 4.1-4.3 to filter all candidate leaf tip events. Select the candidate leaf tip time corresponding to the largest and farthest Euclidean distance as the actual leaf tip event, and record the actual leaf tip pixel coordinates (x, y, y) of the actual leaf tip event. b ,y b ) and the corresponding timestamp t.

[0019] In one embodiment, the actual distance from the leaf tip to the optical axis is calculated using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. This includes:

[0020] The actual distance from the leaf tip to the optical axis is calculated using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground.

[0021]

[0022] Where u is the distance from the blade tip to the camera when the blade rotates to be perpendicular to the ground, which is determined by the structural dimensions of the wind turbine, f is the camera focal length, and l is the distance from the blade tip to the camera principal point.

[0023] In one embodiment, the coordinates of the leaf tip in the left and right camera pixel coordinate systems are calculated based on the actual leaf tip pixel coordinates, and the distance from the actual leaf tip to the optical axis is calculated based on the coordinates, including:

[0024] Based on the actual leaf tip pixel coordinates, let the coordinates of the leaf tip in the left camera pixel coordinate system o1-x1y1 be q1, and the coordinates in the right camera pixel coordinate system o2-x2y2 be q2. Let the left camera coordinate system O1-X1Y1Z1 be the reference coordinate system, and the right camera coordinate system O2-X2Y2Z2. Using binocular intersection, obtain the three-dimensional coordinates Q(X1, X2, Y1, Y1, Z1) of the leaf tip in the left camera coordinate system. Q ,Y Q Z Q If the actual distance from the leaf tip to the optical axis of the left camera is L = Y, then... Q .

[0025] In one embodiment, the minimum blade clearance distance is calculated using the distance from the camera optical axis to the tower centerline and the actual distance from the blade tip to the optical axis, including:

[0026] The minimum blade clearance distance was calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis.

[0027] D c =L+wR

[0028] Where R is the cross-sectional radius of the tower at the tip height when the blade rotates to the position of minimum clearance distance, w is the distance from the camera optical axis to the tower central axis, and L is the actual distance from the blade tip to the optical axis.

[0029] A wind turbine blade sweeping monitoring device based on an event camera, the device comprising:

[0030] The candidate blade tip event extraction module is used to install an event camera at the hub of the wind turbine and pre-calibrate the intrinsic parameters of the event camera; the calibrated event camera is used to monitor the rotating wind turbine blades to obtain the event flow; when the blade rotates to the position of minimum clearance distance, the blade tip search area is determined in the event flow; the polarity of the blade tip event is determined according to the rotation direction of the blade in the blade tip search area; and candidate blade tip events are extracted according to the polarity of the blade tip event and the pixel coordinates of the blade tip.

[0031] The leaf tip to optical axis distance calculation module is used to extract the actual leaf tip pixel coordinates from candidate leaf tip events. If the event camera is a monocular event camera, the distance from the leaf tip to the camera's principal point is calculated based on the actual leaf tip pixel coordinates. The actual distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground are used to calculate the actual distance from the leaf tip to the optical axis. If the event camera is a binocular event camera, the coordinates of the actual leaf tip pixel coordinates in the left and right camera pixel coordinate systems are calculated, and the actual distance from the leaf tip to the optical axis is calculated based on these coordinates.

[0032] The monitoring module is used to calculate the distance from the camera's optical axis to the tower's central axis by adding the distance from the camera's installation location to the tower surface and the tower's radius at the camera's installation location; it also calculates the minimum blade clearance distance using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis; and it issues a monitoring warning when the minimum blade clearance distance is less than the safe distance.

[0033] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0034] An event camera is installed at the hub of the wind turbine and its internal parameters are pre-calibrated. The calibrated event camera is used to monitor the rotating wind turbine blades to obtain an event flow. When the blade rotates to the position with the minimum clearance distance, the blade tip search area is determined in the event flow. The polarity of the blade tip event is determined according to the rotation direction of the blade in the blade tip search area. Candidate blade tip events are extracted according to the polarity of the blade tip event and the pixel coordinates of the blade tip.

[0035] Extract the actual leaf tip pixel coordinates from the candidate leaf tip events. If the event camera is a monocular event camera, calculate the distance from the leaf tip to the camera's principal point based on the actual leaf tip pixel coordinates. Calculate the actual distance from the leaf tip to the optical axis using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. If the event camera is a binocular event camera, calculate the coordinate points in the left and right camera pixel coordinate systems based on the actual leaf tip pixel coordinates, and calculate the actual distance from the leaf tip to the optical axis based on these coordinate points.

[0036] The distance from the camera's optical axis to the tower's central axis is obtained by adding the distance from the camera's installation location to the tower surface and the tower's radius at the camera's installation location. The minimum blade clearance distance is calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis. When the minimum blade clearance distance is less than the safe distance, a monitoring warning is issued.

[0037] The aforementioned wind turbine blade sweep monitoring method, device, and equipment based on event cameras first install an event camera at the wind turbine hub to monitor the turbine blades online, obtain event streams, and analyze the polarity of blade tip events in real time to extract candidate blade tip targets. Then, the actual blade tip pixel coordinates are extracted from the candidate blade tip events. If the event camera is a monocular event camera, the distance from the blade tip to the camera's principal point is calculated based on the actual blade tip pixel coordinates. The actual distance from the blade tip to the optical axis is calculated using the distance from the blade tip to the camera's principal point, the camera's focal length, and the distance from the blade tip to the camera when the blade rotates to be perpendicular to the ground. If the event camera is a binocular event camera, the distance between the left and right cameras is calculated based on the actual blade tip pixel coordinates. The distance from the actual blade tip to the optical axis is calculated based on the coordinates of the points in the pixel coordinate system. The distance from the camera's optical axis to the tower's central axis is obtained by adding the distance from the camera's installation position to the tower surface and the tower's radius at the camera's installation position. The minimum blade clearance distance is calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis, thus realizing blade sweeping tower early warning. This method fully utilizes the advantages of event cameras, such as high dynamic range, low latency, low data volume, and low power consumption. It is the first time that event cameras have been used for monitoring and early warning of wind turbine blade sweeping towers, realizing long-term online monitoring of the minimum clearance distance of wind turbine blades under complex lighting conditions, which can effectively avoid the occurrence of blade sweeping tower accidents. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating a wind turbine blade sweep monitoring method based on an event camera in one embodiment.

[0039] Figure 2 This is a schematic diagram of the event camera installation in one embodiment;

[0040] Figure 3 This is a schematic diagram of a binocular event camera tracking the three-dimensional coordinates of a leaf tip in one embodiment;

[0041] Figure 4 This is a structural block diagram of a wind turbine blade sweeping monitoring device based on an event camera in one embodiment;

[0042] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] In one embodiment, such as Figure 1 As shown, a method for monitoring wind turbine blade sweeping based on an event camera is provided, including the following steps:

[0045] Step 102: Install an event camera at the hub of the wind turbine and pre-calibrate the intrinsic parameters of the event camera; use the calibrated event camera to monitor the rotating wind turbine blades to obtain the event flow; when the blade rotates to the position with the minimum clearance distance, determine the blade tip search area in the event flow; determine the polarity of the blade tip event based on the rotation direction of the blade in the blade tip search area; and extract candidate blade tip events based on the polarity of the blade tip event and the pixel coordinates of the blade tip.

[0046] like Figure 2 As shown, an event camera is installed at the hub of a wind turbine and its intrinsic parameters are pre-calibrated. The event camera only reports triggered pixel-level brightness changes, using positive and negative polarities to represent increases and decreases in event brightness, respectively, and outputs an asynchronous event stream with microsecond-level resolution. It has advantages such as high dynamic range, low latency, low data volume, and low power consumption. Utilizing the high dynamic range of the event camera, online monitoring of wind turbine blades under complex lighting conditions in the field can be achieved; the low latency characteristic enables high temporal resolution observation of wind turbine blades, reducing measurement errors caused by the frame rate limitations of traditional optical cameras; the low data volume and low power consumption characteristics effectively reduce computational load and improve monitoring efficiency, which is beneficial for completing online monitoring tasks in field measurement environments. This application uses a calibrated event camera to monitor rotating wind turbine blades to obtain an event stream. When the blade rotates to the position with the minimum clearance distance, a blade tip search area is determined in the event stream. Within the blade tip search area, the polarity of the blade tip event is determined based on the blade's rotation direction. The intersection of the positive and negative polarity event edges of the blade at the same timestamp is extracted as candidate blade tip events.

[0047] Step 104: Extract the actual leaf tip pixel coordinates from the candidate leaf tip events. If the event camera is a monocular event camera, calculate the distance from the leaf tip to the camera's principal point based on the actual leaf tip pixel coordinates. Calculate the actual distance from the leaf tip to the optical axis using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. If the event camera is a binocular event camera, calculate the coordinate points in the left and right camera pixel coordinate systems based on the actual leaf tip pixel coordinates, and calculate the actual distance from the leaf tip to the optical axis based on these coordinate points.

[0048] For a stereo event camera, let the coordinates of the leaf tip in the left camera's pixel coordinate system o1-x1y1 be q1, and the coordinates in the right camera's pixel coordinate system o2-x2y2 be q2. Let the left camera coordinate system O1-X1Y1Z1 be the reference coordinate system, and the right camera coordinate system be O2-X2Y2Z2. Using stereo intersection, the three-dimensional coordinates Q(X1, X2, Y1, Y2) of the leaf tip in the left camera coordinate system are obtained. Q ,Y Q Z Q ),like Figure 3As shown, the distance L from the leaf tip to the optical axis of the left camera is Y. Q .

[0049] Step 106: The distance from the camera's optical axis to the tower's central axis is obtained by adding the distance from the camera's installation position to the tower surface and the tower's radius at the camera's installation position; the minimum blade clearance distance is calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis; when the minimum blade clearance distance is less than the safe distance, a monitoring warning is issued.

[0050] In the aforementioned wind turbine blade scanning monitoring method based on event cameras, an event camera is first installed at the wind turbine hub to monitor the turbine blades online, obtain event streams, and analyze the polarity of blade tip events in real time to extract candidate blade tip targets. Then, the actual blade tip pixel coordinates are extracted from the candidate blade tip events. If the event camera is a monocular event camera, the distance from the blade tip to the camera's principal point is calculated based on the actual blade tip pixel coordinates. The actual distance from the blade tip to the optical axis is calculated using the distance from the blade tip to the camera's principal point, the camera's focal length, and the distance from the blade tip to the camera when the blade rotates to be perpendicular to the ground. If the event camera is a binocular event camera, the distance from the blade tip to the optical axis is calculated based on the actual blade tip pixel coordinates. The coordinates in the reference system are used to calculate the actual distance from the blade tip to the optical axis. The distance from the camera's optical axis to the tower's central axis is obtained by adding the distance from the camera's installation position to the tower surface and the tower radius at the camera's installation position. The minimum blade clearance distance is calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis, thus enabling early warning of blade sweeping. This method fully utilizes the advantages of event cameras, such as high dynamic range, low latency, low data volume, and low power consumption. It is the first time that event cameras have been used to monitor and warn of wind turbine blade sweeping, enabling long-term online monitoring of the minimum clearance distance of wind turbine blades under complex lighting conditions, which can effectively prevent blade sweeping accidents from occurring.

[0051] In one embodiment, determining the polarity of the tip-generating event in the region where the leaf tip appears, based on the leaf's rotation direction, includes:

[0052] In the region where the light intensity decreases in the leaf tip area, a negative polarity event occurs on the leaf edge on the side with reduced light intensity, while a positive polarity event occurs on the leaf edge on the other side.

[0053] In one embodiment, candidate leaf tip events are extracted based on the polarity of the leaf tip event and the pixel coordinates of the leaf tip, including:

[0054] Candidate leaf tip events are extracted based on the polarity of the leaf tip event and the pixel coordinates of the leaf tip.

[0055] e b =(x b ,y b ,p,t),t k ≤t≤tk +Δt

[0056] Where, x b y b Here are the pixel coordinates of the candidate leaf tip event; p is the polarity of the event, representing an increase or decrease in light intensity, and t represents the timestamp of the event. k Δt is the moment when the leaf tip rotates into the search area, and Δt is the time within the leaf tip search area.

[0057] In one embodiment, extracting the actual leaf tip pixel coordinates from candidate leaf tip events includes:

[0058] Step 4.1: Using candidate leaf tip events e b Centered on the candidate leaf tip event, search for events with the same timestamp. If the pixel coordinates of the searched event are adjacent to the candidate leaf tip event, denoted as (x... i ,y i ,p,t);

[0059] Step 4.2: Calculate the Euclidean distance d between the searched events and the candidate leaf tip events. i ;

[0060] Step 4.3: Using the searched event as the center, repeat steps 4.1-4.2 to calculate the farthest Euclidean distance d between the searched event and the candidate leaf tip event. i_max ;

[0061] Step 4.4: Repeat steps 4.1-4.3 to filter all candidate leaf tip events. Select the candidate leaf tip time corresponding to the largest and farthest Euclidean distance as the actual leaf tip event, and record the actual leaf tip pixel coordinates (x, y, y) of the actual leaf tip event. b ,y b ) and the corresponding timestamp t.

[0062] In one embodiment, the actual distance from the leaf tip to the optical axis is calculated using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. This includes:

[0063] The actual distance from the leaf tip to the optical axis is calculated using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground.

[0064]

[0065] Where u is the distance from the blade tip to the camera when the blade rotates to be perpendicular to the ground, which is determined by the structural dimensions of the wind turbine, f is the camera focal length, and l is the distance from the blade tip to the camera principal point.

[0066] In one embodiment, the coordinates of the leaf tip in the left and right camera pixel coordinate systems are calculated based on the actual leaf tip pixel coordinates, and the distance from the actual leaf tip to the optical axis is calculated based on the coordinates, including:

[0067] Based on the actual leaf tip pixel coordinates, let the coordinates of the leaf tip in the left camera pixel coordinate system o1-x1y1 be q1, and the coordinates in the right camera pixel coordinate system o2-x2y2 be q2. Let the left camera coordinate system O1-X1Y1Z1 be the reference coordinate system, and the right camera coordinate system O2-X2Y2Z2. Using binocular intersection, obtain the three-dimensional coordinates Q(X1, X2, Y1, Y1, Z1) of the leaf tip in the left camera coordinate system. Q ,Y Q Z Q If the actual distance from the leaf tip to the optical axis of the left camera is L = Y, then... Q .

[0068] In one embodiment, the minimum blade clearance distance is calculated using the distance from the camera optical axis to the tower centerline and the actual distance from the blade tip to the optical axis, including:

[0069] The minimum blade clearance distance was calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis.

[0070] D c =L+wR

[0071] Where R is the cross-sectional radius of the tower at the tip height when the blade rotates to the position of minimum clearance distance, w is the distance from the camera optical axis to the tower central axis, and L is the actual distance from the blade tip to the optical axis.

[0072] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0073] In one embodiment, such as Figure 4 As shown, a wind turbine blade scanning monitoring device based on an event camera is provided, including: a candidate blade tip event extraction module 402, a distance calculation module 404 from the blade tip to the optical axis, and a monitoring module 406, wherein:

[0074] The candidate blade tip event extraction module is used to install an event camera at the hub of the wind turbine and pre-calibrate the intrinsic parameters of the event camera; the calibrated event camera is used to monitor the rotating wind turbine blades to obtain the event flow; when the blade rotates to the position of minimum clearance distance, the blade tip search area is determined in the event flow; the polarity of the blade tip event is determined according to the rotation direction of the blade in the blade tip search area; and candidate blade tip events are extracted according to the polarity of the blade tip event and the pixel coordinates of the blade tip.

[0075] The leaf tip to optical axis distance calculation module is used to extract the actual leaf tip pixel coordinates from candidate leaf tip events. If the event camera is a monocular event camera, the distance from the leaf tip to the camera's principal point is calculated based on the actual leaf tip pixel coordinates. The actual distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground are used to calculate the actual distance from the leaf tip to the optical axis. If the event camera is a binocular event camera, the coordinates of the actual leaf tip pixel coordinates in the left and right camera pixel coordinate systems are calculated, and the actual distance from the leaf tip to the optical axis is calculated based on these coordinates.

[0076] The monitoring module is used to calculate the distance from the camera's optical axis to the tower's central axis by adding the distance from the camera's installation location to the tower surface and the tower's radius at the camera's installation location; it also calculates the minimum blade clearance distance using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis; and it issues a monitoring warning when the minimum blade clearance distance is less than the safe distance.

[0077] Specific limitations regarding the event-camera-based wind turbine blade sweeping monitoring device can be found in the above-described limitations of the event-camera-based wind turbine blade sweeping monitoring method, and will not be repeated here. Each module in the aforementioned event-camera-based wind turbine blade sweeping monitoring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0078] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores time-stream data acquired by the event camera. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a wind turbine blade sweeping monitoring method based on an event camera.

[0079] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0080] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0081] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for monitoring wind turbine blade sweeping based on event cameras, characterized in that, The method includes: An event camera is installed at the hub of the wind turbine and its internal parameters are pre-calibrated. The calibrated event camera is used to monitor the rotating wind turbine blades to obtain an event flow. When the blade rotates to the position with the minimum clearance distance, the blade tip search area is determined in the event flow. The polarity of the blade tip event is determined according to the rotation direction of the blade in the blade tip search area. Candidate blade tip events are extracted according to the polarity of the blade tip event and the pixel coordinates of the blade tip. The actual leaf tip pixel coordinates are extracted from the candidate leaf tip events. If the event camera is a monocular event camera, the distance from the leaf tip to the camera's principal point is calculated based on the actual leaf tip pixel coordinates. The actual distance from the leaf tip to the optical axis is calculated using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. If the event camera is a binocular event camera, the coordinate points in the left and right camera pixel coordinate systems are calculated based on the actual leaf tip pixel coordinates, and the actual distance from the leaf tip to the optical axis is calculated based on these coordinate points. The distance from the camera's optical axis to the tower's central axis is obtained by adding the distance from the camera's installation position to the tower surface and the tower's radius at the camera's installation position. The minimum blade clearance distance is calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis. When the minimum blade clearance distance is less than the safe distance, a monitoring warning is issued. In the region where the leaf tip appears, the polarity of the event that generates the leaf tip is determined based on the direction of leaf rotation, including: In the region where the light intensity decreases, negative polarity events occur on the leaf edge on the side where the light intensity decreases, while positive polarity events occur on the leaf edge on the other side. Candidate leaf tip events are extracted based on the polarity of the leaf tip event and the pixel coordinates of the leaf tip, including: Candidate leaf tip events are extracted based on the polarity of the leaf tip event and the pixel coordinates of the leaf tip. in, , The pixel coordinates of the candidate leaf tip event; It represents the polarity of the event, indicating an increase or decrease in light intensity. Indicates the timestamp of the event. It is the moment when the leaf tip rotates into the search area. It refers to the time within the leaf tip search area.

2. The method according to claim 1, characterized in that, Extracting the actual leaf tip pixel coordinates from the candidate leaf tip events includes: Step 2.1: Using candidate leaf tip events Centered on the candidate leaf tip event, search for events with the same timestamp. If the pixel coordinates of the searched event are adjacent to the candidate leaf tip event, then record it as such. ; Step 2.2: Calculate the Euclidean distance between the searched events and the candidate leaf tip events. ; Step 2.3: Using the searched event as the center, repeat steps 2.1-2.2 to calculate the farthest Euclidean distance between the searched event and the candidate leaf tip event. ; Step 2.4: Repeat steps 2.1-2.3 to filter all candidate leaf tip events. Select the candidate leaf tip time corresponding to the largest farthest Euclidean distance as the actual leaf tip event, and record the actual leaf tip pixel coordinates of the actual leaf tip event. and the corresponding timestamp .

3. The method according to claim 1, characterized in that, The actual distance from the leaf tip to the optical axis is calculated using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. This includes: The actual distance from the leaf tip to the optical axis is calculated using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. in, It is the distance from the blade tip to the camera when the blade rotates to be perpendicular to the ground, and is determined by the structural dimensions of the wind turbine. It's the camera's focal length. It is the distance from the leaf tip to the main point of the camera.

4. The method according to claim 1, characterized in that, The coordinates of the leaf tip in the left and right camera pixel coordinate systems are calculated based on the actual leaf tip pixel coordinates. The distance from the actual leaf tip to the optical axis is then calculated based on these coordinates, including: Based on the actual leaf tip pixel coordinates, set the leaf tip in the left camera pixel coordinate system. The coordinates of the points are In the right camera pixel coordinate system The coordinates of the points are Let the left camera coordinate system be... Using the reference coordinate system, the right camera coordinate system is... By using binocular intersection, the three-dimensional coordinates of the leaf tip in the left camera coordinate system are obtained. The actual distance from the leaf tip to the optical axis of the left camera is... .

5. The method according to claim 1, characterized in that, The minimum blade clearance distance is calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis, including: The minimum blade clearance distance is calculated using the distance from the camera's optical axis to the tower's central axis and the actual distance from the blade tip to the optical axis. in, The radius of the tower's cross-section at the blade tip height when the blade rotates to the position of minimum clearance. This is the distance from the camera's optical axis to the tower's central axis. This is the actual distance from the leaf tip to the optical axis.

6. A wind turbine blade sweeping monitoring device based on an event camera, characterized in that, The device includes: The candidate blade tip event extraction module is used to install an event camera at the wind turbine hub and pre-calibrate the camera's intrinsic parameters. Using the calibrated event camera, it monitors the rotating wind turbine blades to obtain the event flow. When the blade rotates to the position of minimum clearance, a blade tip search region is determined in the event flow. Within the blade tip search region, the polarity of the generated blade tip event is determined based on the blade's rotation direction, including: In the region where the leaf tip appears, negative polarity events are generated on the leaf edge where the light intensity decreases, and positive polarity events are generated on the leaf edge where the light intensity decreases. Candidate leaf tip events are extracted based on the polarity of the leaf tip events and the pixel coordinates of the leaf tip. in, , The pixel coordinates of the candidate leaf tip event; It represents the polarity of the event, indicating an increase or decrease in light intensity. Indicates the timestamp of the event. It is the moment when the leaf tip rotates into the search area. It refers to the time within the leaf tip search area; The leaf tip to optical axis distance calculation module is used to extract the actual leaf tip pixel coordinates from the candidate leaf tip events. If the event camera is a monocular event camera, the distance from the leaf tip to the camera's principal point is calculated based on the actual leaf tip pixel coordinates. The actual leaf tip to optical axis distance is calculated using the distance from the leaf tip to the camera's principal point, the camera's focal length, and the distance from the leaf tip to the camera when the leaf is rotated to be perpendicular to the ground. If the event camera is a binocular event camera, the coordinate points in the left and right camera pixel coordinate systems are calculated based on the actual leaf tip pixel coordinates, and the actual leaf tip to optical axis distance is calculated based on these coordinate points. The monitoring module is used to obtain the distance from the camera optical axis to the tower centerline by adding the distance from the camera's installation position to the tower surface and the tower radius at the camera's installation position; it calculates the minimum blade clearance distance using the distance from the camera optical axis to the tower centerline and the actual distance from the blade tip to the optical axis; and it issues a monitoring warning when the minimum blade clearance distance is less than the safe distance.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

Images