Dynamic focusing method and system for fan unmanned aerial vehicle inspection

Abstract

The invention discloses a fan unmanned aerial vehicle routing inspection dynamic focusing method, which comprises the following steps: constructing a motion prediction model of fan parts based on fan operation parameters and unmanned aerial vehicle flight path planning, and obtaining target core parameters at the next moment, the target core parameters comprise a fan component predicted position coordinate, a linear distance between the predicted unmanned aerial vehicle and the fan component and a distance change acceleration; based on the target core parameter at the next moment, calculating and obtaining a target focal length required by the pan-tilt camera at the next moment; constructing a mapping relation, and obtaining a motion advance for adjusting a camera focusing motor; and based on the motion advance, the camera focusing motor performs dynamic driving adjustment and performs real-time feedback correction based on the encoder to complete dynamic focusing adaptation. Based on target motion trail prediction and advanced focusing control, dynamic targets such as fan rotating blades can be accurately tracked, the focusing response speed is high, the'out-of-focus' rate is reduced, and the image definition in a dynamic scene is ensured.

Description

Technical Field

[0001] This patent application relates to the field of wind turbine inspection technology, and in particular to a method and system for dynamic focusing during wind turbine unmanned aerial vehicle (UAV) inspection. Background Technology

[0002] Currently, in the field of wind turbine inspection, drones have widely replaced traditional manual inspection methods due to their advantages such as high flexibility, convenient operation, and ability to cover complex working environments. Among existing wind turbine drone inspection technologies, the mainstream technology is autofocus based on contrast detection or phase detection. Contrast detection analyzes the changes in brightness and darkness of pixels in an image to find the focal position with the highest contrast; phase detection calculates the focal shift by measuring the phase difference of light received by a sensor, achieving rapid focusing. Existing autofocus technology performs well in static shooting or scenarios with slow target movement and has been applied to some wind turbine drone inspection equipment. For example, some commercial inspection drones are equipped with cameras that can automatically focus and photograph the surface of the wind turbine nacelle. However, existing technologies have the following problems: 1. Weak adaptability to dynamic scenes. During wind turbine operation, the blades rotate continuously at a certain speed (usually 10-20 rpm), and the drone needs to fly around the wind turbine to capture images of components from different angles, resulting in relative motion between the target (such as rotating blades) and the drone. Existing autofocus technologies are mostly designed for static or low-speed moving targets. When the target is moving dynamically, the focusing system struggles to quickly track changes in the target's position, easily resulting in "out-of-focus" phenomena and a significant decrease in image clarity. Secondly, there is insufficient distance adaptability. During the inspection process, the distance between the UAV and wind turbine components changes with the flight trajectory (e.g., from far from the nacelle to near the blade tip). The focusing distance adjustment range and response speed of existing autofocus systems are difficult to match this dynamic distance change, especially when shooting blade details at close range, easily resulting in focusing lag and failure to capture target features clearly in a timely manner. Existing focusing technologies do not fully integrate the "dynamic target + distance change" scenario characteristics of wind turbine inspection, lacking precise focusing control logic that considers the motion patterns of wind turbine components, the correlation between the UAV's flight trajectory and the target distance, resulting in focusing accuracy and response speed that cannot meet the high-precision and high-efficiency requirements of wind turbine inspection. Therefore, we propose a method and system for dynamic focusing during wind turbine UAV inspection. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, the first objective of this invention is to propose a dynamic focusing method for wind turbine drone inspection. Based on target motion trajectory prediction and pre-focusing control, it can accurately track dynamic targets such as rotating wind turbine blades, with fast focusing response speed, ensuring image clarity in dynamic scenes. It eliminates the need for operators to manually adjust the focus. Through automatic target recognition and dynamic focusing control, it achieves automatic clear imaging of wind turbine components, reducing manual intervention. The inspection time for a single wind turbine can be shortened by 30%-50%, significantly improving the inspection efficiency of large-scale wind farms.

[0004] The second objective of this invention is to propose a dynamic focusing system for wind turbine unmanned aerial vehicle (UAV) inspection.

[0005] The third objective of this invention is to provide an electronic device.

[0006] The fourth objective of this invention is to provide a computer-readable storage medium.

[0007] To achieve the above objectives, a first aspect of the present invention provides a method for dynamic focusing during wind turbine unmanned aerial vehicle (UAV) inspections, comprising the following steps:

[0008] Based on wind turbine operating parameters and UAV flight trajectory planning, a motion prediction model for wind turbine components is constructed to obtain the target core parameters at the next moment. The target core parameters include the predicted position coordinates of the wind turbine components, the predicted straight-line distance between the UAV and the wind turbine components, and the acceleration of distance change.

[0009] Based on the target's core parameters at the next moment, calculate and obtain the target focal length required by the gimbal camera at the next moment;

[0010] Establish a mapping relationship to obtain the advance of motion for adjusting the camera's focus motor;

[0011] Based on the motion lead, the camera's focusing motor performs dynamic drive adjustments and is corrected in real time based on encoder feedback to complete dynamic focusing adaptation.

[0012] The dynamic focusing method for wind turbine drone inspection according to embodiments of the present invention solves the problem that existing focusing technologies cannot match the dynamic changes in the distance between the drone and wind turbine components, ensuring that the drone can adjust the focus in a timely manner at different flight distances to obtain clear detailed images of wind turbine components, providing high-quality data support for subsequent fault detection.

[0013] In addition, the method for dynamic focusing of wind turbine UAV inspection proposed in the above embodiments of the present invention may also have the following additional technical features:

[0014] According to one embodiment of the present invention, before constructing the motion prediction model of the wind turbine components, the method further includes:

[0015] Images of different wind turbine models are collected, and the contour features, texture features, and color features of wind turbine components are extracted. A three-dimensional model and feature database of the wind turbine components are constructed using computer vision. The contour features include the streamlined edges of the wind turbine blades and the circular structure of the hub. The texture features include the coating texture on the surface of the wind turbine blades. The color features include the specific paint color of the nacelle. The wind turbine components include blades, hubs, and nacelles.

[0016] According to one embodiment of the present invention, the step of constructing a motion prediction model for wind turbine components based on wind turbine operating parameters and UAV flight trajectory planning, and obtaining the target core parameters at the next moment, includes:

[0017] The drone uses its onboard GPS to acquire the real-time position information of the wind turbine components, records the position changes of the wind turbine components in continuous frame images, and acquires the drone's flight speed, acceleration and attitude angle based on the IMU, and obtains the straight-line distance and distance change acceleration between the drone and the wind turbine components at the next moment;

[0018] Based on wind turbine operating parameters and UAV flight trajectory planning, a motion prediction model for wind turbine components is constructed to obtain the motion trend of the wind turbine relative to the UAV and the motion trend of wind turbine components relative to the UAV.

[0019] Based on the fusion of UAV motion data and wind turbine component displacement data using Kalman filtering, the position of the wind turbine component in the image at the next moment is predicted, and the predicted position coordinates are output.

[0020] According to an embodiment of the present invention, calculating the target focal length required by the gimbal camera at the next moment based on the target core parameters at the next moment includes:

[0021] Based on the geometric relationship of pinhole imaging and combined with the predicted straight-line distance at the next moment, the basic focal length of the camera at the next moment is obtained. The calculation formula is as follows:

[0022]

[0023] in, The focal length at the current moment. The predicted straight-line distance for the next moment. The straight-line distance at the current moment. For calibration coefficients;

[0024] Based on the positional offset of the wind turbine components in the image, positional compensation is added to the base focal length to obtain the compensated focal length. The formula for calculating the compensated focal length is as follows:

[0025]

[0026] in, The coordinates of the camera sensor center. The coordinates of the wind turbine in the image at the current moment. The coordinates of the wind turbine in the image are the predicted coordinates for the next moment. The position offset between the next moment and the current moment;

[0027] , ;

[0028] By combining the acceleration due to distance changes, the final target focal length to be adjusted is obtained. The formula for calculating the target focal length is:

[0029]

[0030] in, For the acceleration due to distance change, through Determine whether the fan is moving at a constant speed, accelerating, or decelerating. For dynamic weighting factors;

[0031] Dynamic weighting factor The rules for determining the value are as follows:

[0032] (at a constant speed) ;

[0033] (Accelerating away / approaching) ;

[0034] (Deceleration) .

[0035] According to one embodiment of the present invention, the step of establishing a mapping relationship and adjusting the advance of the camera focusing motor includes:

[0036] Based on the previous calibration, a mapping relationship between focal length and camera focusing motor step size is established;

[0037] Based on the mapping relationship, obtain the position of the camera focusing motor at the next moment corresponding to the target focal length;

[0038] Based on the position and distance change acceleration of the camera focusing motor at the next moment, the motion advance of the camera focusing motor is calculated, and the motion advance includes the start-up advance time and the motion speed.

[0039] According to one embodiment of the present invention, the dynamic focus adaptation is achieved by dynamically adjusting the camera's focus motor based on motion lead and correcting it in real time based on encoder feedback, including:

[0040] Based on the motion lead, the camera's focus motor is controlled to perform motion by outputting a PWM drive signal with an adjustable duty cycle;

[0041] The actual position of the encoder is read every 0.5ms, and the deviation from the target position is calculated. When the position deviation is greater than the threshold, the PWM duty cycle is finely adjusted through the PID algorithm to achieve deviation compensation. When the position deviation is less than the threshold, the current drive signal is maintained. When the position deviation is less than 1 / 2 of the threshold, it is determined that the camera focusing motor has reached the target position, the drive signal is stopped, and dynamic focusing adaptation is completed.

[0042] A second aspect of the present invention provides a dynamic focusing system for wind turbine unmanned aerial vehicle (UAV) inspection, comprising:

[0043] The model building module constructs a motion prediction model of the wind turbine components based on the wind turbine operating parameters and the UAV flight trajectory planning, and obtains the target core parameters at the next moment. The target core parameters include the predicted position coordinates of the wind turbine components, the predicted straight-line distance between the UAV and the wind turbine components, and the acceleration of distance change.

[0044] The calculation module calculates the target focal length required by the gimbal camera in the next moment based on the target's core parameters at the next moment.

[0045] The module acquires the mapping relationship and obtains the advance of motion for adjusting the camera's focus motor.

[0046] The control module dynamically drives and adjusts the camera's focusing motor based on motion lead and corrects it in real time based on encoder feedback, thus completing dynamic focusing adaptation.

[0047] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the dynamic focusing method for wind turbine drone inspection as described in any of the preceding claims.

[0048] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dynamic focusing method for wind turbine unmanned aerial vehicle inspection as described in any of the preceding claims.

[0049] Compared with the prior art, the beneficial effects of the present invention are: no need for operators to manually adjust the focus, automatic target recognition and dynamic focus control are used to achieve automatic clear imaging of wind turbine components, reduce manual intervention, shorten the inspection time of a single wind turbine, and greatly improve the inspection efficiency of large-scale wind farms; based on target motion trajectory prediction and pre-focus control, dynamic targets such as wind turbine rotating blades can be accurately tracked, the focus response speed is fast, the "out-of-focus" rate is reduced, and the image clarity is ensured in dynamic scenes. Attached Figure Description

[0050] Figure 1 This is a flowchart of the dynamic focusing method for wind turbine unmanned aerial vehicle (UAV) inspection according to the present invention;

[0051] Figure 2 This is a detailed flowchart of step S1 of the present invention;

[0052] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0053] The following specific examples illustrate the implementation of this patent application. Those skilled in the art can easily understand other advantages and effects of this patent application from the content disclosed in this specification. This patent application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this patent application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0054] The method and system for dynamic focusing of wind turbine drone inspection provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0055] In this embodiment, a dynamic focusing system for wind turbine drone inspection can be used to execute a dynamic focusing method for wind turbine drone inspection within an automatic wind turbine inspection system. Based on target motion trajectory prediction and pre-focusing control, dynamic targets such as rotating blades of the wind turbine can be accurately tracked, ensuring image clarity in dynamic scenes.

[0056] The wind turbine drone inspection dynamic focusing system includes:

[0057] The model building module constructs a motion prediction model of the wind turbine components based on the wind turbine operating parameters and the UAV flight trajectory planning, and obtains the target core parameters at the next moment. The target core parameters include the predicted position coordinates of the wind turbine components, the predicted straight-line distance between the UAV and the wind turbine components, and the acceleration of distance change.

[0058] The calculation module calculates the target focal length required by the gimbal camera in the next moment based on the target's core parameters at the next moment.

[0059] The module acquires the mapping relationship and obtains the advance of motion for adjusting the camera's focus motor.

[0060] The control module dynamically drives and adjusts the camera's focusing motor based on motion lead and corrects it in real time based on encoder feedback, thus completing dynamic focusing adaptation.

[0061] This system can be applied to a terminal, specifically executed by the hardware or software within the terminal.

[0062] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0063] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0064] The wind turbine drone inspection dynamic focusing method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can realize the wind turbine drone inspection dynamic focusing method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras and wearable devices. The following uses an electronic device as the execution subject to describe the wind turbine drone inspection dynamic focusing method provided in this application embodiment.

[0065] Figure 1 This is a flowchart of a method for dynamic focusing during wind turbine drone inspection according to an embodiment of the present invention, as follows: Figure 1 As shown, a method for dynamic focusing during wind turbine drone inspection includes the following steps:

[0066] S1. Based on the wind turbine operating parameters and the UAV flight trajectory planning, construct a motion prediction model for the wind turbine components and obtain the target core parameters at the next moment. The target core parameters include the predicted position coordinates of the wind turbine components, the predicted straight-line distance between the UAV and the wind turbine components, and the acceleration of distance change.

[0067] Specifically, such as Figure 2 As shown:

[0068] S11. The UAV uses its onboard GPS to obtain the position information of the wind turbine components in real time, records the position changes of the wind turbine components in continuous frame images, obtains the flight speed, acceleration and attitude angle of the UAV based on the IMU, and obtains the straight distance and distance change acceleration between the UAV and the wind turbine components at the next moment.

[0069] S12. Based on the wind turbine operating parameters and UAV flight trajectory planning, construct a motion prediction model for wind turbine components, obtain the motion trend of the wind turbine relative to the UAV, and obtain the motion trend of wind turbine components relative to the UAV.

[0070] In practice, for example, for rotating wind turbine blades, the circular motion trajectory and linear velocity of a point on the blade (such as the blade tip) are calculated based on its rotational speed; for drone flight, the position and attitude of the drone at the next moment are predicted based on GPS and IMU data, and then the motion trend of the wind turbine components relative to the drone is deduced.

[0071] S13. Based on the fusion of UAV motion data and wind turbine component displacement data using Kalman filtering, the position of the wind turbine component in the image at the next moment is predicted, and the predicted position coordinates are output to provide data support for dynamic focusing and avoid defocusing due to focusing lag.

[0072] S2. Based on the target's core parameters at the next moment, calculate the target focal length required by the gimbal camera at the next moment. The specific calculation process includes the following steps:

[0073] First, based on the geometric relationship of pinhole imaging and combined with the predicted straight-line distance at the next moment, the basic focal length of the camera at the next moment is obtained. The calculation formula is as follows:

[0074]

[0075] in, The focal length at the current moment. The predicted straight-line distance for the next moment. The straight-line distance at the current moment. The calibration coefficients were pre-calibrated through multiple sets of experiments, with values ​​ranging from 0.98 to 1.02, to compensate for the prediction error of the Kalman filter.

[0076] If the predicted straight-line distance This means that the wind turbine components are relatively far away from the drone. To make distant targets clear, the focal length needs to be increased; conversely, the focal length should be decreased.

[0077] The positional offset of the wind turbine components in the image causes them to deviate from the optical axis center. If only the base focal length is adjusted, the edges may appear blurry due to slightly lower sharpness at the edge of the camera's field of view. Therefore, positional compensation is added to the base focal length to obtain a compensating focal length. This small increase in focal length compensates for the resolution loss at the edges of the field of view, ensuring the overall image of the wind turbine components remains sharp. The formula for calculating the compensating focal length is as follows:

[0078]

[0079] in, The coordinates of the camera sensor center. The coordinates of the wind turbine in the image at the current moment. The coordinates of the wind turbine in the image are the predicted coordinates for the next moment. The position offset between the next moment and the current moment;

[0080] , ;

[0081] By combining the acceleration of distance changes, the final target focal length to be adjusted is obtained. The motion state of the wind turbine components is sensed through acceleration, avoiding the problem that "uniform speed prediction" cannot adapt to variable speed motion, such as the speed change of the wind turbine blades during startup or braking. The formula for calculating the target focal length is:

[0082]

[0083] in, For the acceleration due to distance change, through Determine whether the fan is moving at a constant speed, accelerating, or decelerating. For dynamic weighting factors;

[0084] Dynamic weighting factor The rules for determining the value are as follows:

[0085] (at a constant speed) ;

[0086] (Accelerating away / approaching) ;

[0087] (Deceleration) .

[0088] In some examples, the formula for calculating the acceleration due to distance change is:

[0089] ;

[0090] in, To predict the rate of change of distance at the next moment, To predict the rate of change of distance at the current time from the previous time step, For frame interval;

[0091] The formula for predicting the rate of change of distance at the next moment is:

[0092] .

[0093] S3. Establish a mapping relationship and obtain the advance of motion for adjusting the camera's focus motor;

[0094] Based on the previous calibration, a mapping relationship between focal length and camera focusing motor step size is established;

[0095] Because the camera's focusing motor is a precision linear drive, the mapping relationship between the focal length and the camera's focusing motor step size is approximately linear, expressed as:

[0096]

[0097] in, This refers to the step size of the camera's focusing motor. For the camera's focal length, This is the step size coefficient. Zero-point offset, , It needs to be obtained by fitting multiple sets of calibration data.

[0098] Based on the mapping relationship, the position of the camera's focusing motor at the next moment corresponding to the target focal length is obtained, and the following formula is obtained by applying the mapping relationship expression:

[0099]

[0100] in, The step size of the camera's focusing motor for the predicted next moment. The predicted focal length of the camera at the next moment.

[0101] Based on the position and distance change acceleration of the camera's focusing motor at the next moment, the motion lead of the camera's focusing motor is calculated. The motion lead includes the start-up advance time and the motion velocity. The specific calculation process is as follows:

[0102] Let the current time be t, and the next time the wind turbine enters the field of view be t+1, with a time interval of t. (Frame interval, usually 10ms) to ensure that the camera's focusing motor reaches its maximum speed at time t+1. It is necessary to calculate the start-up time and movement speed of the camera's focusing motor;

[0103] Step 1: Calculate the required adjustment step size for the camera's focus motor. The formula is:

[0104]

[0105] Step 2: Calculate the time required for the camera's focus motor to complete the adjustment. The formula is:

[0106]

[0107] in, The rated movement speed of the camera's focusing motor, typically 50-100 steps / ms, can be dynamically adjusted;

[0108] Step 3: Determine the advance start time, and the adjustment rules are as follows:

[0109] like (If the camera's focusing motor can complete the adjustment within one frame), then the motor will start immediately at time t in the current frame. Uniform motion, ensuring t+ Arrive at the right time;

[0110] like (If the adjustment step size is too large, it cannot be completed within one frame), then... The adjustment is broken down into multiple iterative frames, with each frame adjusting a portion of the step size while simultaneously updating the next prediction. To avoid motor overload, the iterative adjustment formula is as follows:

[0111] ;

[0112] To avoid focusing shake caused by the start-stop shock of the camera's focusing motor, the system adjusts the focusing speed according to changes in acceleration. The formula for adjusting the speed of the camera's focus motor is:

[0113]

[0114] in, This is the maximum speed of the camera's focusing motor. When the fan blades accelerate more (such as with sudden acceleration), the motor speed is closer to the maximum value to ensure fast focusing. When the acceleration is smaller, the motor moves at a low speed and smoothly to improve focusing accuracy.

[0115] S4. Based on the motion lead, the camera's focusing motor performs dynamic drive adjustment and real-time feedback correction based on the encoder to avoid mechanical errors and complete dynamic focusing adaptation.

[0116] Based on the motion lead, the camera's focus motor is controlled to perform motion by outputting a PWM drive signal with an adjustable duty cycle;

[0117] The actual position of the encoder is read every 0.5ms. Calculate the target position deviation ;

[0118] When the position deviation is greater than the threshold, that is The PID algorithm is used to fine-tune the PWM duty cycle to achieve deviation compensation.

[0119] When the position deviation is less than the threshold, i.e. Maintain the current drive signal;

[0120] Position confirmation occurs when the position deviation is less than half of the threshold. Once it is determined that the camera's focusing motor has reached the target position, the drive signal is stopped, dynamic focusing adaptation is completed, and the system waits for the next frame's prediction parameters to be updated.

[0121] The dynamic focusing method for wind turbine drone inspection according to embodiments of the present invention solves the problem that existing focusing technologies cannot match the dynamic changes in the distance between the drone and wind turbine components, ensuring that the drone can adjust the focus in a timely manner at different flight distances to obtain clear detailed images of wind turbine components, providing high-quality data support for subsequent fault detection.

[0122] In some examples, before constructing the motion prediction model of wind turbine components, a three-dimensional model and feature database of the wind turbine components should be constructed using computer vision technology. Images of different models of wind turbines should be collected, and the contour features, texture features, and color features of the wind turbine components should be extracted. A feature template library of wind turbine components should be established for rapid target identification during inspection. The contour features include the streamlined edges of the wind turbine blades and the circular structure of the hub. The texture features include the coating texture on the surface of the wind turbine blades. The color features include the specific paint color of the nacelle. The wind turbine components include blades, hubs, and nacelles.

[0123] The wind turbine drone inspection dynamic focusing system in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific implementation.

[0124] The wind turbine drone inspection dynamic focusing system in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit its application.

[0125] The wind turbine drone inspection dynamic focusing system provided in this application embodiment can achieve... Figures 1 to 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0126] In some embodiments, such as Figure 3 As shown, this application embodiment also provides an electronic device 700, including a processor 701, a memory 702, and a computer program stored in the memory 702 and executable on the processor 701. When the program is executed by the processor 701, it implements the various processes of the above-described method embodiment for dynamic focusing of wind turbine drone inspection and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0127] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0128] The present invention also proposes a computer-readable storage medium storing a computer program, which is configured to run and implement the dynamic focusing method for wind turbine drone inspection as proposed in the embodiments of the present invention.

[0129] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0130] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0131] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0132] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0133] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0134] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0135] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0136] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for dynamic focusing during wind turbine unmanned aerial vehicle (UAV) inspection, characterized in that, Includes the following steps: Based on wind turbine operating parameters and UAV flight trajectory planning, a motion prediction model for wind turbine components is constructed to obtain the target core parameters at the next moment. The target core parameters include the predicted position coordinates of the wind turbine components, the predicted straight-line distance between the UAV and the wind turbine components, and the acceleration of distance change. Based on the target's core parameters at the next moment, calculate and obtain the target focal length required by the gimbal camera at the next moment; Establish a mapping relationship to obtain the advance of motion for adjusting the camera's focus motor; Based on the motion lead, the camera's focusing motor performs dynamic drive adjustments and is corrected in real time based on encoder feedback to complete dynamic focusing adaptation.

2. The method for dynamic focusing during wind turbine unmanned aerial vehicle (UAV) inspection according to claim 1, characterized in that, Before constructing the motion prediction model for the wind turbine components, the following is also included: Images of different wind turbine models are collected, and the contour features, texture features, and color features of wind turbine components are extracted. A three-dimensional model and feature database of the wind turbine components are constructed using computer vision. The contour features include the streamlined edges of the wind turbine blades and the circular structure of the hub. The texture features include the coating texture on the surface of the wind turbine blades. The color features include the specific paint color of the nacelle. The wind turbine components include blades, hubs, and nacelles.

3. The method for dynamic focusing during wind turbine unmanned aerial vehicle (UAV) inspection according to claim 2, characterized in that, The method involves constructing a motion prediction model for wind turbine components based on wind turbine operating parameters and UAV flight trajectory planning, and obtaining the target's core parameters for the next moment, including: The drone uses its onboard GPS to acquire the real-time position information of the wind turbine components, records the position changes of the wind turbine components in continuous frame images, and acquires the drone's flight speed, acceleration and attitude angle based on the IMU, and obtains the straight-line distance and distance change acceleration between the drone and the wind turbine components at the next moment; Based on wind turbine operating parameters and UAV flight trajectory planning, a motion prediction model for wind turbine components is constructed to obtain the motion trend of the wind turbine relative to the UAV and the motion trend of wind turbine components relative to the UAV. Based on the fusion of UAV motion data and wind turbine component displacement data using Kalman filtering, the position of the wind turbine component in the image at the next moment is predicted, and the predicted position coordinates are output.

4. The method for dynamic focusing during wind turbine unmanned aerial vehicle (UAV) inspection according to claim 3, characterized in that, The calculation of the target focal length required by the gimbal camera at the next moment based on the target's core parameters at the next moment includes: Based on the geometric relationship of pinhole imaging and combined with the predicted straight-line distance at the next moment, the basic focal length of the camera at the next moment is obtained. The calculation formula is as follows: in, The focal length at the current moment. The predicted straight-line distance for the next moment. The straight-line distance at the current moment. For calibration coefficients; Based on the positional offset of the wind turbine components in the image, positional compensation is added to the base focal length to obtain the compensated focal length. The formula for calculating the compensated focal length is as follows: in, The coordinates of the camera sensor center. The coordinates of the wind turbine in the image at the current moment. The coordinates of the wind turbine in the image are the predicted coordinates for the next moment. The position offset between the next moment and the current moment; ， ； By combining the acceleration due to distance changes, the final target focal length to be adjusted is obtained. The formula for calculating the target focal length is: in, For the acceleration due to distance change, through Determine whether the fan is moving at a constant speed, accelerating, or decelerating. For dynamic weighting factors; Dynamic weighting factor The rules for determining the value are as follows: (at a constant speed) ; (Accelerating away / approaching) ; (Deceleration) .

5. The method for dynamic focusing during wind turbine unmanned aerial vehicle (UAV) inspection according to claim 4, characterized in that, The process of establishing a mapping relationship and adjusting the advance of the camera's focusing motor includes: Based on the previous calibration, a mapping relationship between focal length and camera focusing motor step size is established; Based on the mapping relationship, obtain the position of the camera focusing motor at the next moment corresponding to the target focal length; Based on the position and distance change acceleration of the camera focusing motor at the next moment, the motion advance of the camera focusing motor is calculated, and the motion advance includes the start-up advance time and the motion speed.

6. The method for dynamic focusing during wind turbine unmanned aerial vehicle (UAV) inspection according to claim 5, characterized in that, The dynamic focus adaptation is achieved by dynamically adjusting the camera's focusing motor based on motion lead and correcting it in real time based on encoder feedback, including: Based on the motion lead, the camera's focus motor is controlled to perform motion by outputting a PWM drive signal with an adjustable duty cycle; The actual position of the encoder is read every 0.5ms, and the deviation from the target position is calculated. When the position deviation is greater than the threshold, the PWM duty cycle is finely adjusted through the PID algorithm to achieve deviation compensation. When the position deviation is less than the threshold, the current drive signal is maintained. When the position deviation is less than 1 / 2 of the threshold, it is determined that the camera focusing motor has reached the target position, the drive signal is stopped, and dynamic focusing adaptation is completed.

7. A dynamic focusing system for wind turbine unmanned aerial vehicle (UAV) inspection, characterized in that, include: The model building module constructs a motion prediction model of the wind turbine components based on the wind turbine operating parameters and the UAV flight trajectory planning, and obtains the target core parameters at the next moment. The target core parameters include the predicted position coordinates of the wind turbine components, the predicted straight-line distance between the UAV and the wind turbine components, and the acceleration of distance change. The calculation module calculates the target focal length required by the gimbal camera in the next moment based on the target's core parameters at the next moment. The module acquires the mapping relationship and obtains the advance of motion for adjusting the camera's focus motor. The control module dynamically drives and adjusts the camera's focusing motor based on motion lead and corrects it in real time based on encoder feedback, thus completing dynamic focusing adaptation.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the dynamic focusing method for wind turbine drone inspection as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the dynamic focusing method for wind turbine drone inspection as described in any one of claims 1-6.

Images