Method, device and storage medium for cleaning multi-sensor fusion positioning control of unmanned aerial vehicle

Abstract

The application discloses a multi-sensor fusion positioning control method and device for cleaning unmanned aerial vehicles and a storage medium, relates to the technical field of unmanned aerial vehicle control, and comprises the following steps: acquiring laser radar data, visual image data and millimeter wave radar data of the same timestamp; determining weight coefficients of the laser radar data, the visual image data and the millimeter wave radar data according to the working states of the laser radar, the visual sensor and the millimeter wave radar and environmental data; based on the weight coefficients, first performing bottom-layer fusion on the laser radar data and the visual image data to obtain intermediate positioning data, and then performing middle-layer fusion on the intermediate positioning data and the millimeter wave radar data to obtain target positioning data; and adjusting the flight pose of the cleaning unmanned aerial vehicle and the control parameters of a cleaning module according to the target positioning data. The application can achieve the technical effects of stable flight and effective cleaning of the unmanned aerial vehicle when the unmanned aerial vehicle approaches a working surface.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a multi-sensor fusion positioning and control method, device and storage medium for a cleaning UAV. Background Technology

[0002] With the continuous increase in global photovoltaic power plant installed capacity and the growing demand for cleaning the exterior walls of high-rise buildings in cities, drone-based automated cleaning is gradually replacing traditional manual high-altitude operations and becoming a core technology direction in the fields of photovoltaic operation and maintenance and building cleaning, thanks to its significant advantages such as high operating efficiency, low labor costs, and low safety risks.

[0003] The core prerequisite for autonomous drone cleaning operations is high-precision and highly stable real-time positioning. Currently, the industry-wide satellite positioning technology can provide centimeter-level positioning accuracy in open, unobstructed outdoor environments, but it has insurmountable technical limitations in near-surface operations such as those involving walls or solar panels. When drones operate close to building exteriors or solar panel arrays, the wall structure, solar panel supports, and components severely obstruct satellite signals and cause multipath reflections, leading to a sharp attenuation or even complete loss of satellite signal strength, resulting in positioning errors increasing dramatically to several meters or more. Summary of the Invention

[0004] The main objective of this application is to provide a multi-sensor fusion positioning and control method, device, and storage medium for cleaning drones, aiming to solve the technical problem that when drones operate close to building exterior walls or photovoltaic panel arrays, the wall structure, photovoltaic panel supports, and components will cause severe obstruction and multipath reflection effects on satellite signals, resulting in a sharp attenuation of satellite signal strength.

[0005] To achieve the above objectives, this application provides a multi-sensor fusion positioning and control method for a cleaning drone, applied to a cleaning drone. The sensors of the cleaning drone include lidar, visual sensors, and millimeter-wave radar. The multi-sensor fusion positioning and control method for the cleaning drone includes: Acquire LiDAR data, visual image data, and millimeter-wave radar data at the same timestamp; Based on the operating status and environmental data of the lidar, the vision sensor, and the millimeter-wave radar, the weighting coefficients of the lidar data, the visual image data, and the millimeter-wave radar data are determined. Based on the weighting coefficients, the lidar data and the visual image data are first fused at the bottom layer to obtain intermediate positioning data, and then the intermediate positioning data and the millimeter-wave radar data are fused at the middle layer to obtain target positioning data. The flight attitude of the cleaning drone and the control parameters of the cleaning module are adjusted based on the target positioning data.

[0006] In one embodiment, acquiring lidar data, visual image data, and millimeter-wave radar data at the same timestamp includes: Acquire raw point cloud data from LiDAR, raw visual image data, and raw echo data from millimeter-wave radar at the same timestamp; The raw point cloud data of the lidar is subjected to statistical filtering, the average distance between each point and a preset number of neighboring points is calculated, discrete noise points with a distance greater than the mean plus a preset multiple of the standard deviation are removed, and the planar features and edge features of the cleaned target surface are extracted to obtain the lidar data. Gaussian denoising is performed on the original visual image data to extract SIFT or SURF feature points from the image, and invalid feature points in areas with gray-level variance lower than a preset reflectivity threshold are removed to obtain the visual image data. The raw echo data from the millimeter-wave radar is subjected to constant false alarm rate (CFAR) filtering, and the signal-to-noise ratio (SNR) threshold and radar cross section (RCS) threshold are adjusted according to the water mist concentration of the current cleaning operation to obtain the millimeter-wave radar data.

[0007] In one embodiment, determining the weighting coefficients of the lidar data, the visual image data, and the millimeter-wave radar data based on the operating status and environmental data of the lidar, the visual sensor, and the millimeter-wave radar includes: The point cloud density and point cloud plane fitting degree of the lidar are obtained, the number of effective feature points and feature point matching success rate of the vision sensor are obtained, and the echo signal strength and target detection confidence of the millimeter-wave radar are obtained as the working state. The ambient light intensity, air humidity, water mist concentration, and reflectivity of the target surface being cleaned are obtained as the environmental data for the current working environment. The weighting coefficients of the lidar data, the visual image data, and the millimeter-wave radar data are determined based on the operating status and the environmental data.

[0008] In one embodiment, determining the weighting coefficients of the lidar data, the visual image data, and the millimeter-wave radar data based on the operating state and the environmental data includes: Determine the working status score of each sensor based on the described working status; Determine the environmental correction coefficients for each sensor based on the environmental data; The working status scores of each sensor are multiplied by the corresponding environmental correction coefficients to obtain the comprehensive scores of LiDAR, vision sensors, and millimeter-wave radar. The weighting coefficients for the lidar, vision sensor, and millimeter-wave radar are determined based on the comprehensive score of the lidar, the comprehensive score of the vision sensor, and the comprehensive score of the millimeter-wave radar.

[0009] In one embodiment, the weighting coefficients include lidar weighting coefficients, visual sensor weighting coefficients, and millimeter-wave radar weighting coefficients. The step of first performing a low-level fusion of the lidar data and the visual image data to obtain intermediate positioning data based on these weighting coefficients, and then performing a mid-level fusion of the intermediate positioning data and the millimeter-wave radar data to obtain target positioning data, includes: Based on the extrinsic parameter matrix between the lidar and the visual sensor, as well as the lidar weight coefficient and the visual sensor weight coefficient, the intermediate positioning data is obtained by fusing the lidar data and the visual image data. The intermediate positioning data is converted to the millimeter-wave radar coordinate system and timestamped with the target distance and angle information extracted from the millimeter-wave radar data. Initial positioning data is obtained by combining the intermediate positioning data after weighted fusion and alignment based on the millimeter-wave radar weight coefficients with the millimeter-wave radar data. The target positioning data is determined based on the jump magnitude of the initial positioning data.

[0010] In one embodiment, the intermediate positioning data is obtained by fusing the lidar data and the visual image data based on the extrinsic parameter matrix between the lidar and the visual sensor, the lidar weight coefficients, and the visual sensor weight coefficients, including: Based on the pre-calibrated extrinsic matrix between the lidar and the vision sensor, the pre-processed lidar data and vision image data are unified under the UAV coordinate system. The extrinsic matrix is ​​obtained through factory calibration or online calibration. An extended Kalman filter state equation is constructed, with the three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle of the cleaning UAV as the system state vector, and the power system input of the cleaning UAV as the control input. The target surface distance and plane normal vector extracted from the preprocessed lidar data are used as the first observation vector, and the lateral offset and rotation angle calculated from the feature point matching results extracted from the preprocessed visual image data are used as the second observation vector. The first and second observation vectors are input into the extended Kalman filter for updating. The high-precision distance information of the lidar data is used to correct the cumulative error caused by feature point drift in the visual image data, and the texture features of the visual image data are used to correct the lateral positioning deviation of the lidar data. The output includes intermediate positioning data containing the relative position and attitude angle of the cleaning drone relative to the surface of the cleaning target.

[0011] In one embodiment, determining the target positioning data based on the jump magnitude of the initial positioning data includes: The initial positioning data is smoothed and filtered. When the amplitude of the initial positioning data jump exceeds a preset threshold, the weight coefficient of the target sensor is reduced and the fusion calculation is performed again. The output target positioning data is processed by smoothing and filtering. The target positioning data includes at least the vertical distance, horizontal offset, pitch angle, roll angle and yaw angle of the cleaning drone relative to the surface of the cleaning target.

[0012] In one embodiment, adjusting the flight attitude of the cleaning drone and the control parameters of the cleaning module based on the target positioning data includes: Extract the vertical distance, horizontal offset, pitch angle, roll angle, and yaw angle of the UAV relative to the target surface to be cleaned from the target positioning data; The vertical distance is compared with a preset benchmark working distance threshold to generate a height control command; Based on the pitch angle, roll angle and yaw angle, attitude control commands are generated to control the UAV to adjust its attitude so that the UAV fuselage remains parallel to the surface of the target being cleaned. Based on the horizontal offset, a horizontal movement control command is generated so that the cleaning drone moves horizontally along the surface of the cleaning target and remains on the preset cleaning trajectory. Adjust the water pump output pressure of the cleaning module according to the vertical distance; Based on the target surface type in the target positioning data, adjust the nozzle angle and water flow rate of the cleaning module.

[0013] In addition, to achieve the above objectives, this application also provides a multi-sensor fusion positioning and control device for a cleaning drone. The multi-sensor fusion positioning and control device for a cleaning drone includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the multi-sensor fusion positioning and control method for a cleaning drone as described above.

[0014] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium, storing a program for implementing a multi-sensor fusion positioning and control method for a cleaning drone. The program for implementing the multi-sensor fusion positioning and control method for a cleaning drone is executed by a processor to implement the steps of the multi-sensor fusion positioning and control method for a cleaning drone as described above.

[0015] This application provides a multi-sensor fusion positioning and control method for a cleaning drone. First, it acquires lidar data, visual image data, and millimeter-wave radar data at the same timestamp to obtain multi-source independent environmental perception data, completely eliminating reliance on GNSS satellite signals. Then, based on the operating status of the lidar, visual sensor, and millimeter-wave radar, as well as environmental data, it determines weighting coefficients for each data source to dynamically adjust the fusion contribution of each sensor according to the actual operational scenario. Next, based on these weighting coefficients, it first performs low-level fusion of the lidar data and visual image data to obtain intermediate positioning data, and then performs mid-level fusion of the intermediate positioning data and millimeter-wave radar data to obtain target positioning data. This achieves accurate and robust positioning results through multi-source data complementarity. Finally, it adjusts the flight attitude of the cleaning drone and the control parameters of the cleaning module based on the target positioning data to ensure stable flight and effective cleaning when the drone is close to the work surface.

[0016] In summary, this application overcomes the technical defects of positioning failure caused by the severe obstruction and multipath reflection effects of wall structures, photovoltaic panel supports and components on satellite signals when UAVs operate close to building exteriors or photovoltaic panel arrays, leading to a sharp attenuation of GNSS signal strength and thus positioning failure. This is achieved by combining multi-source data fusion technology of lidar, visual sensors and millimeter-wave radar, and adopting a positioning strategy of dynamic weight allocation and hierarchical fusion. It realizes continuous, stable and high-precision positioning in GNSS-free environments, ensuring the safety and efficiency of cleaning operations. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the multi-sensor fusion positioning and control method for cleaning drones in this application. Figure 2 This is a flowchart illustrating Embodiment 3 of the multi-sensor fusion positioning and control method for cleaning drones in this application. Figure 3 This is a schematic diagram of the hardware structure involved in the multi-sensor fusion positioning and control device for cleaning drones in this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] Currently, the core prerequisite for autonomous drone cleaning operations is high-precision and high-stability real-time positioning. While the GNSS satellite positioning technology widely used in the industry can provide centimeter-level positioning accuracy in open, unobstructed outdoor scenarios, it has insurmountable technical limitations in near-surface operations such as those involving walls or solar panels. When drones operate close to building exteriors or solar panel arrays, the wall structure, solar panel supports, and components severely obstruct satellite signals and cause multipath reflections, leading to a sharp attenuation or even complete loss of GNSS signal strength, resulting in positioning errors increasing dramatically to several meters or more.

[0024] The main solution of this application is as follows: Acquire lidar data, visual image data, and millimeter-wave radar data at the same timestamp; determine weighting coefficients for the lidar data, visual image data, and millimeter-wave radar data based on the operating status and environmental data of the lidar, the visual sensor, and the millimeter-wave radar; based on the weighting coefficients, first perform bottom-level fusion of the lidar data and the visual image data to obtain intermediate positioning data, then perform mid-level fusion of the intermediate positioning data and the millimeter-wave radar data to obtain target positioning data; adjust the flight attitude of the cleaning drone and the control parameters of the cleaning module according to the target positioning data.

[0025] This application overcomes the technical defects of positioning failure caused by the severe obstruction and multipath reflection effects of wall structures, photovoltaic panel supports and components on satellite signals when UAVs operate close to building exteriors or photovoltaic panel arrays, leading to a sharp attenuation of GNSS signal strength and thus positioning failure. It achieves continuous, stable and high-precision positioning in GNSS-free environments, ensuring the safety and efficiency of cleaning operations.

[0026] It should be noted that the executing entity in this embodiment can be a cleaning drone, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a multi-sensor fusion positioning and control device for a cleaning drone capable of the above functions. This embodiment does not specifically limit the specific implementation. The following uses a cleaning drone as the executing entity to describe this embodiment and the following embodiments.

[0027] Based on this, Embodiment 1 of this application proposes a multi-sensor fusion positioning and control method for cleaning drones. Please refer to... Figure 1 The multi-sensor fusion positioning and control method for the cleaning drone includes steps S10 to S40: S10: Acquire lidar data, visual image data, and millimeter-wave radar data at the same timestamp.

[0028] In this embodiment, the same timestamp means that the system time base corresponding to the data collected by the lidar, vision sensor and millimeter-wave radar is completely consistent, and the synchronization error of data acquisition does not exceed 1 millisecond.

[0029] As an optional implementation, the flight control system simultaneously sends a unified acquisition trigger signal to the three sensors. Upon receiving the trigger signal, the three sensors initiate data acquisition at the same moment, appending a corresponding system timestamp to the acquired raw data before sending it to the flight control system. The flight control system performs preliminary noise filtering and effective information extraction on the received raw data, generating lidar data, visual image data, and millimeter-wave radar data, respectively.

[0030] As an alternative implementation, the flight control system employs a combination of hardware synchronization and software interpolation to achieve time synchronization of multi-sensor data. The flight control system provides a unified clock reference to the three sensors via a hardware clock line, enabling each sensor to independently acquire data at a preset frequency. Based on the acquisition frequency and timestamp of each sensor, the flight control system performs linear interpolation on the data from the sensor with the lower acquisition frequency, generating synchronized data that is perfectly aligned with the timestamp of the data from the sensor with the highest acquisition frequency. This implementation solves the problem of insufficient time synchronization accuracy caused by differences in the acquisition frequencies of different sensors, further improving the accuracy of subsequent data fusion.

[0031] S20, based on the operating status and environmental data of the lidar, the vision sensor, and the millimeter-wave radar, determine the weighting coefficients of the lidar data, the visual image data, and the millimeter-wave radar data.

[0032] In this embodiment, the weighting coefficient refers to the proportion of each sensor data in the subsequent fusion calculation process. The magnitude of the weighting coefficient reflects the reliability and effectiveness of the corresponding sensor data in the current working environment.

[0033] As an optional implementation, the flight control system collects the operating status parameters of each sensor and environmental data of the current operating environment in real time, and calculates the comprehensive effective score of each sensor. The flight control system determines the weighting coefficient corresponding to each sensor's data based on the proportion of each sensor's comprehensive effective score to the total comprehensive effective score of all sensors. The sum of the weighting coefficients of all sensors is 1.

[0034] As an alternative implementation, the flight control system incorporates a historical weight smoothing factor when calculating the weight coefficients. The flight control system calculates a weighted average of the initial weight coefficients obtained at the current moment and the final weight coefficients at the previous moment to obtain the final weight coefficients at the current moment. This implementation avoids sudden changes in weight coefficients caused by instantaneous fluctuations in sensor data, effectively suppressing positioning result jitter and improving the stability of the UAV flight.

[0035] S30, based on the weighting coefficients, first perform bottom-level fusion of the lidar data and the visual image data to obtain intermediate positioning data, and then perform mid-level fusion of the intermediate positioning data and the millimeter-wave radar data to obtain target positioning data.

[0036] In this embodiment, hierarchical fusion refers to dividing multi-sensor data into two fusion levels according to accuracy level and complementary characteristics for step-by-step processing. First, sensor data with higher accuracy and strong complementarity are fused to obtain preliminary positioning results. Then, redundant sensor data is introduced to verify and correct the preliminary positioning results.

[0037] As an optional implementation, the flight control system first unifies the lidar data and visual image data into the same body coordinate system. It then fuses the two types of data based on the lidar weighting coefficient and the visual sensor weighting coefficient to obtain intermediate positioning data containing the relative position and attitude information of the UAV relative to the target surface being cleaned. The flight control system then converts the intermediate positioning data to a millimeter-wave radar coordinate system and fuses the intermediate positioning data and the millimeter-wave radar data based on the millimeter-wave radar weighting coefficient to obtain the final target positioning data.

[0038] As an alternative implementation, the flight control system employs a state estimation fusion algorithm in the low-level fusion stage and a decision-level fusion algorithm in the mid-level fusion stage. The flight control system fuses the raw measurement data from the LiDAR and visual sensors using the low-level fusion algorithm to obtain the UAV's state estimate. The mid-level fusion algorithm then performs a decision-level fusion of the state estimate obtained from the low-level fusion with the target detection results from the millimeter-wave radar, correcting and optimizing the state estimate. This implementation fully leverages the advantages of different fusion algorithms, improving the real-time performance of the fusion calculation while maintaining positioning accuracy.

[0039] S40, adjust the flight attitude of the cleaning drone and the control parameters of the cleaning module according to the target positioning data.

[0040] In this embodiment, the flight attitude includes the vertical distance and horizontal offset of the UAV relative to the target surface to be cleaned, as well as the pitch angle, roll angle and yaw angle of the UAV.

[0041] As an optional implementation, the flight control system extracts the UAV's flight attitude parameters from the target positioning data, compares these parameters with preset baseline operating parameters, and generates corresponding flight control commands to control the UAV to adjust its flight altitude, horizontal position, and attitude angle. Simultaneously, the flight control system adjusts control parameters of the cleaning module, such as the water pump output pressure, nozzle angle, and water flow rate, based on the vertical distance between the UAV and the target surface being cleaned.

[0042] As an alternative implementation, the flight control system employs a feedforward control strategy to adjust the UAV's flight attitude and the control parameters of the cleaning module. Based on multiple consecutive frames of target positioning data, the flight control system calculates the UAV's motion trend and attitude change rate, generating control commands in advance to adjust the UAV's flight attitude. Simultaneously, the flight control system adjusts the cleaning module's control parameters based on the UAV's motion trend, ensuring the cleaning effect remains optimal. This implementation solves the lag problem inherent in traditional feedback control, improving the UAV's response speed and control accuracy.

[0043] For example, when a cleaning drone is performing cleaning operations on photovoltaic power plant components, the GNSS signal strength gradually weakens as the drone flies above and approaches the surface of the photovoltaic panel array. The drone automatically activates a multi-sensor fusion positioning mode. The flight control system sends a unified acquisition trigger signal to the lidar, visual sensor, and millimeter-wave radar. The three sensors simultaneously collect environmental data from the photovoltaic panel surface and attach a unified timestamp. The flight control system collects the working status parameters of each sensor and environmental data such as the current light intensity and water mist concentration in real time, calculates the comprehensive effective score of each sensor, and determines the weight coefficient of each sensor data based on the comprehensive effective score. The flight control system first performs low-level fusion of lidar data and visual image data to obtain intermediate positioning data of the drone relative to the photovoltaic panel surface, and then performs mid-level fusion of intermediate positioning data with millimeter-wave radar data to obtain the final target positioning data. The flight control system adjusts the drone's flight altitude and attitude according to the target positioning data to maintain a constant working distance between the drone and the photovoltaic panel surface. At the same time, it adjusts the water pump output pressure and water spray flow rate of the cleaning module according to the actual distance between the drone and the photovoltaic panel surface to ensure consistent cleaning results.

[0044] This embodiment employs multi-source data fusion technology using lidar, visual sensors, and millimeter-wave radar, completely eliminating reliance on GNSS signals and effectively solving the positioning failure problem caused by GNSS signal blockage when the UAV operates close to building exteriors or photovoltaic panel arrays. Through a dynamic weight allocation mechanism and hierarchical fusion strategy, the advantages of each sensor are fully utilized, achieving complementary fusion of multi-sensor data and improving the accuracy and robustness of the positioning results. By deeply linking the positioning results with flight control and cleaning module control, stable flight and efficient cleaning of the UAV are ensured in complex operating environments, significantly improving the safety and efficiency of cleaning operations.

[0045] Based on any of the above embodiments, in Embodiment 2 of this application, acquiring lidar data, visual image data, and millimeter-wave radar data with the same timestamp includes: S11: Acquire raw point cloud data from LiDAR, raw visual image data, and raw echo data from millimeter-wave radar at the same timestamp.

[0046] In this embodiment, the same timestamp means that the system clock time deviation corresponding to the data collected by the three sensors does not exceed 0.2 milliseconds, and the time base of all data frames is completely synchronized with the main clock of the flight control system.

[0047] As an optional implementation, the flight control system simultaneously sends edge-triggered signals to three sensors at a fixed frequency of 50 Hz via hardware GPIO pins. Each sensor pre-stores its own inherent trigger delay parameters: 12 microseconds for the LiDAR, 85 microseconds for the vision sensor, and 3 microseconds for the millimeter-wave radar. After receiving the trigger signal, the sensor waits for the corresponding inherent delay time before initiating data acquisition, using the system clock value at the completion of acquisition as the timestamp of the data frame. After receiving the data frames from the three sensors, the flight control system matches the data frames according to the timestamps, grouping three data frames with a time difference of less than 0.2 milliseconds into a set of synchronized data. This implementation solves the implicit synchronization error problem caused by differences in the hardware response speeds of different sensors, improving the time synchronization accuracy of multi-sensor data from 1 millisecond to 0.2 milliseconds.

[0048] S12, perform statistical filtering on the raw point cloud data of the lidar, calculate the average distance between each point and its preset number of neighboring points, remove discrete noise points whose distance is greater than the mean plus a preset multiple of the standard deviation, extract the planar features and edge features of the cleaned target surface, and obtain the lidar data.

[0049] In this embodiment, statistical filtering refers to the process of identifying and removing isolated noise points that have significant spatial distance differences from the surrounding point clouds by analyzing the spatial distribution characteristics of point cloud data.

[0050] As an optional implementation, the flight control system sets the number of nearest neighbor points to 20 and the standard deviation factor to 3. For each lidar point cloud data point, the flight control system calculates the average distance to its 20 closest spatial points, and then calculates the global mean and global standard deviation of the average distances of all points. The flight control system removes all points whose average distance is greater than the global mean plus 3 times the global standard deviation. The flight control system uses a random sampling consensus algorithm to extract planar features from the surface of the target to be cleaned, setting the maximum number of iterations to 100 and the planar fitting error threshold to 0.05 meters. The flight control system performs edge detection on the extracted planar point cloud, calculates the angle between the normal vectors of adjacent point clouds, and marks points with an angle greater than 15 degrees as edge feature points. This implementation solves the problem of false point clouds generated by fine dust particles adhering to the photovoltaic panel surface being misjudged as valid targets.

[0051] S13, Gaussian denoising processing is performed on the original visual image data to extract SIFT or SURF feature points in the image, and invalid feature points in areas where the gray-level variance is lower than a preset reflectivity threshold are removed to obtain the visual image data.

[0052] In this embodiment, Gaussian denoising refers to the process of smoothing high-frequency noise in an image while preserving image edge information by performing convolution operations on the image using a Gaussian kernel function.

[0053] As an optional implementation, the flight control system uses a 5×5 Gaussian kernel to convolve the original visual image, with the standard deviation of the Gaussian kernel set to 1.5. The flight control system extracts SIFT feature points from the processed image, setting a feature point response threshold of 0.04. The flight control system divides the image into 16×16 pixel sub-regions and calculates the gray-level variance of all pixels within each sub-region. The flight control system sets a preset reflection threshold of 15, removing all SIFT feature points from sub-regions with a gray-level variance less than 15. The flight control system then performs descriptor calculations on the remaining effective feature points, generating a 128-dimensional feature description vector. This implementation solves the image blurring problem caused by the thin water film formed on the lens surface during the cleaning process, improving the retention rate of effective feature points.

[0054] S14, perform constant false alarm rate (CFAR) filtering on the raw echo data of the millimeter-wave radar, and adjust the signal-to-noise ratio (SNR) threshold and radar cross section (RCS) threshold according to the water mist concentration of the current cleaning operation to obtain the millimeter-wave radar data.

[0055] In this embodiment, constant false alarm rate (CFAR) filtering refers to a signal processing method that maintains a constant false alarm probability under varying background noise and interference conditions by adaptively adjusting the detection threshold.

[0056] As an optional implementation, the flight control system employs a unit-averaged CFAR algorithm, setting the number of reference units to 16 and the number of protection units to 2. The flight control system calculates the current water mist concentration using the average reflectivity of the lidar point cloud. The water mist concentration equals 1 minus the ratio of the average reflectivity of the lidar point cloud to the average reflectivity under clean conditions. The flight control system dynamically adjusts the SNR and RCS thresholds based on the water mist concentration. The SNR threshold equals the base SNR threshold (12 dB) multiplied by 1 plus the square of the water mist concentration; the RCS threshold equals the base RCS threshold - 20 dB / m² multiplied by 1 minus 0.6 times the water mist concentration. The flight control system discards all echo signals with both an SNR and RCS less than the adjusted SNR and RCS thresholds. This implementation solves the problem of excessively high false alarm rates or missed detections of real targets by millimeter-wave radar under different water mist concentrations, improving target detection accuracy within the water mist concentration range of 0 to 0.8.

[0057] For example, when a cleaning drone performs photovoltaic power station module cleaning operations, the flight control system simultaneously sends edge trigger signals to three sensors at a frequency of 50 Hz via hardware GPIO pins. The lidar starts point cloud acquisition after 12 microseconds, the vision sensor starts image acquisition after 85 microseconds, and the millimeter-wave radar starts echo acquisition after 3 microseconds. The three sensors send the system clock value at the time of acquisition completion as the timestamp of the data frame to the flight control system. The flight control system groups three data frames with a time difference of less than 0.2 milliseconds into a set of synchronized data. The flight control system performs statistical filtering on the raw lidar point cloud data, sets the number of nearest neighbors to 20, the standard deviation factor to 3, removes discrete noise points, and uses a random sampling consensus algorithm to extract the planar and edge features of the photovoltaic panel surface. The flight control system performs 5×5 Gaussian kernel convolution denoising on the raw vision image data, extracts SIFT feature points, and removes invalid feature points in reflective areas with a gray-level variance of less than 15. The flight control system performs cell-averaged CFAR filtering on the raw millimeter-wave radar echo data. Based on the average reflectivity of the lidar point cloud, the current water mist concentration is calculated to be 0.3. The SNR threshold is adjusted to 13.08 dB, and the RCS threshold is adjusted to -23.6 dB / m². Echo signals that do not meet the threshold requirements are removed. The flight control system ultimately obtains the pre-processed lidar data, visual image data, and millimeter-wave radar data.

[0058] This embodiment achieves high-precision time synchronization of multi-sensor data through hardware triggering combined with inherent delay calibration, eliminating implicit errors caused by differences in the hardware response speeds of different sensors. Parameter-optimized statistical filtering and random sampling consensus algorithms effectively remove pseudo-point clouds caused by fine dust on the photovoltaic panel surface, improving the accuracy of planar feature extraction. An adaptive reflective region removal algorithm solves the problem of losing effective feature points caused by lens water film and strong surface reflection. A water mist concentration detection algorithm based on lidar reflectivity and a dynamic threshold adjustment algorithm achieves stable target detection for millimeter-wave radar under different water mist concentrations. This embodiment comprehensively improves the preprocessing quality of raw data from multiple sensors, providing a reliable data foundation for subsequent multi-sensor fusion positioning.

[0059] Based on any of the above embodiments, in Embodiment 3 of this application, referring to Figure 2 Based on the operating status and environmental data of the lidar, the vision sensor, and the millimeter-wave radar, weighting coefficients are determined for the lidar data, the visual image data, and the millimeter-wave radar data, including: S21, obtain the point cloud density and point cloud plane fitting degree of the lidar, obtain the number of effective feature points and feature point matching success rate of the vision sensor, and obtain the echo signal strength and target detection confidence of the millimeter-wave radar as the working state.

[0060] In this embodiment, point cloud density refers to the number of effective LiDAR point clouds corresponding to a unit area of ​​the target surface to be cleaned. Point cloud plane fitting degree refers to the average distance deviation between the LiDAR point cloud data and the extracted plane of the target surface to be cleaned. Feature point matching success rate refers to the proportion of the number of visual feature points in the current frame that successfully match the visual feature points in the previous frame to the total number of effective feature points in the current frame. Target detection confidence refers to the probability value that the target detected by the millimeter-wave radar is a real target surface to be cleaned.

[0061] As an optional implementation, the flight control system counts the total number of point clouds located on the plane of the target surface in the lidar data, divides this number by the projected area of ​​the target surface within the lidar's field of view to obtain the point cloud density. The flight control system calculates the vertical distance from each point cloud in the lidar data to the fitted plane, and takes the arithmetic mean of all vertical distances to obtain the point cloud plane fitting degree. The flight control system counts the total number of effective feature points in the visual image data after removing reflective areas, divides the number of feature points successfully matched between the current frame and the previous frame by the total number of effective feature points to obtain the feature point matching success rate. The flight control system extracts the echo peak intensity of each detected target from the millimeter-wave radar data as the echo signal intensity, and extracts the probability value of the millimeter-wave radar target detection output as the target detection confidence level. This implementation comprehensively reflects the actual sensing capabilities of each sensor through quantified working state parameters, solving the problem of inaccurate weight allocation caused by traditional qualitative assessment of sensor states.

[0062] S22, acquire the light intensity, air humidity, water mist concentration, and reflectivity of the target surface being cleaned in the current working environment as the environmental data.

[0063] In this embodiment, the reflectivity of the target surface being cleaned refers to the ratio of the light intensity reflected from the target surface to the total light intensity incident on the surface.

[0064] As an optional implementation, the flight control system calculates the average grayscale value of all pixels in the original visual image data, multiplies this average grayscale value by a preset illumination conversion coefficient of 0.39, and obtains the illumination intensity of the current working environment. The flight control system directly collects the air humidity of the current working environment through the UAV's built-in temperature and humidity sensors. The flight control system calculates the average reflectance of the lidar point cloud, subtracts the current average reflectance from the standard reflectance under clean conditions, and then divides by the standard reflectance to obtain the current water mist concentration. The flight control system divides the visual image data into multiple 32×32 pixel sub-regions, calculates the difference between the maximum and minimum grayscale values ​​in each sub-region, takes the average of all sub-region differences, and divides this average by 255 to obtain the reflectivity of the target surface. This implementation indirectly calculates environmental data using existing sensor data, eliminating the need for additional dedicated environmental detection hardware, and simultaneously solves the technical problems of high cost and slow response when directly measuring water mist concentration and surface reflectivity.

[0065] S23, determine the weighting coefficients of the lidar data, the visual image data, and the millimeter-wave radar data based on the operating status and the environmental data.

[0066] In this embodiment, the weighting coefficient refers to the proportion of each sensor data in the subsequent fusion calculation process, and the sum of all weighting coefficients is 1.

[0067] As an optional implementation, the flight control system normalizes each operating status parameter and sums the normalized parameter values ​​to obtain the operating status score of each sensor. Based on the influence of environmental data on different sensors, the flight control system calculates the environmental correction coefficient for each sensor. The flight control system multiplies the operating status score of each sensor by the corresponding environmental correction coefficient to obtain the comprehensive effective score of each sensor. The flight control system divides the comprehensive effective score of each sensor by the sum of the comprehensive effective scores of all sensors to obtain the weight coefficient corresponding to each sensor. When the comprehensive effective score of any sensor is lower than a preset failure threshold of 0.1, the flight control system sets the weight coefficient of that sensor to 0 and recalculates the weight coefficients of the other two sensors. This implementation achieves dynamic adaptive adjustment of the weight coefficients through dual weighted calculation of operating status and environmental data, solving the technical problem that single-factor weight allocation cannot adapt to complex and changing cleaning environments.

[0068] For example, when a cleaning drone is performing glass curtain wall cleaning operations, the flight control system calculates the following: the lidar point cloud density is 65 points per square meter, the point cloud plane fitting degree is 0.04 meters, the number of effective feature points of the visual sensor is 45, the feature point matching success rate is 35%, the millimeter-wave radar echo signal intensity is -55 dBmW, and the target detection confidence level is 0.95. The flight control system calculates the current operating environment's illumination intensity as 8000 lux, air humidity as 70%, water mist concentration as 0.25, and the glass curtain wall surface reflectivity as 0.7. After normalizing the various operating parameters, the flight control system calculates the lidar operating status score as 0.82, the visual sensor operating status score as 0.38, and the millimeter-wave radar operating status score as 0.91. Based on the environmental data, the flight control system calculates the lidar environment correction coefficient as 0.9, the visual sensor environment correction coefficient as 0.2, and the millimeter-wave radar environment correction coefficient as 0.95. The flight control system calculated the overall effective score for the lidar to be 0.738, the overall effective score for the vision sensor to be 0.076, and the overall effective score for the millimeter-wave radar to be 0.8645. The flight control system ultimately determined the weighting coefficients for the lidar to be 0.44, the vision sensor to be 0.045, and the millimeter-wave radar to be 0.515.

[0069] This embodiment comprehensively evaluates the actual sensing performance of each sensor through quantified multi-dimensional working state parameters, and indirectly calculates key environmental data using existing sensor data without additional hardware costs. Through a dual weighted calculation mechanism of working state and environmental data, precise dynamic adjustment of weight coefficients is achieved, accurately reflecting the reliability of sensor data under different environments. This embodiment effectively solves the technical problem that traditional fixed-weight or single-factor weight allocation methods cannot adapt to complex cleaning environments, significantly improving the accuracy and robustness of multi-sensor fusion positioning.

[0070] Based on any of the above embodiments, in Embodiment 4 of this application, determining the weighting coefficients of the lidar data, the visual image data, and the millimeter-wave radar data according to the working state and the environmental data includes: S231, Determine the working status score of each sensor based on the working status.

[0071] In this embodiment, the working status score refers to a numerical value that quantitatively reflects the sensing ability and data quality of each sensor. The score ranges from 0 to 1, and the larger the value, the better the sensor's working status and the higher the data quality.

[0072] As an optional implementation, the flight control system normalizes the operating status parameters of the lidar, vision sensor, and millimeter-wave radar respectively. For lidar, the normalized point cloud density value is equal to the current point cloud density divided by the maximum point cloud density of 120 points per square meter, and the normalized point cloud plane fit degree value is equal to 1 minus the current point cloud plane fit degree divided by the maximum allowable fitting error of 0.2 meters. The flight control system multiplies the normalized point cloud density value by 0.6 and adds it to the normalized point cloud plane fit degree value multiplied by 0.4 to obtain the lidar operating status score. For the vision sensor, the normalized effective feature point number value is equal to the current effective feature point number divided by the maximum effective feature point number of 200, and the feature point matching success rate is directly used as the normalized matching success rate value. The flight control system multiplies the normalized effective feature point number value by 0.4 and adds it to the normalized feature point matching success rate value multiplied by 0.6 to obtain the vision sensor operating status score. For millimeter-wave radar, the normalized value of the echo signal intensity is equal to 1 minus the absolute value of the current echo signal intensity divided by the absolute value of the maximum signal intensity (100 dB / mW). The target detection confidence score is directly used as the normalized confidence score. The flight control system multiplies the normalized echo signal intensity value by 0.3 and adds it to the normalized target detection confidence score by 0.7 to obtain the millimeter-wave radar operating status score. This implementation method reflects the degree of influence of different operating status parameters on sensor performance through differentiated weight allocation, solving the technical problem that the score caused by traditional average weighting cannot accurately reflect the actual operating status of the sensor.

[0073] S232, Determine the environmental correction coefficients for each sensor based on the environmental data.

[0074] In this embodiment, the environmental correction coefficient is a numerical value that quantifies the degree of influence of external environmental factors on the sensing performance of each sensor. The coefficient ranges from 0 to 1. The larger the value, the smaller the influence of the environment on the sensor and the higher the data reliability.

[0075] As an optional implementation, the flight control system calculates the impact factors of various environmental data on different sensors. For lidar, the water mist concentration impact factor is equal to 1 minus water mist concentration multiplied by 0.8, and the air humidity impact factor is equal to 1 minus air humidity multiplied by 0.3. The flight control system multiplies the water mist concentration impact factor and the air humidity impact factor to obtain the lidar environmental correction coefficient. For visual sensors, the illumination intensity impact factor is calculated using a piecewise function: when the illumination intensity is less than 1000 lux, the illumination intensity impact factor is equal to the illumination intensity divided by 1000 lux; when the illumination intensity is greater than 100000 lux, the illumination intensity impact factor is equal to 1 minus the illumination intensity minus 100000 lux divided by 100000 lux; and when the illumination intensity is between 1000 lux and 100000 lux, the illumination intensity impact factor is equal to 1. The surface reflectivity impact factor is equal to 1 minus the surface reflectivity multiplied by 0.9. The flight control system multiplies the light intensity influence factor with the surface reflectivity influence factor to obtain the visual sensor environmental correction coefficient. For millimeter-wave radar, the water mist concentration influence factor equals 1 minus water mist concentration multiplied by 0.2, and the air humidity influence factor equals 1 minus air humidity multiplied by 0.1. The flight control system multiplies the water mist concentration influence factor with the air humidity influence factor to obtain the millimeter-wave radar environmental correction coefficient. This implementation method accurately quantifies the degree of influence of different environmental factors on each sensor through piecewise functions and differentiated influence coefficients, solving the technical problem that traditional single correction coefficients cannot adapt to complex environmental changes.

[0076] S233: Multiply the working status scores of each sensor by the corresponding environmental correction coefficients to obtain the comprehensive scores of the LiDAR, the vision sensor, and the millimeter-wave radar.

[0077] In this embodiment, the comprehensive score refers to the comprehensive performance evaluation value that takes into account both the sensor's own working status and the influence of the external environment. The score ranges from 0 to 1, and the larger the value, the higher the credibility of the sensor data in the fusion calculation.

[0078] As an optional implementation, the flight control system multiplies the lidar operating status score by the lidar environmental correction coefficient to obtain a comprehensive lidar score. Similarly, the flight control system multiplies the visual sensor operating status score by the visual sensor environmental correction coefficient to obtain a comprehensive visual sensor score. Finally, the flight control system multiplies the millimeter-wave radar operating status score by the millimeter-wave radar environmental correction coefficient to obtain a comprehensive millimeter-wave radar score. When the comprehensive score of any sensor is less than 0.05, the flight control system sets the comprehensive score of that sensor to 0, marking it as a failure. This implementation achieves a superimposed evaluation of operating status and environmental influence through multiplication operations, accurately reflecting the actual reliability of sensor data under different operating conditions.

[0079] S234, Based on the comprehensive score of the lidar, the comprehensive score of the vision sensor, and the comprehensive score of the millimeter-wave radar, determine the weight coefficient of the lidar, the weight coefficient of the vision sensor, and the weight coefficient of the millimeter-wave radar.

[0080] In this embodiment, the weighting coefficient refers to the proportion of each sensor data in the subsequent fusion calculation process, and the sum of all weighting coefficients is 1.

[0081] As an optional implementation, the flight control system calculates the sum of the comprehensive scores of all sensors. When the sum of the comprehensive scores is greater than 0, the weight coefficient of the lidar is equal to the lidar comprehensive score divided by the sum of the comprehensive scores, the weight coefficient of the vision sensor is equal to the vision sensor comprehensive score divided by the sum of the comprehensive scores, and the weight coefficient of the millimeter-wave radar is equal to the millimeter-wave radar comprehensive score divided by the sum of the comprehensive scores. When the sum of the comprehensive scores is equal to 0, the flight control system triggers an emergency return-to-home command. When only one sensor has a comprehensive score greater than 0, the weight coefficient of that sensor is set to 1, and the weight coefficients of the other two sensors are set to 0. This implementation achieves automatic allocation of weight coefficients through normalization processing, and can dynamically adjust the fusion strategy according to the actual comprehensive performance of each sensor, ensuring the accuracy and reliability of the fusion results.

[0082] For example, when a cleaning drone is performing cleaning operations on photovoltaic power station modules, the flight control system calculates the following: the lidar point cloud density is 80 points per square meter, the point cloud plane fitting degree is 0.03 meters, the number of effective feature points of the visual sensor is 120, the feature point matching success rate is 0.85, the millimeter-wave radar echo signal intensity is -45 dBmW, and the target detection confidence level is 0.92. The flight control system calculates the lidar operating status score as 0.94, the visual sensor operating status score as 0.75, and the millimeter-wave radar operating status score as 0.941. The flight control system also calculates the current operating environment's illumination intensity as 50,000 lux, air humidity as 60%, water mist concentration as 0.2, and photovoltaic panel surface reflectivity as 0.3. The flight control system calculates the lidar environment correction coefficient as 0.66, the visual sensor environment correction coefficient as 0.73, and the millimeter-wave radar environment correction coefficient as 0.9. The flight control system calculated the comprehensive score for the lidar to be 0.6204, the comprehensive score for the vision sensor to be 0.5475, and the comprehensive score for the millimeter-wave radar to be 0.8469. The total comprehensive score calculated by the flight control system was 2.0148. Finally, the weighting coefficients for lidar, vision sensor, and millimeter-wave radar were determined to be 0.308, 0.272, and 0.42, respectively.

[0083] This embodiment accurately quantifies the impact of different parameters on sensor performance through a differentiated weight allocation-based working state scoring calculation method. By employing a piecewise function and a differentiated influence coefficient calculation method for environmental correction, it precisely reflects the differences in the impact of various environmental factors on each sensor. Through the superposition evaluation of working state and environmental influence, and a normalized weight allocation mechanism, it achieves adaptive dynamic adjustment of the weight coefficients. This embodiment effectively solves the technical problem that traditional weight allocation methods cannot accurately reflect the actual performance of sensors and environmental impacts, significantly improving the accuracy and robustness of multi-sensor fusion positioning, and enabling it to adapt to various complex and changing cleaning operation environments.

[0084] Optionally, after determining the environmental correction coefficients for each sensor, the drone's flight attitude parameters, historical positioning residual data, cleaning module operating status parameters, and prior parameters of the cleaning target structure are obtained, and the multi-dimensional additional correction coefficients for each sensor are calculated.

[0085] In this embodiment, the multi-dimensional additional correction coefficient refers to a value that quantifies the impact of the unique operational factors of the cleaning drone on the reliability of the data from each sensor, and the coefficient ranges from 0 to 1.

[0086] As an optional implementation, the flight control system calculates additional correction coefficients in four dimensions. First, the flight attitude correction coefficient: the flight control system extracts the pitch and roll angle data of the UAV. When the absolute values ​​of both pitch and roll angles are less than 5 degrees, the flight attitude correction coefficient for each sensor is 1; when the angle is between 5 and 15 degrees, the correction coefficient decreases linearly to 0.5 as the angle increases; when the angle is greater than 15 degrees, the correction coefficient remains at 0.2. This correction coefficient addresses the implicit problem of sensor field of view deviating from the target being cleaned due to attitude tilt during UAV surface-flying. Second, the historical positioning residual correction coefficient: the flight control system calculates the mean square error between the individual positioning results and the fused positioning results of each sensor in the previous 5 frames. After normalizing the mean square error, subtracting this value from 1 yields the historical positioning residual correction coefficient. This correction coefficient addresses the problem of the cumulative error of a single sensor on a continuous trajectory not being identified in a timely manner. Third, the feedforward correction coefficient for the cleaning module: The flight control system acquires the water pump output pressure and the number of nozzles opened in the cleaning module to predict the water mist concentration change within the next 100 milliseconds. When the water pump pressure increases by 10%, the feedforward correction coefficients of the lidar and visual sensors decrease by 8% and 12%, respectively, while the feedforward correction coefficient of the millimeter-wave radar remains at 1. This correction coefficient enables feedforward prediction of water mist interference, solving the 100 to 200 millisecond lag problem of traditional feedback adjustment. Fourth, the target structure prior correction coefficient: Based on pre-stored cleaning target structure parameters, when the UAV flies to the position of the photovoltaic panel gap or glass curtain wall grid line, the flight control system increases the target structure correction coefficient of the lidar to 1.2, and simultaneously increases the correction coefficient of the millimeter-wave radar to 1.1. This correction coefficient solves the problem of temporary degradation of sensor data quality caused by target structure changes being mistakenly judged as sensor failure.

[0087] S234. The working status scores of each sensor are multiplied by the corresponding environmental correction coefficients and multi-dimensional additional correction coefficients in sequence to obtain the comprehensive score of LiDAR, the comprehensive score of visual sensor, and the comprehensive score of millimeter-wave radar.

[0088] In this embodiment, the comprehensive score refers to a full-dimensional performance evaluation value that takes into account the sensor's own state, external environment, flight attitude, historical performance, the impact of cleaning operations, and target structural characteristics. The score range is from 0 to 1.2.

[0089] As an optional implementation, the flight control system calculates the comprehensive score of each sensor according to the following formula: the comprehensive score of the lidar equals the lidar operating status score multiplied by the lidar environmental correction coefficient multiplied by the lidar flight attitude correction coefficient multiplied by the lidar historical positioning residual correction coefficient multiplied by the lidar cleaning module feedforward correction coefficient multiplied by the lidar target structure prior correction coefficient. The comprehensive scores of the visual sensors and the millimeter-wave radar use the same calculation logic. When the comprehensive score of any sensor is less than 0.05, the flight control system sets the comprehensive score of that sensor to 0, marking the sensor as in a failed state. This implementation achieves a comprehensive and accurate evaluation of the reliability of sensor data through the cascaded operation of multi-dimensional coefficients.

[0090] S235, Based on the comprehensive score of the lidar, the comprehensive score of the vision sensor, and the comprehensive score of the millimeter-wave radar, determine the weight coefficient of the lidar, the weight coefficient of the vision sensor, and the weight coefficient of the millimeter-wave radar.

[0091] In this embodiment, the weighting coefficient refers to the proportion of each sensor data in the subsequent fusion calculation process, and the sum of all weighting coefficients is 1.

[0092] As an optional implementation, the flight control system calculates the sum of the comprehensive scores of all sensors. When the sum of the comprehensive scores is greater than 0, the weight coefficient of the LiDAR is equal to the LiDAR comprehensive score divided by the sum of the comprehensive scores; the weight coefficient of the visual sensor is equal to the visual sensor comprehensive score divided by the sum of the comprehensive scores; and the weight coefficient of the millimeter-wave radar is equal to the millimeter-wave radar comprehensive score divided by the sum of the comprehensive scores. When the sum of the comprehensive scores is equal to 0, the flight control system triggers an emergency return-to-home command. When only one sensor has a comprehensive score greater than 0, the weight coefficient of that sensor is set to 1, and the weight coefficients of the other two sensors are set to 0. The flight control system also performs a smoothing process on the weight coefficients. The weight coefficient of the current frame is equal to 0.7 multiplied by the currently calculated initial weight coefficient plus 0.3 multiplied by the final weight coefficient of the previous frame, to avoid sudden changes in the weight coefficients that could cause jitter in the positioning results.

[0093] For example, when a cleaning drone is performing photovoltaic power station module cleaning operations, the flight control system calculates the working status score of the lidar as 0.94, the visual sensor as 0.75, and the millimeter-wave radar as 0.941. The flight control system also calculates the environmental correction coefficients for the lidar as 0.66, the visual sensor as 0.73, and the millimeter-wave radar as 0.9. At this time, the drone's pitch angle is 8 degrees, and the flight control system calculates the flight attitude correction coefficients for each sensor as 0.85. The positioning data from the previous 5 frames shows the historical positioning residual correction coefficients for the lidar as 0.92, the visual sensor as 0.88, and the millimeter-wave radar as 0.95. The cleaning module's water pump output pressure is 120% of the standard pressure. The flight control system predicts that the water mist concentration will rise to 0.3 in the next 100 milliseconds; therefore, the feedforward correction coefficients for the lidar are 0.84, the visual sensor as 0.76, and the millimeter-wave radar as 1. The drone is flying over the gap between two photovoltaic panels. Based on pre-stored structural parameters, the flight control system sets the target structure correction coefficient for the lidar to 1.2 and the millimeter-wave radar to 1.1. The flight control system ultimately calculates the comprehensive score as follows: lidar: 0.94 × 0.66 × 0.85 × 0.92 × 0.84 × 1.2 = 0.458; visual sensor: 0.75 × 0.73 × 0.85 × 0.88 × 0.76 × 1 = 0.279; millimeter-wave radar: 0.941 × 0.9 × 0.85 × 0.95 × 1 × 1.1 = 0.753. The total comprehensive score calculated by the flight control system is 1.49. Therefore, the final weighting coefficients are determined as follows: lidar 0.307, visual sensor 0.187, and millimeter-wave radar 0.506.

[0094] This embodiment overcomes the limitations of existing technologies that rely solely on sensor operating status and environmental data to determine weights. It introduces for the first time correction parameters specific to cleaning drones, encompassing four dimensions: flight attitude, historical positioning residuals, cleaning module operating status, and target structure prior knowledge. The flight attitude correction coefficient addresses the implicit problem of field-of-view deviation during surface-hugging flight; the historical positioning residual correction coefficient enables dynamic identification of sensor cumulative errors; the cleaning module feedforward correction coefficient reduces the adjustment lag due to water mist interference; and the target structure prior knowledge correction coefficient avoids sensor misjudgments caused by changes in the target structure. This embodiment constructs a full-dimensional dynamic weight adjustment model, significantly improving the accuracy and timeliness of weight allocation, enabling the multi-sensor fusion positioning system to better adapt to the complex and dynamic environment of cleaning operations.

[0095] Based on any of the above embodiments, in Embodiment 5 of this application, the weighting coefficients include lidar weighting coefficients, visual sensor weighting coefficients, and millimeter-wave radar weighting coefficients. Based on the weighting coefficients, the lidar data and the visual image data are first fused at a low level to obtain intermediate positioning data, and then the intermediate positioning data and the millimeter-wave radar data are fused at a mid-level to obtain target positioning data, including: S31, based on the extrinsic parameter matrix between the lidar and the visual sensor, as well as the lidar weight coefficient and the visual sensor weight coefficient, the lidar data and the visual image data are fused to obtain the intermediate positioning data.

[0096] In this embodiment, the extrinsic parameter matrix refers to the transformation matrix that describes the relative position and attitude relationship between the lidar coordinate system and the vision sensor coordinate system, and includes two parts: rotation matrix and translation vector.

[0097] As an optional implementation, the flight control system loads a pre-calibrated extrinsic parameter matrix between the LiDAR and the visual sensor, while simultaneously acquiring real-time UAV fuselage temperature data and correcting the extrinsic parameter matrix online based on temperature changes. The flight control system transforms all point cloud coordinates from the LiDAR data to the visual sensor coordinate system via the extrinsic parameter matrix, achieving spatial alignment between the two sensor data sets. The flight control system constructs an extended Kalman filter state equation, using the UAV's three-dimensional position, three-dimensional velocity, and three-dimensional attitude angles relative to the target surface as the system state vector, and the throttle, pitch, roll, and yaw control variables of the UAV's power system as control inputs. The flight control system uses the target surface distance and plane normal vector extracted from the LiDAR data as the first observation vector, and the lateral offset and rotation angle calculated from the feature point matching results extracted from the visual image data as the second observation vector. Based on the LiDAR weighting coefficients and the visual sensor weighting coefficients, the flight control system adjusts the observation noise covariance matrices of the first and second observation vectors, respectively; a larger weighting coefficient corresponds to a smaller observation noise covariance. The flight control system inputs the adjusted observation vectors into an extended Kalman filter for prediction and updating, outputting intermediate positioning data containing the UAV's relative position and attitude information with respect to the target surface being cleaned. This implementation addresses the implicit problem of extrinsic parameter matrix drift caused by temperature changes, which affects fusion accuracy and reduces the positioning error of laser-vision fusion.

[0098] S32, the intermediate positioning data is converted to the millimeter-wave radar coordinate system and timestamped with the target distance and angle information extracted from the millimeter-wave radar data.

[0099] In this embodiment, timestamp alignment refers to the process of unifying intermediate positioning data and millimeter-wave radar data collected at different times under the same time reference.

[0100] As an optional implementation, the flight control system loads a pre-calibrated extrinsic parameter matrix between the airframe coordinate system and the millimeter-wave radar coordinate system, transforming the intermediate positioning data from the airframe coordinate system to the millimeter-wave radar coordinate system. The flight control system records the timestamps corresponding to the intermediate positioning data and the millimeter-wave radar data. When the millimeter-wave radar data arrives, it searches for the two frames of intermediate positioning data with the closest timestamps. The flight control system uses a linear interpolation algorithm to calculate the interpolated intermediate positioning data that is perfectly aligned with the millimeter-wave radar data timestamps, based on the time difference and numerical difference between the two frames of intermediate positioning data. The flight control system sets a maximum allowable time difference of 5 milliseconds. When the time difference between two frames of intermediate positioning data exceeds 5 milliseconds, the nearest neighbor interpolation algorithm is used for alignment. This implementation solves the time alignment error problem caused by different sensor data acquisition frequencies, further improving the time synchronization accuracy from 0.2 milliseconds to 0.05 milliseconds.

[0101] S33, Initial positioning data is obtained by weighting and fusion-aligning the intermediate positioning data with the millimeter-wave radar data based on the millimeter-wave radar weight coefficients.

[0102] In this embodiment, weighted fusion refers to the process of assigning different weights to each data point based on its credibility, and then combining information from multiple data sources to obtain a more accurate result.

[0103] As an optional implementation, the flight control system extracts the vertical and horizontal distances of the UAV relative to the target surface from the aligned intermediate positioning data, and extracts the target distance and horizontal angle information from the millimeter-wave radar data. The flight control system converts the horizontal angle information from the millimeter-wave radar into a horizontal distance offset, using the formula: horizontal distance offset equals target distance multiplied by the sine of the horizontal angle. The flight control system performs weighted fusion of the intermediate positioning data and the millimeter-wave radar data according to the millimeter-wave radar weighting coefficient. The fused vertical distance value equals the vertical distance of the intermediate positioning data multiplied by 1 minus the millimeter-wave radar weighting coefficient plus the target distance of the millimeter-wave radar multiplied by the millimeter-wave radar weighting coefficient. The fused horizontal distance value equals the horizontal distance of the intermediate positioning data multiplied by 1 minus the millimeter-wave radar weighting coefficient plus the horizontal distance offset of the millimeter-wave radar multiplied by the millimeter-wave radar weighting coefficient. The flight control system simultaneously performs the same weighted fusion process on the attitude angle data to obtain initial positioning data containing fused position and attitude information. This implementation solves the problem of large horizontal positioning errors caused by the low angular resolution of millimeter-wave radar, improving horizontal positioning accuracy.

[0104] S34, determine the target positioning data based on the jump magnitude of the initial positioning data.

[0105] In this embodiment, the jump amplitude refers to the absolute value of the difference between the initial positioning data of the current frame and the target positioning data of the previous frame.

[0106] As an optional implementation, the flight control system calculates the position jump amplitude and attitude jump amplitude between the initial positioning data of the current frame and the target positioning data of the previous frame. The flight control system sets a position jump threshold of 0.1 meters and an attitude jump threshold of 3 degrees. When the position jump amplitude is less than the position jump threshold and the attitude jump amplitude is less than the attitude jump threshold, the initial positioning data is directly used as the target positioning data. When the jump amplitude exceeds the corresponding threshold, the flight control system uses a sliding window filtering algorithm to smooth the initial positioning data, with the window size set to 5 frames. The flight control system calculates the weighted average of the initial positioning data of all frames within the window, with the weight coefficient decreasing over time: 0.4 for the current frame, 0.3 for the previous frame, 0.2 for the two previous frames, and 0.1 for the three previous frames. The flight control system uses the weighted average as the target positioning data. This implementation solves the problem of positioning result jumps caused by instantaneous anomalies in sensor data and effectively suppresses altitude and attitude jitter during UAV flight.

[0107] For example, when a cleaning drone performs glass curtain wall cleaning operations, the flight control system loads pre-calibrated extrinsic parameter matrices for the LiDAR and visual sensors, and performs online corrections to these matrices based on the current fuselage temperature of 25 degrees Celsius. The flight control system converts the LiDAR point cloud data to the visual sensor coordinate system and constructs an extended Kalman filter state equation. Based on the LiDAR weighting coefficient of 0.307 and the visual sensor weighting coefficient of 0.187, the flight control system adjusts the observation noise covariance matrix, setting the LiDAR observation noise covariance to 0.0025 and the visual sensor observation noise covariance to 0.0064. The flight control system inputs the glass curtain wall surface distance and plane normal vector extracted by the LiDAR, along with the lateral offset and rotation angle extracted by the visual sensor, into the extended Kalman filter to obtain intermediate positioning data. The flight control system converts this intermediate positioning data to the millimeter-wave radar coordinate system, finds the two frames of intermediate positioning data with the closest timestamps, and uses a linear interpolation algorithm to calculate an interpolation result that is perfectly aligned with the millimeter-wave radar data timestamp. The flight control system uses a millimeter-wave radar weighting coefficient of 0.506 to perform weighted fusion of intermediate positioning data and millimeter-wave radar data to obtain initial positioning data. The flight control system calculates that the position jump amplitude between the current frame's initial positioning data and the previous frame's target positioning data is 0.08 meters, and the attitude jump amplitude is 2 degrees, both of which are less than the corresponding thresholds. Therefore, the initial positioning data is directly used as the target positioning data.

[0108] This embodiment solves the sensor spatial alignment error problem caused by temperature changes by employing an online correction technique for the extrinsic parameter matrix with temperature compensation. A linear interpolation timestamp alignment algorithm achieves high-precision time synchronization of data from sensors of different frequencies. A mid-level fusion strategy using angle transformation and weighted fusion effectively compensates for the low angular resolution of millimeter-wave radar. An adaptive sliding window filtering algorithm suppresses instantaneous jumps in positioning results. This embodiment comprehensively improves the accuracy and stability of multi-sensor hierarchical fusion.

[0109] Based on any of the above embodiments, in Embodiment Six of this application, the intermediate positioning data is obtained by fusing the lidar data and the visual image data based on the extrinsic parameter matrix between the lidar and the visual sensor, the lidar weight coefficient, and the visual sensor weight coefficient, including: S311, based on the pre-calibrated extrinsic parameter matrix between the lidar and the vision sensor, unifies the pre-processed lidar data and visual image data into the UAV coordinate system. The extrinsic parameter matrix is ​​obtained through factory calibration or online calibration.

[0110] In this embodiment, the extrinsic parameter matrix refers to a 4×4 homogeneous transformation matrix composed of a 3×3 rotation matrix R and a 3×1 translation vector t, which is used to realize the coordinate transformation between the lidar coordinate system and the vision sensor coordinate system.

[0111] As an optional implementation, the flight control system uses Zhang's calibration method to complete the initial extrinsic parameter calibration of the LiDAR and vision sensor during the factory stage. During calibration, a 1200mm × 900mm checkerboard calibration board is used to collect 20 sets of calibration images and corresponding point cloud data at different angles and distances. The initial extrinsic parameter matrix is ​​solved using the least squares method. During UAV operation, the flight control system performs online calibration every 10 seconds, extracting planar features of the target surface as a common calibration object. The angular deviation between the LiDAR plane normal vector and the vision image plane normal vector is calculated, and the extrinsic parameter matrix is ​​iteratively corrected using the gradient descent method. The number of iterations is set to 50, and the convergence threshold is set to 0.001 radians. Simultaneously, the flight control system collects fuselage temperature data and compensates for the translation vector of the extrinsic parameter matrix based on temperature changes, with a compensation coefficient of 0.002 mm per degree Celsius. This implementation solves the technical problem of extrinsic parameter matrix drift caused by temperature changes and mechanical vibrations during UAV operation, reducing the spatial alignment error from 0.02 meters to within 0.005 meters.

[0112] S312, construct the extended Kalman filter state equation, using the three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle of the cleaning UAV as the system state vector, and the power system input of the cleaning UAV as the control input.

[0113] In this embodiment, the system state vector refers to a 9-dimensional vector describing the motion state of the UAV, and the control input refers to a 4-dimensional vector describing the output of the UAV's power system.

[0114] As an optional implementation, the system state vector X constructed by the flight control system is [x,y,z,vx,vy,vz,roll,pitch,yaw]^T, where x, y, and z are the three-dimensional positions of the UAV relative to the target surface being cleaned, vx,vy, and vz are the three-dimensional velocities, and roll, pitch, and yaw are the three-dimensional attitude angles. The control input U constructed by the flight control system is [throttle,elevator,aileron,rudder]^T, where throttle is the throttle control, elevator is the pitch control,aileron is the roll control, and rudder is the yaw control. The flight control system establishes a discrete-time state transition equation X. k =F k- 1X k-1 +B k-1 U k-1 +W k-1 F k-1 B is a 9×9 state transition matrix. k-1 The control input matrix is ​​9×4, W k-1 This represents the process noise vector. The flight control system sets the diagonal elements of the process noise covariance matrix Q to [0.0001,0.0001,0.0001,0.001,0.001,0.001,0.0001,0.0001,0.0001,0.0001]. This implementation comprehensively describes the UAV's motion state through a complete 9-dimensional state vector and 4-dimensional control input, laying the foundation for high-precision state estimation.

[0115] S313, the distance to the cleaned target surface and the plane normal vector extracted from the preprocessed lidar data are used as the first observation vector, and the lateral offset and rotation angle calculated from the feature point matching results extracted from the preprocessed visual image data are used as the second observation vector.

[0116] In this embodiment, the first observation vector refers to the 3D observation vector obtained from lidar data, and the second observation vector refers to the 2D observation vector obtained from visual image data.

[0117] As an optional implementation, the flight control system uses a random sampling consensus algorithm to extract the plane equation ax + by + cz + d = 0 from the lidar data to clean the target surface. It calculates the vertical distance from the origin of the UAV coordinate system to the plane as the observation distance, and the plane normal vector is [a, b, c]^T, constructing the first observation vector Z1 = [d, a, b]^T. The flight control system matches the visual feature points of the current frame with those of the previous frame, setting the matching threshold to 0.7, and uses a random sampling consensus algorithm to remove mismatched points. The flight control system calculates the pixel coordinate difference between the matched feature point pairs, and combines this with the camera intrinsic parameter matrix to calculate the lateral offset and rotation angle of the UAV relative to the target surface, constructing the second observation vector Z2 = [dx, dθ]^T. The flight control system sets the observation noise covariance matrix R1 of the first observation vector to diag([0.0025, 0.0001, 0.0001]), and the observation noise covariance matrix R2 of the second observation vector to diag([0.01, 0.0004]). This implementation method achieves comprehensive observation of the UAV's position and attitude by combining planar features and feature point matching.

[0118] S314, the first observation vector and the second observation vector are input into the extended Kalman filter for updating. The high-precision distance information of the lidar data is used to correct the cumulative error of the visual image data caused by feature point drift, and the texture features of the visual image data are used to correct the lateral positioning deviation of the lidar data. The output includes intermediate positioning data containing the relative position and attitude angle of the cleaning drone relative to the surface of the cleaning target.

[0119] In this embodiment, the extended Kalman filter update process includes two stages: a prediction step and an update step.

[0120] As an optional implementation, the flight control system first performs a prediction step, calculating the prior state estimate Xk and the prior covariance matrix Pk according to the state transition equation. The flight control system then performs an update step, calculating the Jacobian matrices H1 and H2 for the first and second observation vectors, respectively. Based on the lidar weight coefficients and the visual sensor weight coefficients, the flight control system dynamically adjusts the observation noise covariance matrices R1 and R2, using the formulas R1'=R1 / WL and R2'=R2 / WV, where WL is the lidar weight coefficient and WV is the visual sensor weight coefficient. The flight control system calculates the Kalman gains K1 and K2, updating the prior state estimate using the first and second observation vectors, respectively, to obtain the posterior state estimate Xk. The flight control system updates the covariance matrix Pk and outputs the posterior state estimate as intermediate positioning data. This implementation, by dynamically adjusting the observation noise covariance matrix, achieves optimal fusion under different weight coefficients, solving the technical problem that a single fixed observation noise cannot adapt to changes in sensor data quality.

[0121] For example, when a cleaning drone performs photovoltaic power station module cleaning operations, the flight control system loads the factory-calibrated initial extrinsic parameter matrix and compensates the translation vector by 0.02 mm based on the current fuselage temperature of 30 degrees Celsius. The flight control system performs online calibration every 10 seconds, iteratively correcting the extrinsic parameter matrix using the planar features of the photovoltaic panel surface, converging to 0.0008 radians after 35 iterations. The flight control system constructs a 9-dimensional system state vector and 4-dimensional control input, establishes discrete-time state transition equations, and sets the process noise covariance matrix. The flight control system extracts the planar equation of the photovoltaic panel surface from the lidar data, calculates the observation distance as 1.5 meters, and the planar normal vector as [0,0,1]^T, constructing the first observation vector. The flight control system extracts 120 effective feature points from the visual image data, matches them with feature points from the previous frame, obtains 95 correctly matched point pairs, calculates the lateral offset as 0.02 meters, and the rotation angle as 0.5 degrees, constructing the second observation vector. The flight control system executes an extended Kalman filter prediction step to obtain a priori state estimates. Based on the lidar weighting coefficient of 0.307 and the visual sensor weighting coefficient of 0.187, the flight control system adjusts the observation noise covariance matrix. The flight control system calculates the Kalman gain and updates the priori state estimates using the two observation vectors to obtain posterior state estimates as intermediate positioning data.

[0122] This embodiment solves the spatial alignment error problem caused by extrinsic parameter matrix drift by combining factory calibration with online calibration technology with temperature compensation. A complete 9-dimensional state vector and 4-dimensional control input comprehensively and accurately describe the UAV's motion state. Multi-dimensional observation of the UAV's position and attitude is achieved through a combination of planar features and feature point matching. An extended Kalman filter algorithm that dynamically adjusts the observation noise covariance matrix fully leverages the complementary advantages of LiDAR and visual sensors, effectively correcting visual cumulative errors and LiDAR lateral positioning deviations, significantly improving the stability and safety of the UAV's surface-following flight.

[0123] Based on any of the above embodiments, in Embodiment Seven of this application, determining the target positioning data according to the jump amplitude of the initial positioning data includes: S341, the initial positioning data is smoothed and filtered. When the amplitude of the initial positioning data jump exceeds a preset threshold, the weight coefficient of the target sensor is reduced and the fusion calculation is performed again.

[0124] In this embodiment, the jump amplitude refers to the absolute value of the difference between the corresponding parameters of the initial positioning data of the current frame and the target positioning data of the previous frame. The target sensor refers to the sensor whose individual positioning result jumps in the same direction and amplitude as the initial positioning data.

[0125] As an optional implementation, the flight control system employs an exponentially weighted moving average filtering algorithm to smooth the initial positioning data. The smoothing formula is: the smoothed value of the current frame equals 0.7 multiplied by the initial positioning data of the current frame plus 0.3 multiplied by the target positioning data of the previous frame. The flight control system calculates the jump magnitudes of vertical distance, horizontal offset, pitch angle, roll angle, and yaw angle. The flight control system sets the vertical distance jump threshold to 0.08 meters, the horizontal offset jump threshold to 0.1 meters, and the attitude angle jump threshold to 2 degrees. When the jump magnitude of any parameter exceeds the corresponding preset threshold, the flight control system calculates the residuals between the individual positioning results of the lidar, visual sensor, and millimeter-wave radar and the target positioning data of the previous frame. The flight control system identifies the sensor with the largest residual and whose jump direction is consistent with the initial positioning data jump direction as the target sensor. The flight control system multiplies the weight coefficient of the target sensor by an attenuation coefficient of 0.5, while keeping the weight coefficient ratios of the other two sensors unchanged, recalculates the weight coefficients of all sensors, and performs fusion calculation. When the same target sensor is detected in two consecutive frames, causing a jump, the flight control system further multiplies the weight coefficient of that sensor by 0.3. This implementation solves the technical problem that traditional smoothing filters can only suppress jump behavior but cannot trace the root cause of the jump, eliminating positioning fluctuations caused by instantaneous sensor anomalies from the source of data fusion.

[0126] S342, output the target positioning data after smoothing and filtering. The target positioning data includes at least the vertical distance, horizontal offset, pitch angle, roll angle and yaw angle of the cleaning drone relative to the surface of the cleaning target.

[0127] In this embodiment, target positioning data refers to the final positioning result after fusion calculation and smoothing processing, which serves as the sole basis for the flight control system to execute flight control and cleanup control.

[0128] As an optional implementation, the flight control system encapsulates the target positioning data according to a preset data format, adding a timestamp and data checksum. The flight control system outputs vertical distance parameters to the altitude control module, horizontal offset parameters to the trajectory tracking module, and pitch, roll, and yaw angle parameters to the attitude control module. Simultaneously, the flight control system stores the complete target positioning data in the flight log memory at a frequency of 50 Hz. The flight control system sets the data output delay to no more than 2 milliseconds to ensure that the flight control system can respond to changes in positioning data in real time. This implementation achieves accurate distribution and reliable storage of positioning data, ensuring the real-time performance and traceability of flight control and cleanup control.

[0129] For example, when a cleaning drone is performing photovoltaic power station module cleaning operations, the flight control system calculates that the vertical distance of the initial positioning data in the current frame is 1.62 meters, while the vertical distance of the target positioning data in the previous frame is 1.5 meters, a jump of 0.12 meters, exceeding the preset threshold of 0.08 meters. The flight control system calculates the residuals between the individual positioning results of the three sensors and the target positioning data in the previous frame. The vertical distance residuals for the lidar individual positioning results are 0.13 meters, for the visual sensor 0.03 meters, and for the millimeter-wave radar 0.05 meters. The flight control system identifies the lidar as the target sensor, reduces its weight coefficient from 0.307 to 0.1535 by multiplying it by 0.5, while keeping the weight ratios of the visual sensor and the millimeter-wave radar unchanged. The recalculated weight coefficients are 0.18 for the lidar, 0.22 for the visual sensor, and 0.6 for the millimeter-wave radar. The flight control system then re-performs the fusion calculation using the new weight coefficients, obtaining a corrected initial positioning data with a vertical distance of 1.53 meters. The flight control system uses an exponentially weighted moving average filtering algorithm to smooth the corrected initial positioning data, resulting in final target positioning data with a vertical distance of 1.52 meters, a horizontal offset of 0.01 meters, a pitch angle of 0.3 degrees, a roll angle of 0.2 degrees, and a yaw angle of 0.1 degrees. The flight control system encapsulates the target positioning data and outputs it to the altitude control module, trajectory tracking module, and attitude control module, while simultaneously storing it in the flight log storage.

[0130] This embodiment achieves smoothing of positioning data through an exponentially weighted moving average filtering algorithm, effectively suppressing positioning fluctuations caused by instantaneous sensor noise. Target sensor identification technology using residual analysis can accurately locate the sensor causing the jump, and eliminate the root cause of the jump by dynamically attenuating its weight coefficient. This implementation solves the continuous jump problem that traditional smoothing filters cannot address, significantly improving the stability of UAV flight and the uniformity of cleaning operations.

[0131] As an optional implementation, when performing hierarchical fusion calculations, the flight control system first loads the extrinsic parameter matrices of the LiDAR and visual sensors, which have undergone temperature compensation and online calibration correction. The preprocessed LiDAR point cloud data is then converted to the visual sensor coordinate system, and finally unified to the UAV coordinate system using a body coordinate system transformation matrix. The flight control system constructs a 9-dimensional extended Kalman filter state vector containing three-dimensional position, three-dimensional velocity, and three-dimensional attitude angles. Using the throttle, pitch, roll, and yaw control variables of the power system as 4-dimensional control inputs, a discrete-time state transition equation is established. The flight control system divides the target surface to be cleaned into 16×16 local grid regions. For each grid region, the LiDAR point cloud density and the number of effective visual feature points are calculated to obtain the local confidence coefficient for each region. The local confidence coefficient is equal to the normalized value of the point cloud density multiplied by 0.6 plus the normalized value of the number of feature points multiplied by 0.4. The flight control system dynamically adjusts the observation noise covariance matrix of the corresponding LiDAR and visual sensors based on the local confidence coefficient of each region. The adjustment formulas are R1_ij'=R1_ij / (WL×C_ij) and R2_ij'=R2_ij / (WV×C_ij), where R1_ij and R2_ij are the initial observation noise covariance of the i-th row and j-th column grid region, WL and WV are global weight coefficients, and C_ij is the local confidence coefficient of that region. The flight control system extracts the surface distance and plane normal vector of each grid region from the LiDAR data to construct the first observation vector, and extracts the feature point matching results of each grid region from the visual image data to calculate the lateral offset and rotation angle to construct the second observation vector. The two observation vectors are then input into an extended Kalman filter for prediction and updating. The flight control system calculates the fusion residual in real time. When the fusion residual exceeds a preset residual threshold of 0.03 meters, the diagonal elements of the process noise covariance matrix are linearly increased according to the residual ratio, with the increase factor being the residual value divided by 0.03 meters, thus achieving adaptive adjustment of the filter. The flight control system outputs intermediate positioning data that includes local positioning information for each region and global positioning information.

[0132] The flight control system converts the intermediate positioning data to the millimeter-wave radar coordinate system, and simultaneously acquires the UAV's current pitch and roll angle data. It then performs attitude compensation for the target distance from the millimeter-wave radar using the formula d_compensated = d_raw × cos(pitch) × cos(roll), where d_raw is the original measured distance from the millimeter-wave radar, pitch is the pitch angle, and roll is the roll angle. The flight control system utilizes the angular velocity and acceleration data from the UAV's built-in IMU to compensate for the residual time synchronization error between the intermediate positioning data and the millimeter-wave radar data, predicting the UAV's position at the moment of millimeter-wave radar acquisition using the formula p_predicted = p_middle + v_middle × Δt + 0.5 × a_middle × Δt², where p_middle is the position of the intermediate positioning data, v_middle is the velocity of the intermediate positioning data, a_middle is the acceleration measured by the IMU, and Δt is the residual time synchronization error. The flight control system timestamps the attitude-compensated millimeter-wave radar range data with the intermediate positioning data after position prediction. It then performs weighted fusion based on the global weight coefficient of the millimeter-wave radar and the local confidence coefficients of each region to obtain the initial positioning data. The flight control system uses a third-order exponential weighted moving average filtering algorithm to smooth the initial positioning data. The smoothing formula is current_smoothed = 0.6 × current_raw + 0.3 × previous_smoothed + 0.1 × second_previous_smoothed. The flight control system calculates the jump amplitude of vertical range, horizontal offset, and each attitude angle. When the jump amplitude of any parameter exceeds the corresponding preset threshold, residual analysis identifies the target sensor and specific grid region causing the jump. The local confidence coefficient of the target sensor in that region is multiplied by an attenuation coefficient of 0.3, and its global weight coefficient is multiplied by an attenuation coefficient of 0.6. While maintaining the weight ratios of other sensors, the fusion calculation is re-executed. When the same sensor detects a jump in the same area for three consecutive frames, the flight control system sets the local confidence coefficient of that sensor in that area to 0, relying entirely on data from other sensors for positioning calculations in that area. The flight control system ultimately outputs target positioning data after smoothing and filtering, including the UAV's vertical distance relative to the target surface being cleaned, horizontal offset, pitch angle, roll angle, yaw angle, and local positioning confidence information for each area.

[0133] For example, when a cleaning drone is performing a photovoltaic power station module cleaning operation, it flies to an area with heavily soiled surfaces and adjacent to the gaps between photovoltaic panels. The flight control system divides this area into a 16×16 grid, calculating a local confidence coefficient of 0.2 for the soiled area, 0.3 for the photovoltaic panel gap area, and 0.9 for other cleaned areas. Based on the local confidence coefficients, the flight control system dynamically adjusts the observation noise covariance of each area, increasing the observation noise covariance of the lidar and the visual sensor by a factor of 5 for the soiled area. The flight control system constructs an extended Kalman filter state equation, inputs the dynamic system control quantity to perform a prediction step, and then updates the data using the adjusted observation vector to obtain intermediate positioning data. At this point, the drone's pitch angle is 3 degrees and its roll angle is 2 degrees. The flight control system performs attitude compensation on the original millimeter-wave radar measurement distance of 1.52 meters, obtaining a compensated distance of 1.517 meters. The flight control system uses IMU data to compensate for a 0.003-second residual error in time synchronization, predicting the intermediate positioning data distance at the millimeter-wave radar acquisition time to be 1.505 meters. The flight control system performs weighted fusion based on the millimeter-wave radar weighting coefficient of 0.506, obtaining an initial positioning data distance of 1.511 meters. The flight control system uses a third-order exponential weighted moving average filtering algorithm for smoothing, obtaining a smoothed distance of 1.508 meters. The flight control system detects a horizontal offset jump of 0.12 meters, exceeding the preset threshold of 0.1 meters. Residual analysis identifies this jump as being caused by the visual sensor in the gap region of the photovoltaic panel. The flight control system sets the local confidence coefficient of the visual sensor in this gap region to 0, while simultaneously reducing its global weighting coefficient from 0.187 to 0.112, and re-performs the fusion calculation, obtaining a corrected horizontal offset of 0.02 meters. The flight control system ultimately outputs target positioning data: vertical distance of 1.508 meters, horizontal offset of 0.02 meters, pitch angle of 0.3 degrees, roll angle of 0.2 degrees, and yaw angle of 0.1 degrees.

[0134] This embodiment overcomes the limitation of existing layered fusion technologies that only perform global fusion, and for the first time introduces a local region reliability assessment mechanism to solve the problem of local data anomalies caused by local stains and gaps on the surface of the cleaning target. Through attitude compensation formulas and IMU-assisted time synchronization residual error compensation technology, the impact of UAV maneuver attitude changes and sensor time asynchrony on fusion accuracy is eliminated. Through a closed-loop feedback mechanism of the fusion residual, adaptive adjustment of the extended Kalman filter parameters is achieved. Through a joint attenuation mechanism of local reliability coefficients and global weight coefficients, positioning jumps caused by sensor anomalies are eliminated at both the global and local levels. This embodiment maintains stable positioning performance even in photovoltaic panel areas with heavy stains and continuous gaps, significantly improving the operational capability of cleaning UAVs under complex conditions.

[0135] Based on any of the above embodiments, in Embodiment 8 of this application, adjusting the flight attitude of the cleaning drone and the control parameters of the cleaning module according to the target positioning data includes: S41, extract the vertical distance, horizontal offset, pitch angle, roll angle and yaw angle of the UAV relative to the target surface to be cleaned from the target positioning data.

[0136] In this embodiment, the local positioning reliability information refers to the positioning accuracy evaluation value of each grid area on the surface of the target to be cleaned, which is included in the target positioning data. The value ranges from 0 to 1, and the larger the value, the more reliable the positioning result of that area.

[0137] As an optional implementation, the flight control system extracts the vertical distance, horizontal offset, pitch angle, roll angle, and yaw angle of the UAV relative to the target surface from the target positioning data, and simultaneously extracts the local positioning reliability information for each 16×16 grid region. The flight control system verifies the validity of the extracted parameters; when a parameter value exceeds a preset reasonable range, it replaces it with a valid parameter value from the previous frame. The flight control system stores the extracted parameters in a control parameter buffer with a buffer depth of 10 frames for subsequent feedforward control and filtering processing. This implementation solves the problem of sudden changes in control commands caused by instantaneous anomalies in positioning data, ensuring the continuity and stability of the control process.

[0138] S42, compare the vertical distance with a preset benchmark working distance threshold, and generate a height control command.

[0139] In this embodiment, the baseline operating distance threshold refers to the optimal operating distance between the drone and the target surface to be cleaned, and the safe distance threshold refers to the minimum safe distance between the drone and the target surface to be cleaned.

[0140] As an optional implementation, the flight control system sets a baseline operating distance threshold of 1.5 meters and a safety distance threshold of 0.8 meters. The flight control system calculates the deviation between the vertical distance and the baseline operating distance threshold. When the deviation is greater than 0, an altitude ascent control command is generated, with an ascent rate equal to the deviation multiplied by 0.5 meters per second. When the deviation is less than 0 and the vertical distance is greater than the safety distance threshold, an altitude descent control command is generated, with a descent rate equal to the absolute value of the deviation multiplied by 0.3 meters per second. When the vertical distance is less than the safety distance threshold, an emergency ascent control command is generated, with an ascent rate set to 2 meters per second, and an obstacle avoidance warning signal is issued. The flight control system uses an incremental PID control algorithm to generate the final altitude control command, with a proportional coefficient set to 1.2, an integral coefficient set to 0.5, and a derivative coefficient set to 0.1. The flight control system also introduces a feedforward control term, with the feedforward control quantity equal to the UAV's vertical speed multiplied by 0.1 seconds, to adjust the control quantity in advance. This implementation solves the 100-millisecond lag problem of traditional feedback control, improving altitude control accuracy to ±2 centimeters.

[0141] S43, based on the pitch angle, roll angle and yaw angle, generate attitude control commands to control the UAV to adjust its attitude so that the UAV fuselage remains parallel to the surface of the target being cleaned.

[0142] In this embodiment, the target attitude angle refers to the attitude angle when the UAV fuselage is parallel to the surface to be cleaned, which is calculated from the plane normal vector of the surface to be cleaned.

[0143] As an optional implementation, the flight control system calculates the target attitude angle based on the plane normal vector of the target surface being cleaned. The flight control system calculates the deviation between the current attitude angle and the target attitude angle. For pitch and roll angle deviations, a PID control algorithm is used to generate control commands, with a proportional gain set to 2.0, an integral gain set to 0.8, and a derivative gain set to 0.3; for yaw angle deviations, the proportional gain is set to 1.5, the integral gain to 0.6, and the derivative gain to 0.2. The flight control system sets a dead zone of 0.2 degrees for the attitude angle deviation; no control commands are output when the deviation is less than the dead zone. When the attitude angle deviation exceeds 5 degrees, the flight control system automatically increases the proportional gain to 3.0. The flight control system limits the generated attitude control commands, ensuring the maximum control quantity does not exceed 80% of the total control quantity. This implementation solves the flight jitter problem caused by frequent small adjustments of the UAV, achieving an attitude control accuracy of ±0.2 degrees.

[0144] S44, Based on the horizontal offset, generate a horizontal movement control command so that the cleaning drone moves horizontally along the surface of the cleaning target and remains on the preset cleaning trajectory.

[0145] In this embodiment, the forward-looking distance refers to the distance between the current position of the UAV and the target tracking point in the pure tracking algorithm.

[0146] As an optional implementation, the flight control system decomposes the horizontal offset into lateral and longitudinal offsets. The system employs a pure tracking algorithm for trajectory tracking, setting the standard forward look-ahead distance (FLAD) to 0.5 meters. Based on the UAV's current position and the preset trajectory, the system calculates the target tracking point and generates a horizontal movement speed command. When the horizontal offset exceeds 0.2 meters, the system increases the horizontal movement speed to 1.5 times the standard speed; when the horizontal offset is less than 0.05 meters, it maintains the standard horizontal movement speed of 0.5 meters per second. The system adjusts the FLAD based on local positioning reliability information; when the local reliability coefficient is below 0.5, the FLAD is increased to 0.8 meters. The system smooths the horizontal movement speed, limiting the acceleration to 1 meter per second squared. This implementation solves the trajectory fluctuation problem caused by abnormal local positioning data, controlling the trajectory tracking error within ±3 centimeters.

[0147] S45, adjust the water pump output pressure of the cleaning module according to the vertical distance.

[0148] In this embodiment, the standard water pump output pressure refers to the optimal cleaning pressure of the drone at the reference operating distance.

[0149] As an optional implementation, the flight control system sets the standard water pump output pressure to 8 MPa, with an upper limit of 12 MPa and a lower limit of 4 MPa. The flight control system establishes a linear mapping relationship between vertical distance and water pump output pressure. When the vertical distance is within ±0.1 meters of the reference working distance, the water pump output pressure remains at the standard pressure; when the vertical distance is more than 0.1 meters greater than the reference working distance, the water pump output pressure increases by 0.8 MPa for every 0.1 meter increase; when the vertical distance is less than 0.1 meters less than the reference working distance, the water pump output pressure decreases by 0.6 MPa for every 0.1 meter decrease. The flight control system also incorporates pressure feedforward control, predicting vertical distance changes 0.2 seconds in advance based on altitude control commands and adjusting the water pump output pressure accordingly. This implementation solves the 150-millisecond lag problem in water pump pressure adjustment, ensuring consistent cleaning pressure at different distances.

[0150] S46, Adjust the nozzle angle and water flow rate of the cleaning module according to the target surface type in the target positioning data.

[0151] In this embodiment, the target surface types for cleaning include three types: photovoltaic panels, glass curtain walls, and stone walls, which are determined by the task information imported before the drone operation.

[0152] As an optional implementation, the flight control system calls upon the corresponding cleaning parameter configuration based on the surface type of the target surface in the target positioning data. When the target is a photovoltaic panel, the nozzle angle is adjusted to be perpendicular to the photovoltaic panel surface, and the water flow rate is set to 15 liters per minute; when the target is a glass curtain wall, the nozzle angle is adjusted to a 45-degree angle to the glass curtain wall surface, and the water flow rate is set to 18 liters per minute; when the target is a stone wall, the nozzle angle is adjusted to a 30-degree angle to the wall surface, and the water flow rate is set to 22.5 liters per minute. The flight control system also adjusts the water flow rate based on local positioning reliability information; when the local reliability coefficient is below 0.5, the water flow rate is increased by 10%. The flight control system controls the nozzle angle adjustment motor to perform the angle adjustment, with an adjustment accuracy of ±1 degree. This implementation solves the problem of large differences in cleaning effects for different surface materials, achieving targeted and optimal cleaning.

[0153] S47, when the vision sensor detects a heavily soiled area on the surface of the target to be cleaned, the location of the heavily soiled area is determined based on the target positioning data, and a deceleration control command is generated.

[0154] In this embodiment, heavily stained areas refer to areas with stubborn stains that are difficult to remove, which are identified by the grayscale and texture features of the visual image.

[0155] As an optional implementation, the flight control system identifies heavily soiled areas by analyzing the grayscale histogram and texture features of the visual image. When the average grayscale value of pixels within a region is lower than 60% of the average grayscale value of the cleaned region and the texture complexity exceeds a preset threshold, it is determined to be a heavily soiled area. The flight control system determines the location and extent of the heavily soiled area based on target positioning data and calculates the time it takes for the drone to reach that area. The flight control system generates a deceleration control command 0.5 seconds in advance, reducing the drone's horizontal movement speed to 40% of the standard speed. When the drone enters the heavily soiled area, the flight control system simultaneously increases the water pump output pressure by 20% and the water spray flow rate by 30%. After the drone leaves the heavily soiled area, the flight control system automatically restores the standard parameters. The flight control system records the location information of the heavily soiled area and generates a secondary cleaning task. This implementation solves the problem of incomplete cleaning of heavily soiled areas and improves the cleaning pass rate.

[0156] For example, when a cleaning drone is performing photovoltaic power station module cleaning operations, the flight control system extracts the following data from the target positioning data: vertical distance 1.55 meters, horizontal offset 0.08 meters, pitch angle 0.4 degrees, roll angle 0.3 degrees, yaw angle 0.2 degrees, and local positioning reliability coefficients for each area are all greater than 0.8. The flight control system calculates the deviation between the vertical distance and the reference working distance to be 0.05 meters, generates an altitude descent control command, sets the descent speed to 0.015 meters per second, and calculates the final altitude control value using an incremental PID algorithm. The flight control system calculates the attitude angle deviation, generates attitude control commands, and adjusts the drone's attitude to keep the fuselage parallel to the photovoltaic panel surface. The flight control system generates lateral control commands based on the horizontal offset and uses a pure tracking algorithm to track the preset cleaning trajectory. Based on the vertical distance of 1.55 meters, the flight control system adjusts the water pump output pressure to 8.4 MPa. Since the cleaning target is the photovoltaic panel, the flight control system adjusts the nozzle angle to be perpendicular to the photovoltaic panel surface and sets the water flow rate to 15 liters per minute. At this point, the visual sensor detects a heavily soiled area 0.3 meters ahead. The flight control system determines the location of this area based on target positioning data and generates a deceleration control command 0.5 seconds in advance, reducing the horizontal movement speed from 0.5 meters per second to 0.2 meters per second. When the drone enters the heavily soiled area, the flight control system increases the water pump output pressure to 10.08 MPa and the water spray flow rate to 19.5 liters per minute. After the drone leaves the heavily soiled area, the flight control system automatically restores standard parameters, continues the cleaning operation, and records the location of the area to generate a secondary cleaning task.

[0157] This embodiment effectively solves the lag problem of traditional feedback control by employing an incremental PID control combined with feedforward control for altitude control, achieving centimeter-level altitude control accuracy. A dead-zone control and adaptive PID parameter adjustment attitude control strategy suppresses high-frequency jitter in the UAV, ensuring stable flight attitude. A trajectory tracking strategy combining a pure tracking algorithm with local reliability information achieves high-precision trajectory maintenance, avoiding the impact of local data anomalies on the trajectory. A water pump pressure adjustment strategy combining linear mapping and feedforward control ensures consistent cleaning pressure at different operating distances. Differentiated nozzle angles and water flow rates achieve optimal cleaning results for different surface materials. Heavy stain area identification and adaptive cleaning parameter adjustment strategies significantly improve the cleaning effect on stubborn stains. This embodiment achieves deep linkage between positioning data, flight control, and cleaning control, comprehensively improving the operational accuracy, efficiency, and reliability of the cleaning UAV.

[0158] As an optional implementation, the flight control system extracts the vertical distance, horizontal offset, pitch angle, roll angle, and yaw angle of the UAV relative to the target surface from the target positioning data. Simultaneously, it extracts local positioning reliability information for each region as an auxiliary basis for adjusting control parameters. The flight control system compares the vertical distance with preset baseline operating distance thresholds and safety distance thresholds. The baseline operating distance threshold is set to 1.5 meters, and the safety distance threshold is set to 0.8 meters. When the vertical distance is greater than the baseline operating distance threshold, an altitude ascent control command is generated, with the ascent rate proportional to the distance deviation at a ratio of 0.5 meters per second. When the vertical distance is less than the baseline operating distance threshold but greater than the safety distance threshold, an altitude descent control command is generated, with a descent rate of 0.3 meters per second. When the vertical distance is less than the safety distance threshold, an emergency ascent control command is generated, with an ascent rate set to 2 meters per second, and an obstacle avoidance warning signal is issued. The flight control system uses an incremental PID control algorithm to generate altitude control commands, with the proportional coefficient set to 1.2, the integral coefficient set to 0.5, and the derivative coefficient set to 0.1. At the same time, a feedforward control term is introduced to predict the position change in the next 0.1 seconds based on the vertical speed of the UAV and adjust the control quantity in advance, thus solving the lag problem of traditional feedback control.

[0159] The flight control system generates attitude control commands based on the extracted pitch, roll, and yaw angles, controlling the UAV to adjust its attitude to keep the fuselage parallel to the target surface being cleaned. The flight control system calculates the deviation between the current attitude angle and the target attitude angle, which is the attitude angle corresponding to the plane normal vector of the target surface. For pitch and roll angle deviations, a PID control algorithm is used to generate control commands, with a proportional gain set to 2.0, an integral gain set to 0.8, and a derivative gain set to 0.3; for yaw angle deviations, the proportional gain is set to 1.5, the integral gain to 0.6, and the derivative gain to 0.2. The flight control system sets a dead zone of 0.2 degrees for attitude angle deviations; when the deviation is less than the dead zone, no control commands are output to avoid flight jitter caused by frequent small adjustments by the UAV. When the attitude angle deviation exceeds 5 degrees, the flight control system automatically increases the proportional gain to 3.0 to accelerate the attitude adjustment speed and ensure that the UAV remains parallel to the target surface during maneuvers.

[0160] The flight control system generates horizontal movement control commands based on the horizontal offset, enabling the UAV to move horizontally along the target surface and maintain a preset cleaning trajectory. The system decomposes the horizontal offset into lateral and longitudinal offsets, generating corresponding lateral and longitudinal control commands respectively. A pure tracking algorithm is used for trajectory tracking, with a forward look-ahead distance of 0.5 meters. Based on the UAV's current position and the preset trajectory, the system calculates the target tracking point and generates horizontal movement speed commands. When the horizontal offset exceeds 0.2 meters, the system automatically increases the horizontal movement speed to 1.5 times the standard speed to quickly correct trajectory deviations; when the horizontal offset is less than 0.05 meters, it maintains the standard horizontal movement speed. Simultaneously, the system adjusts trajectory tracking accuracy based on local positioning reliability information. When the local reliability coefficient is below 0.5, the forward look-ahead distance is appropriately increased to 0.8 meters to reduce trajectory tracking sensitivity and avoid trajectory fluctuations caused by abnormal local positioning data.

[0161] The flight control system adjusts the water pump output pressure of the cleaning module based on the vertical distance, establishing a linear mapping relationship between vertical distance and water pump output pressure. When the vertical distance is within ±0.1 meters of the reference working distance, the water pump output pressure remains at the standard pressure of 8 MPa. When the vertical distance is more than 0.1 meters greater than the reference working distance, the water pump output pressure increases by 0.8 MPa for every 0.1 meter increase. When the vertical distance is less than 0.1 meters less than the reference working distance, the water pump output pressure decreases by 0.6 MPa for every 0.1 meter decrease. The flight control system sets an upper limit of 12 MPa and a lower limit of 4 MPa for the water pump output pressure to prevent damage to the cleaning equipment from excessive pressure or impact on the cleaning effect from excessively low pressure. The flight control system also incorporates pressure feedforward control, predicting the vertical distance change in the next 0.2 seconds based on altitude control commands and adjusting the water pump output pressure in advance, thus solving the 150-millisecond lag problem in water pump pressure adjustment.

[0162] The flight control system adjusts the nozzle angle and water flow rate of the cleaning module based on the target surface type in the target positioning data. When the target is a photovoltaic panel, the nozzle angle is adjusted to be perpendicular to the photovoltaic panel surface, and the water flow rate is set to the standard flow rate of 15 liters per minute. When the target is a glass curtain wall, the nozzle angle is adjusted to a 45-degree angle to the glass curtain wall surface, and the water flow rate is set to 1.2 times the standard flow rate, i.e., 18 liters per minute. When the target is a stone wall, the nozzle angle is adjusted to a 30-degree angle to the wall surface, and the water flow rate is set to 1.5 times the standard flow rate, i.e., 22.5 liters per minute. The flight control system also adjusts the water flow rate based on local positioning reliability information. When the local reliability coefficient is below 0.5, the water flow rate is increased by 10% to ensure good cleaning results even in areas with low positioning accuracy.

[0163] When the visual sensor detects a heavily soiled area on the target surface, the flight control system determines the location and extent of the heavily soiled area based on target positioning data and generates a deceleration control command. The flight control system identifies heavily soiled areas by analyzing the grayscale histogram and texture features of the visual image. An area is considered heavily soiled when the average grayscale value of pixels within the area is lower than 60% of the average grayscale value of the clean area and the texture complexity exceeds a threshold. The flight control system calculates the time it takes for the drone to reach the heavily soiled area and generates a deceleration control command 0.5 seconds in advance, reducing the drone's horizontal movement speed to 40% of the standard speed. When the drone enters the heavily soiled area, the flight control system simultaneously increases the water pump output pressure by 20% and the water spray flow rate by 30%, extending the cleaning time. After the drone leaves the heavily soiled area, the flight control system automatically restores the standard horizontal movement speed, water pump output pressure, and water spray flow rate. The flight control system also records the location information of the heavily soiled area, generates a secondary cleaning task, and performs a second cleaning of the area after the initial cleaning to ensure the stains are completely removed.

[0164] For example, when a cleaning drone is performing a photovoltaic power station module cleaning operation, the flight control system extracts the following data from the target positioning data: vertical distance 1.55 meters, horizontal offset 0.08 meters, pitch angle 0.4 degrees, roll angle 0.3 degrees, and yaw angle 0.2 degrees. The flight control system compares the vertical distance with a baseline operating distance threshold of 1.5 meters and generates an altitude descent control command at a descent speed of 0.15 meters per second. The flight control system calculates the attitude angle deviation, generates attitude control commands, and adjusts the drone's attitude to keep the fuselage parallel to the photovoltaic panel surface. Based on the horizontal offset, the flight control system generates lateral control commands to correct the trajectory deviation. Based on the vertical distance of 1.55 meters, the flight control system adjusts the water pump output pressure to 8.4 MPa. Since the cleaning target is the photovoltaic panel, the flight control system adjusts the nozzle angle to be perpendicular to the photovoltaic panel surface and sets the water flow rate to 15 liters per minute. At this point, the visual sensor detects a heavily soiled area 0.3 meters ahead. The flight control system determines the location of this area based on target positioning data and generates a deceleration control command 0.5 seconds in advance, reducing the horizontal movement speed from 0.5 meters per second to 0.2 meters per second. When the drone enters the heavily soiled area, the flight control system increases the water pump output pressure to 10.08 MPa and the water spray flow rate to 19.5 liters per minute. Once the drone leaves the heavily soiled area, the flight control system automatically restores standard parameters and continues the cleaning operation.

[0165] This embodiment addresses the lag issue of traditional feedback control by employing an incremental PID control combined with feedforward control for altitude control, thereby improving altitude control accuracy. A dead-zone control and adaptive PID parameter adjustment attitude control strategy effectively suppresses UAV flight jitter. A trajectory tracking strategy combining a pure tracking algorithm with local reliability information achieves high-precision trajectory holding, with trajectory tracking errors controlled within ±3 cm. A water pump pressure adjustment strategy combining linear mapping and feedforward control solves the problem of water pump pressure adjustment lag, ensuring consistent cleaning pressure at different distances. Differentiated nozzle angles and water flow rates achieve optimal cleaning results for different cleaning target types. Heavy stain area identification and adaptive cleaning parameter adjustment strategies significantly improve the cleaning effect on heavily soiled areas. This embodiment achieves deep integration of positioning data with flight and cleaning control, comprehensively improving the operational accuracy and efficiency of the cleaning UAV.

[0166] Optionally, addressing the core pain point of GNSS signal obstruction and interference during wall-hugging operations by cleaning drones, a multi-source sensor combination of LiDAR, vision, and millimeter-wave radar is used to completely eliminate reliance on GNSS signals and achieve stable positioning in wall-hugging scenarios. This approach complements the shortcomings of individual sensors, improving environmental adaptability: It compensates for the degradation issues of LiDAR (such as data drift caused by dust and water mist): data calibration is performed through environmental texture recognition by the vision sensor and distance detection by the millimeter-wave radar to maintain positioning accuracy; it addresses the low accuracy of millimeter-wave radar: the high-resolution distance measurement of LiDAR and pixel-level feature matching of the vision sensor optimize the coarse positioning results of the millimeter-wave radar; and it overcomes the scene limitations of the vision sensor: in reflective glass (loss of visual features) and low-light environments (increased image noise), the active detection of LiDAR and the anti-interference characteristics of the millimeter-wave radar ensure uninterrupted positioning.

[0167] Dual improvement in positioning accuracy and stability: By fusing and optimizing data from three sensors through algorithms, the output is more accurate and robust than that of a single sensor, meeting the high-precision requirements of attitude control and distance maintenance when cleaning drones are operating close to walls.

[0168] Strong anti-interference capability: Multiple sensors independently collect data. Even if one sensor is affected by environmental interference (such as the lidar being blocked by sewage or the vision being reflected by strong light), the other sensors can still work normally. The continuous positioning is ensured through redundant algorithm calculation.

[0169] Hardware Architecture Design: LiDAR: Deployed at the front of the drone fuselage to acquire 3D point cloud data of the wall surface, providing high-precision distance (error ≤ 5cm) and contour information; Visual Sensor (HD Camera): Deployed coaxially with the LiDAR to collect image features of the wall surface (such as wall texture and stain boundaries) to assist in visual positioning and feature matching; Millimeter-wave Radar: Deployed at the very front of the drone fuselage for long-range (0.5 to 10m) obstacle detection and coarse distance measurement, resistant to rain, fog, dust, and low-light interference; Data Acquisition Module: Integrated into the drone flight control system, synchronously receiving raw data from the three sensors (timestamp aligned, synchronization error ≤ 1ms).

[0170] Hardware connectivity: The lidar, vision sensor, and millimeter-wave radar are connected to the main controller of the flight control system via serial / CAN interfaces. The main controller is linked with the UAV's attitude control module and power system via the CAN bus to realize the real-time conversion of positioning data into control commands.

[0171] Step 1: Sensor data preprocessing. LiDAR data: Remove noise points from the point cloud (using statistical filtering algorithms), extract planar and edge features of the wall surface; Visual data: Denoise and enhance the image (optimize using histogram equalization in low-light environments), extract SIFT / SURF feature points, and eliminate invalid features from reflective areas; Millimeter-wave radar data: The radar emits 24GHz electromagnetic waves, and the received echoes are filtered for noise and interference using CFA to select the correct target points, accurately outputting the distance from the radar to the wall and glass without penetrating the glass; Adjust SNR and RCS thresholds to avoid water mist interfering with normal ranging.

[0172] Step 2: Data Fusion Algorithm. A "layered fusion" strategy is adopted: Bottom layer fusion: Kalman filtering is used to fuse the position data of LiDAR and vision sensors, and the distance accuracy of LiDAR is used to correct the drift of visual positioning; Middle layer fusion: The bottom layer fusion result is complementarily verified with the distance data of millimeter-wave radar. When LiDAR / visual data fails (such as LiDAR degradation or visual reflection failure), the positioning mode dominated by millimeter-wave radar is automatically switched; Adaptive weight allocation: The working status of each sensor is evaluated in real time through the algorithm (such as the feature point matching success rate of visual sensors, the point cloud density of LiDAR, and the signal strength of millimeter-wave radar), and higher weights are allocated to effective sensors (such as 60% weight for clear visual features, 30% for LiDAR, and 10% for millimeter-wave radar; in reflective environments, the visual weight is reduced to 10%, and the millimeter-wave radar weight is increased to 50%).

[0173] Step 3: Positioning Result Output and Feedback. The fused positioning data (including the distance, attitude angle, and movement speed of the drone relative to the wall) is output to the flight control system in real time to control the drone to maintain a preset wall-hugging distance (e.g., 2 to 3 meters) and a stable attitude. When the data from a certain sensor is abnormal, the algorithm issues a warning in real time and automatically adjusts the fusion strategy to ensure that the positioning error is ≤10cm (meeting the accuracy requirements of the cleaning operation).

[0174] Operation Initiation Phase: After takeoff, as the UAV approaches the working wall (at a distance of 2 to 3 meters), the GNSS signal weakens, and the system automatically switches to multi-sensor fusion positioning mode. All three sensors activate simultaneously, collecting initial environmental data (LiDAR scans the wall's 3D contour, the visual sensor captures an image of the wall, and the millimeter-wave radar measures the initial contact distance with the wall). Positioning Calculation Phase: The LiDAR calculates the relative position and attitude of the UAV to the wall using point cloud data, outputting high-precision distance information. However, this is susceptible to degradation due to sewage and dust; insufficient light can also cause feature point loss, leading to positioning failure. The millimeter-wave radar is unaffected by ambient light and dust, continuously outputting coarse-precision distance data as positioning redundancy. The fusion algorithm receives data from the three sensors in real time, filters invalid information through preprocessing, and then performs hierarchical fusion and adaptive weight allocation to output accurate and stable positioning results.

[0175] Anomaly adaptation phase: When a sensor failure is detected (e.g., the visual sensor has no effective feature points on the reflective glass wall), the algorithm immediately reduces the weight of that sensor and increases the participation of other effective sensors (e.g., fusion of LiDAR and millimeter-wave radar); if the LiDAR degrades (point cloud density decreases), compensation is made through the texture features of the visual sensor and the distance data of the millimeter-wave radar to ensure that the positioning is not interrupted and to maintain the stability of the drone cleaning operation.

[0176] This application provides a multi-sensor fusion positioning and control device for a cleaning drone. The multi-sensor fusion positioning and control device for a cleaning drone includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the multi-sensor fusion positioning and control method for a cleaning drone as described in Embodiment 1 above.

[0177] The following is for reference. Figure 3The diagram illustrates a structural schematic of a multi-sensor fusion positioning and control device suitable for implementing the embodiments of this application for a cleaning drone. The multi-sensor fusion positioning and control device for the cleaning drone in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablets, and vehicle terminals, as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The multi-sensor fusion positioning and control device for cleaning drones shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0178] like Figure 3 As shown, the multi-sensor fusion positioning and control device for a cleaning drone may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the multi-sensor fusion positioning and control device for the cleaning drone. The processing unit 1001, the read-only memory 1002, and the RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the multi-sensor fusion positioning and control device for the cleaning drone to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a multi-sensor fusion positioning and control device for a cleaning drone with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0179] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0180] The multi-sensor fusion positioning and control device for cleaning drones provided in this application, employing the multi-sensor fusion positioning and control method for cleaning drones described in the above embodiments, can solve the technical problem that when a drone operates close to the exterior walls of buildings or photovoltaic panel arrays, the wall structure, photovoltaic panel supports, and components will cause severe obstruction and multipath reflection effects on satellite signals, leading to a sharp attenuation of GNSS signal strength. Compared with the prior art, the beneficial effects of the multi-sensor fusion positioning and control device for cleaning drones provided in this application are the same as those of the multi-sensor fusion positioning and control device for cleaning drones provided in the above embodiments, and other technical features in this multi-sensor fusion positioning and control device for cleaning drones are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0181] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0182] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0183] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the multi-sensor fusion positioning and control method for cleaning drones in the above embodiments.

[0184] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), or any suitable combination thereof.

[0185] The aforementioned computer-readable storage medium may be included in the multi-sensor fusion positioning and control device for the cleaning drone; or it may exist independently and not be assembled into the multi-sensor fusion positioning and control device for the cleaning drone.

[0186] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the multi-sensor fusion positioning and control device of the cleaning drone, the multi-sensor fusion positioning and control device of the cleaning drone causes the following: it acquires lidar data, visual image data, and millimeter-wave radar data at the same timestamp; it determines weighting coefficients for the lidar data, visual image data, and millimeter-wave radar data based on the operating status and environmental data of the lidar, the visual sensor, and the millimeter-wave radar; based on the weighting coefficients, it first performs low-level fusion of the lidar data and the visual image data to obtain intermediate positioning data, and then performs mid-level fusion of the intermediate positioning data and the millimeter-wave radar data to obtain target positioning data; and it adjusts the flight attitude of the cleaning drone and the control parameters of the cleaning module according to the target positioning data.

[0187] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0188] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation that may be implemented in systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0189] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0190] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the multi-sensor fusion positioning and control method for the aforementioned cleaning drone. This solves the technical problem that when a drone operates close to building exteriors or photovoltaic arrays, the wall structure, photovoltaic panel supports, and components can severely obstruct satellite signals and cause multipath reflections, leading to a sharp attenuation of GNSS signal strength. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the multi-sensor fusion positioning and control method for the cleaning drone provided in the above embodiments, and will not be repeated here.

[0191] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multi-sensor fusion positioning and control method for cleaning drones as described above.

[0192] The computer program product provided in this application can solve the technical problem that when a drone operates close to the exterior wall of a building or a photovoltaic panel array, the wall structure, photovoltaic panel support, and components will severely obstruct satellite signals and cause multipath reflection effects, resulting in a sharp attenuation of GNSS signal strength. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the multi-sensor fusion positioning and control method for cleaning drones provided in the above embodiments, and will not be repeated here.

[0193] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A multi-sensor fusion positioning and control method for a cleaning drone, characterized in that, The method for multi-sensor fusion positioning and control of a cleaning drone, which includes sensors such as lidar, vision sensors, and millimeter-wave radar, is applied to cleaning drones. Acquire LiDAR data, visual image data, and millimeter-wave radar data at the same timestamp; The point cloud density and point cloud plane fitting degree of the lidar are obtained, the number of effective feature points and feature point matching success rate of the visual sensor are obtained, and the echo signal strength and target detection confidence of the millimeter-wave radar are obtained as the working status. The current working environment's light intensity, air humidity, water mist concentration, and the reflectivity of the target surface being cleaned are obtained as environmental data. The weighting coefficients for lidar, vision sensor, and millimeter-wave radar are determined based on the operating status and environmental data. Based on the pre-calibrated extrinsic matrix between the lidar and the vision sensor, the pre-processed lidar data and vision image data are unified under the UAV coordinate system. The extrinsic matrix is ​​obtained through factory calibration or online calibration. An extended Kalman filter state equation is constructed, with the three-dimensional position, three-dimensional velocity, and three-dimensional attitude angle of the cleaning UAV as the system state vector, and the power system input of the cleaning UAV as the control input. The target surface distance and plane normal vector extracted from the preprocessed lidar data are used as the first observation vector, and the lateral offset and rotation angle calculated from the feature point matching results extracted from the preprocessed visual image data are used as the second observation vector. The observation noise covariance matrices of the first observation vector and the second observation vector are adjusted according to the weighting coefficients of the lidar and the visual sensor, respectively. The first and second observation vectors are input into the extended Kalman filter for updating. The high-precision distance information of the lidar data is used to correct the cumulative error of the visual image data caused by feature point drift, and the texture features of the visual image data are used to correct the lateral positioning deviation of the lidar data. The output includes intermediate positioning data containing the relative position and attitude angle of the cleaning drone relative to the surface of the cleaning target. The intermediate positioning data is converted to the millimeter-wave radar coordinate system and timestamped with the target distance and angle information extracted from the millimeter-wave radar data. Initial positioning data is obtained by combining the intermediate positioning data after weighted fusion and alignment based on the millimeter-wave radar weight coefficients with the millimeter-wave radar data. The target positioning data is determined based on the jump magnitude of the initial positioning data; The flight attitude of the cleaning drone and the control parameters of the cleaning module are adjusted based on the target positioning data.

2. The multi-sensor fusion positioning and control method for cleaning drones as described in claim 1, characterized in that, The acquisition of lidar data, visual image data, and millimeter-wave radar data at the same timestamp includes: Acquire raw point cloud data from LiDAR, raw visual image data, and raw echo data from millimeter-wave radar at the same timestamp; The raw point cloud data of the lidar is subjected to statistical filtering, the average distance between each point and a preset number of neighboring points is calculated, discrete noise points with a distance greater than the mean plus a preset multiple of the standard deviation are removed, and the planar features and edge features of the cleaned target surface are extracted to obtain the lidar data. Gaussian denoising is performed on the original visual image data to extract SIFT or SURF feature points from the image, and invalid feature points in areas with gray-level variance lower than a preset reflectivity threshold are removed to obtain the visual image data. The raw echo data from the millimeter-wave radar is subjected to constant false alarm rate (CFAR) filtering, and the signal-to-noise ratio (SNR) threshold and radar cross section (RCS) threshold are adjusted according to the water mist concentration of the current cleaning operation to obtain the millimeter-wave radar data.

3. The multi-sensor fusion positioning and control method for cleaning drones as described in claim 1, characterized in that, The step of determining the weighting coefficients for the lidar data, the visual image data, and the millimeter-wave radar data based on the operating status and the environmental data includes: Determine the working status score of each sensor based on the described working status; Determine the environmental correction coefficients for each sensor based on the environmental data; The working status scores of each sensor are multiplied by the corresponding environmental correction coefficients to obtain the comprehensive scores of LiDAR, vision sensors, and millimeter-wave radar. The weighting coefficients for the lidar, vision sensor, and millimeter-wave radar are determined based on the comprehensive score of the lidar, the comprehensive score of the vision sensor, and the comprehensive score of the millimeter-wave radar.

4. The multi-sensor fusion positioning and control method for cleaning drones as described in claim 1, characterized in that, Determining the target positioning data based on the jump magnitude of the initial positioning data includes: The initial positioning data is smoothed and filtered. When the amplitude of the initial positioning data jump exceeds a preset threshold, the weight coefficient of the target sensor is reduced and the fusion calculation is performed again. The output target positioning data is processed by smoothing and filtering. The target positioning data includes at least the vertical distance, horizontal offset, pitch angle, roll angle and yaw angle of the cleaning drone relative to the surface of the cleaning target.

5. The multi-sensor fusion positioning and control method for cleaning drones as described in claim 1, characterized in that, The step of adjusting the flight attitude of the cleaning drone and the control parameters of the cleaning module based on the target positioning data includes: Extract the vertical distance, horizontal offset, pitch angle, roll angle, and yaw angle of the UAV relative to the target surface to be cleaned from the target positioning data; The vertical distance is compared with a preset benchmark working distance threshold to generate a height control command; Based on the pitch angle, roll angle and yaw angle, attitude control commands are generated to control the UAV to adjust its attitude so that the UAV fuselage remains parallel to the surface of the target being cleaned. Based on the horizontal offset, a horizontal movement control command is generated so that the cleaning drone moves horizontally along the surface of the cleaning target and remains on the preset cleaning trajectory. Adjust the water pump output pressure of the cleaning module according to the vertical distance; Based on the target surface type in the target positioning data, adjust the nozzle angle and water flow rate of the cleaning module.

6. A multi-sensor fusion positioning and control device for cleaning drones, characterized in that, The multi-sensor fusion positioning and control device for the cleaning drone includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multi-sensor fusion positioning and control method for the cleaning drone as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, it implements the steps of the multi-sensor fusion positioning and control method for cleaning drones as described in any one of claims 1 to 5.

Images