Control method, device and storage medium of unmanned aerial vehicle

Abstract

The application discloses a kind of unmanned plane control method, equipment and storage medium, the present application relates to unmanned plane control technical field, the control method of unmanned plane includes: infrared data and image data are aligned according to coordinate, obtain fusion data, to eliminate the spatial deviation of infrared data and image data;Water mist concentration and water mist particle size are parsed based on fusion data;After water mist concentration and water mist particle size in the preset continuous detection time period are aligned according to time sequence, the water mist concentration distribution between unmanned plane and wall is fitted;Signal-to-noise ratio threshold and radar scattering cross section threshold of radar sensor are adjusted according to water mist concentration distribution.The present application can realize the technical effect that the water mist interference that unmanned plane sprays avoids wall ranging.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, and particularly relates to a control method, device and storage medium of unmanned aerial vehicle. BACKGROUND

[0002] At present, when a wall surface is cleaned by using an unmanned aerial vehicle, the line of sight of the unmanned aerial vehicle is disturbed by water mist, that is, the water mist sprayed by the unmanned aerial vehicle disturbs the ranging of the unmanned aerial vehicle to the wall surface, which causes a risk to use. SUMMARY

[0003] The main purpose of the present application is to provide a control method, device and storage medium of unmanned aerial vehicle, which aims to solve the technical problem that the water mist sprayed by the unmanned aerial vehicle disturbs the ranging to the wall surface.

[0004] To achieve the above-mentioned purpose, the present application provides a control method of unmanned aerial vehicle, which comprises the following steps:

[0005] The infrared data and the image data are aligned according to coordinates to obtain fusion data, so as to eliminate the spatial deviation of the infrared data and the image data;

[0006] The water mist concentration and the water mist particle size are analyzed based on the fusion data;

[0007] After the water mist concentration and the water mist particle size in a preset continuous detection time period are aligned in time sequence, the water mist concentration distribution between the unmanned aerial vehicle and the wall surface is fitted;

[0008] The signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor are adjusted according to the water mist concentration distribution.

[0009] In an embodiment, the step of aligning the infrared data and the image data according to coordinates comprises:

[0010] The pixel coordinates of the image data are converted into the measurement coordinate system of the infrared sensor based on the attitude data collected by the IMU sensor carried by the unmanned aerial vehicle and the preset calibration parameters of the camera and the infrared sensor;

[0011] The infrared data and the image data are spatially aligned based on the converted coordinates to obtain fusion data.

[0012] In an embodiment, the step of analyzing the water mist concentration and the water mist particle size based on the fusion data comprises:

[0013] The fusion data is subjected to image segmentation processing to extract the pixel features of the water mist region;

[0014] The water mist particle size is inversely obtained based on the scattering intensity of the water mist region in the infrared data and in combination with a preset scattering intensity-particle size mapping relationship;

[0015] The pixel proportion of the water mist area in the fused data image is counted, and the water mist concentration at the corresponding spatial position is calculated based on the field of view angle, pixel resolution of the infrared sensor and the image sensor, and the preset installation parameters.

[0016] In an embodiment, after the water mist concentration and the water mist particle size in the preset continuous detection time period are aligned in time sequence, the step of fitting the water mist concentration distribution between the unmanned aerial vehicle and the wall surface includes:

[0017] The acquisition time stamps of the infrared data and the image data are obtained, the water mist concentration and the water mist particle size at the same time stamp are matched one by one, and a time sequence data sequence is formed.

[0018] A three-dimensional space coordinate system is established with the distance between the unmanned aerial vehicle and the wall surface as the z-axis, the horizontal direction as the x-axis, and the vertical direction as the y-axis.

[0019] Based on the water mist concentration, the water mist particle size, and the corresponding spatial coordinates in the time sequence data sequence, a three-dimensional Gaussian mixture model is used to fit the water mist concentration distribution of the water mist in the three-dimensional space.

[0020] In an embodiment, the step of adjusting the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor according to the water mist concentration distribution includes:

[0021] Based on the fitted water mist concentration distribution, all coordinate points on the detection path of the radar sensor are extracted, the water mist concentration values and the water mist particle size values corresponding to each coordinate point are obtained, and the average concentration and the average particle size on the detection path are obtained through sliding window mean filtering processing.

[0022] A preset basic mapping table is called, the basic mapping table is generated by experimental data training, and records the initial signal-to-noise ratio threshold and the initial radar scattering cross-section threshold corresponding to different concentration intervals and different particle sizes.

[0023] Based on the real-time flight attitude of the unmanned aerial vehicle and the current signal-to-noise ratio of the radar detection signal, a dynamic correction threshold is calculated.

[0024] The proportional integral algorithm is used to fuse the initial signal-to-noise ratio threshold and the initial radar scattering cross-section threshold corresponding to the average concentration and the average particle size, and the dynamic correction coefficient to calculate the target signal-to-noise ratio threshold and the target radar scattering cross-section threshold.

[0025] The radar sensor is adjusted according to the target signal-to-noise ratio threshold and the target radar scattering cross-section threshold.

[0026] In an embodiment, after the step of adjusting the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor according to the water mist concentration distribution, it includes:

[0027] The flight height of the unmanned aerial vehicle, the operation wind speed, and the wall surface material parameter are fused with the water mist concentration distribution to obtain a target water mist concentration distribution.

[0028] The signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor are adjusted according to the target water mist concentration distribution, wherein the wall surface material parameter is obtained by texture feature recognition of image data, and the operation wind speed is obtained by a wind speed sensor carried by the unmanned aerial vehicle.

[0029] In an embodiment, the step of adjusting the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor according to the target water mist concentration distribution comprises:

[0030] The radar detection path is spatially partitioned to obtain a short-range partition and a long-range partition;

[0031] The target signal-to-noise ratio threshold and the target radar scattering cross-section threshold of each spatial partition are calculated according to the target water mist concentration;

[0032] According to the spatial partition, the corresponding target threshold is respectively issued to the radar sensor to adjust the radar parameters of each spatial partition on the detection path.

[0033] In an embodiment, after the step of adjusting the signal-to-noise ratio and the radar scattering cross-section of the radar sensor according to the water mist concentration distribution, the method comprises:

[0034] The real-time distance between the unmanned aerial vehicle and the wall surface is determined according to the adjusted radar sensor;

[0035] The unmanned aerial vehicle is controlled to perform a wall surface cleaning task based on the real-time distance.

[0036] In addition, to achieve the above purpose, the present application also provides a control device of an unmanned aerial vehicle, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the control method of the unmanned aerial vehicle.

[0037] In addition, to achieve the above purpose, the present application also provides a storage medium, which is a computer readable storage medium, and the computer readable storage medium stores a program for implementing the control method of the unmanned aerial vehicle, and the program for implementing the control method of the unmanned aerial vehicle is executed by a processor to implement the steps of the control method of the unmanned aerial vehicle.

[0038] The application provides a control method of a UAV, and first obtains fusion data by aligning infrared data and image data according to coordinates, so as to eliminate the spatial deviation of the infrared data and the image data; water mist concentration and water mist particle size are analyzed based on the fusion data; after the water mist concentration and the water mist particle size in a preset continuous detection time period are aligned in time sequence, the water mist concentration distribution between the UAV and a wall surface is fitted; and the signal-to-noise ratio threshold and the radar scattering cross section threshold of the radar sensor are adjusted according to the water mist concentration distribution. The signal-to-noise ratio threshold and the radar scattering cross section threshold of the radar sensor are adjusted based on the water mist concentration distribution, so that the radar sensor is adapted to the change of the water mist environment, and the ranging of the wall surface is avoided from being interfered by the water mist sprayed by the UAV. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced here, and obviously, other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0041] Figure 1 A flowchart provided by the control method of the UAV in Embodiment One of the present application is shown in the figure;

[0042] Figure 2 A flowchart provided by the control method of the UAV in Embodiment Six of the present application is shown in the figure;

[0043] Figure 3 A flowchart provided by the control method of the UAV in Embodiment Eight of the present application is shown in the figure;

[0044] Figure 4 A structural diagram of the UAV control device in the embodiments of the present application is shown in the figure.

[0045] The purposes, functional features and advantages of the present application will be further explained with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0047] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below with reference to the accompanying drawings and the specific embodiments.

[0048] At present, when the unmanned aerial vehicle is used to clean the wall surface, the line of sight of the unmanned aerial vehicle will be disturbed by the water mist, that is, the water mist sprayed by the unmanned aerial vehicle will interfere with the ranging of the unmanned aerial vehicle to the wall surface, causing risks to the use.

[0049] The main solution of the present application is that the present application firstly aligns the infrared data and the image data according to the coordinates to obtain fusion data, so as to eliminate the spatial deviation of the infrared data and the image data; the water mist concentration and the water mist particle size are analyzed based on the fusion data; after the water mist concentration and the water mist particle size in the preset continuous detection time period are aligned in time sequence, the water mist concentration distribution between the unmanned aerial vehicle and the wall surface is fitted; and the signal-to-noise ratio threshold and the radar scattering cross section threshold of the radar sensor are adjusted according to the water mist concentration distribution. The present application adjusts the signal-to-noise ratio threshold and the radar scattering cross section threshold of the radar sensor based on the water mist concentration distribution, so that the radar sensor adapts to the change of the water mist environment, and avoids the interference of the water mist sprayed by the unmanned aerial vehicle to the ranging of the wall surface.

[0050] It should be noted that the execution subject of the present embodiment can be a control device of an unmanned aerial vehicle, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a control device of an unmanned aerial vehicle capable of realizing the above functions, etc., and the present embodiment does not make specific limitation thereto. The present embodiment and each of the following embodiments will be described below by taking a control device of an unmanned aerial vehicle as an example.

[0051] Based on this, the present application provides a control method of an unmanned aerial vehicle, which will be described below with reference to Figure 1 , Figure 1 The present application provides a flowchart of the control method of the unmanned aerial vehicle. The control method of the unmanned aerial vehicle comprises steps S10-S40.

[0052] In step S10, the infrared data and the image data are aligned according to the coordinates to obtain fusion data, so as to eliminate the spatial deviation of the infrared data and the image data.

[0053] The infrared data is heat radiation information data collected by an infrared sensor, records temperature differences of objects, and is presented in the form of a numerical matrix or a grayscale image. It is not affected by visible light and can identify heat-emitting targets in environments such as darkness and fog, but has low spatial resolution and blurred details. The image data generally refers to visual information data collected by a visible light sensor, reflects the appearance characteristics such as color, texture, and shape of an object, and is a color or black-and-white image that can be directly identified by the human eye. The image data has high spatial resolution and rich details, but depends on lighting conditions and cannot identify temperature information of an object. The fusion data is comprehensive information data generated by integrating the infrared data and the image data after coordinate alignment. The temperature information of the infrared data and the appearance information of the image data are superimposed based on the same spatial coordinate system. The fusion data retains the clear details of the image and contains the temperature characteristics of the infrared data, eliminating the spatial deviation of the two types of data.

[0054] In the embodiment, the same coordinate points of corresponding objects in the infrared data and the image data are determined based on the same spatial coordinate system, and the positions of the infrared data or the image data are adjusted. It is ensured that the coordinates of the same physical target in the two types of data completely coincide. The fusion data is obtained by aligning the infrared data and the image data according to the coordinates. The fusion data retains the clear texture and color details of the image and contains the temperature difference information of the infrared data, eliminating the spatial deviation of the infrared data and the image data.

[0055] In the first feasible implementation, the step S10 can include: synchronously collecting the infrared data and the image data, recording the collection time stamps, the sensor installation positions, and the attitude parameters of the two types of data. Feature points in the image data are extracted, and feature points of corresponding targets in the infrared data are synchronously extracted. A corresponding relationship between the feature points of the two types of data is established based on a feature point matching algorithm, a coordinate conversion matrix is calculated, and preliminary coordinate alignment is completed. Without additional calibration objects, the alignment can be completed only by relying on the characteristics of the data itself, and the operation is flexible and has strong adaptability.

[0056] In the second feasible implementation, the step S10 can include: selecting a calibration board with a known size and placing it in a common field of view of the infrared sensor and the image sensor. Infrared data and image data of the calibration board are collected respectively, and standard coordinate points on the calibration board are identified. A mapping relationship between the standard coordinate points of the calibration board and corresponding points in the infrared data is established based on a pixel coordinate system of the image data. Coordinate conversion parameters are solved by a least squares method, and coordinate alignment of the two types of data is completed based on the parameters. The calibration board provides explicit standard coordinate points, the mapping relationship is intuitive, and the accuracy of the coordinate conversion parameters is high.

[0057] In a third possible implementation, step S10 can include: respectively pre-processing the infrared data and the image data, using a region-based registration method, selecting a target region in the image data, and searching for a corresponding region with similar gray scale distribution in the infrared data. The coordinate offset of the corresponding region is calculated, and a global optimization algorithm is used to correct the spatial deviation of the two types of data. The coordinate-aligned infrared data and image data are output, providing a basis for subsequent fusion processing. The global optimization algorithm can correct local deviations and achieve overall spatial consistency, resulting in smoother alignment.

[0058] The above are only three possible implementations of step S10 provided by the present embodiment, and the present embodiment does not specifically limit the specific implementation of step S10.

[0059] Step S20: Analyzing the water mist concentration and the water mist particle size based on the fusion data.

[0060] The water mist concentration is the mass or volume fraction of water mist per unit volume, with units of mg / m³ or mL / m³, reflecting the density of water mist in a unit space. The higher the concentration, the weaker the penetration of visible light or infrared light. The water mist particle size is a size parameter of individual droplets in the water mist, usually described by the particle diameter. The water mist particle size directly affects the physical behavior of the water mist, such as settling speed and scattering characteristics. The larger the particle, the more obvious the light blocking effect.

[0061] In the present embodiment, the fusion data is processed to extract the water mist region. The characteristics of the water mist region are extracted, and the water mist concentration and the water mist particle size are calculated and analyzed. This achieves accurate and efficient analysis of water mist characteristics, providing reliable data support for decision-making in water mist-related scenarios.

[0062] In a first possible implementation, step S20 can include: separating the image information of the visible light channel and the thermal radiation information of the infrared channel from the fusion data, and extracting the pixel features of the water mist region. A mapping model of water mist concentration and gray scale attenuation value is established, and the water mist concentration is calculated based on the gray scale change of the image pixels. The thermal radiation scattering characteristics of the water mist particles in the infrared channel are used, combined with the pixel point spread function, to back-calculate the equivalent diameter range of the water mist particles. The calculation results are statistically filtered to remove outliers, and the average water mist concentration and the particle size distribution interval are output. Based on the pre-set mapping model and pixel features, the results can be quickly output, supporting real-time monitoring scenarios.

[0063] In the second feasible implementation, step S20 can include: image segmentation of the fusion data, accurate extraction of the water mist area by threshold method, and exclusion of background interference. The light intensity attenuation coefficient and infrared absorption coefficient of the water mist area in the fusion data are input into the laser radar scattering principle adaptive model to calculate the absolute value of the water mist concentration. The scattering difference of the water mist in different wave bands in the fusion data is combined to solve the accurate value of the particle diameter. The single-frame data error is corrected by combining the time sequence analysis of multiple frames of fusion data, and the dynamic change curve of the water mist concentration and the statistical distribution of the particle size are output. Based on the laser radar scattering principle and combined with accurate image segmentation, the concentration analysis error is smaller, and the particle size can be accurately quantified to the micron level.

[0064] In step S30, after aligning the water mist concentration and the water mist particle size in the preset continuous detection time period in time sequence, the water mist concentration distribution between the unmanned aerial vehicle and the wall surface is fitted.

[0065] In this embodiment, all data in the continuous detection time period are extracted, sorted according to the detection time stamp, and the water mist concentration, the water mist particle size, and the unmanned aerial vehicle space coordinates are one-to-one corresponding. The water mist concentration distribution between the unmanned aerial vehicle and the wall surface is fitted taking the wall surface as a fixed reference. Based on the data and coordinate binding in time sequence alignment, the fitted distribution model can accurately map the water mist concentration difference in different positions.

[0066] In the first feasible implementation, step S30 can include: extracting time sequence data in the continuous detection time period, binding the water mist concentration and particle size data with the real-time space coordinates of the unmanned aerial vehicle based on the unmanned aerial vehicle detection time stamp, and eliminating data with time synchronization error exceeding the threshold. A two-dimensional simplified coordinate system is established taking the wall surface as a reference surface, different intervals from the wall surface are divided at fixed distance intervals, and the average water mist concentration in each interval is counted. The concentration blank values in each interval are filled by using a linear interpolation algorithm, the water mist concentration distribution between the unmanned aerial vehicle and the wall surface is fitted, and the average range of the particle size of the corresponding interval is labeled. The water mist concentration distribution trend in different spatial positions is determined to meet the rapid investigation demand.

[0067] In the second feasible implementation, step S30 can include: fine preprocessing of the time sequence data, correction of detection delay by a time synchronization algorithm, ensuring the time sequence consistency of the water mist concentration, the water mist particle size, and the unmanned aerial vehicle three-dimensional coordinates, and completing the missing data. A three-dimensional space coordinate system is established, each time sequence data point is accurately mapped into the three-dimensional coordinates to form a discrete space data set. An interpolation algorithm is used to generate a continuous three-dimensional water mist concentration distribution, and the particle size parameter is associated to label the particle diameter distribution corresponding to different concentration regions. The spatial relationship between the unmanned aerial vehicle and the wall surface is completely restored through the three-dimensional coordinate system, and the spatial gradient change of the concentration can be accurately presented.

[0068] Step S40, adjust the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor according to the water mist concentration distribution.

[0069] The signal-to-noise ratio threshold is a ratio of the effective target signal strength to the background noise strength in the signal received by the radar sensor. The signal received by the radar sensor includes target reflection signal and environmental noise, and the signal-to-noise ratio is the intensity ratio of the two. After setting the signal-to-noise ratio threshold, the radar sensor only retains the signal with a signal-to-noise ratio higher than the threshold, and the signal below the threshold is filtered out as noise. The signal-to-noise ratio threshold is directly related to the water mist concentration. Water mist can enhance environmental noise and attenuate target signal. Therefore, the signal-to-noise ratio threshold needs to be adjusted according to the concentration, such as appropriately reducing the signal-to-noise ratio threshold under high water mist concentration to avoid missing detection, and increasing the signal-to-noise ratio threshold under low water mist concentration to reduce false detection. The radar scattering cross-section threshold is the minimum judgment standard for measuring the reflection ability of the target to the radar wave. The radar scattering cross-section reflects the strength of the target reflecting radar wave. The larger the size of the target and the stronger the reflection of the material, the higher the radar scattering cross-section value. After setting the threshold, the radar only identifies objects with a radar scattering cross-section value higher than the threshold as effective targets, and objects with a radar scattering cross-section value lower than the threshold are ignored. The radar scattering cross-section threshold is related to the water mist concentration and particle size. Water mist changes the propagation path of radar waves, and the water mist particle group produces collective scattering. Therefore, the radar scattering cross-section threshold needs to be adjusted according to the concentration distribution, such as appropriately increasing the radar scattering cross-section threshold under high water mist concentration to avoid misjudging water mist as a target.

[0070] In this embodiment, the radar sensor real-time acquires the water mist concentration of the current detection position from the fitted water mist concentration distribution, and adjusts the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor. The thresholds are dynamically adjusted according to the water mist concentration distribution, adapting to different scenes from low water mist concentration to high water mist concentration.

[0071] In a first feasible implementation, step S40 can include: setting high, medium and low concentration intervals according to the fitted water mist concentration distribution, and clearly defining the spatial distribution range of each interval. Based on the experimental calibration data, the signal-to-noise ratio threshold and the radar scattering cross-section threshold corresponding to each concentration interval are formulated, such as reducing the signal-to-noise ratio threshold by 10%-15% under high water mist concentration, and increasing the radar scattering cross-section threshold by 20%-30%. The radar sensor automatically calls the preset threshold parameters according to the concentration interval to which the current detection position belongs, and completes the rapid switching of the signal-to-noise ratio threshold and the radar scattering cross-section threshold. Every fixed period, the latest water mist concentration distribution is updated, and if the concentration crosses the interval, the corresponding threshold is switched again. The preset threshold according to the high, medium and low concentration intervals has simple switching logic, can quickly respond to changes in water mist concentration distribution, and adapts to real-time detection scenes.

[0072] In a second possible implementation, the step S40 can include: establishing a concentration threshold mapping model, constructing a continuous function model of water mist concentration and signal-to-noise ratio threshold, radar scattering cross-section threshold based on a large amount of experimental data, and determining the threshold adjustment amount corresponding to each change of 1 mg / m³ of water mist concentration. From the fitted water mist concentration distribution, the real-time water mist concentration value and concentration change rate of the current detection position of the radar are accurately obtained. The real-time concentration value and change rate are input into the mapping model to dynamically calculate the optimal signal-to-noise ratio threshold and radar scattering cross-section threshold in the current scene. Based on the continuous function model, the threshold can be finely adjusted according to the real-time concentration value and change rate, rather than jumping in intervals, and adapt to non-uniform water mist concentration distribution.

[0073] The embodiment provides a control method of a UAV. The application first obtains fusion data by aligning infrared data and image data according to coordinates, so as to eliminate the spatial deviation of the infrared data and the image data; water mist concentration and water mist particle size are analyzed based on the fusion data; after the water mist concentration and the water mist particle size in a preset continuous detection time period are aligned in time sequence, the water mist concentration distribution between the UAV and the wall surface is fitted; and the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor are adjusted according to the water mist concentration distribution. The application adjusts the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor based on the water mist concentration distribution, so that the radar sensor adapts to the change of the water mist environment and avoids the ranging interference of the water mist sprayed by the UAV on the wall surface.

[0074] Based on embodiment one, the same or similar content as the above embodiment one can be referred to the above introduction, and will not be described in detail hereinafter. On this basis, the step of aligning the infrared data and the image data according to the coordinates includes:

[0075] In step S11, the pixel coordinates of the image data are converted into the measurement coordinate system of the infrared sensor based on the attitude data collected by the IMU sensor carried by the UAV and the preset calibration parameters of the camera and the infrared sensor.

[0076] The IMU (Inertial Measurement Unit) is one of the core sensors on the UAV, consisting of an accelerometer and a gyroscope. The accelerometer measures the UAV's linear acceleration in three-dimensional space, i.e., acceleration along the X, Y, and Z axes. The gyroscope measures the UAV's angular velocity around its three-dimensional axes, i.e., rotational speed in the roll, pitch, and yaw directions. Attitude data, output by the IMU, describes the UAV's spatial attitude, including Euler angles and quaternions. Euler angles are typically expressed as roll around the X-axis, pitch around the Y-axis, and yaw around the Z-axis, reflecting the UAV's tilt and rotation relative to the geodetic coordinate system. Quaternions are another way to represent attitude, avoiding the gimbal lock problem of Euler angles and more accurately describing the rotational attitude in three-dimensional space. Attitude data can be used to determine the relative angular relationships between the camera, infrared sensor, and the geodetic coordinate system. Preset calibration parameters are fixed parameters obtained through pre-calibration experiments, describing the spatial position and optical characteristics between the camera and the infrared sensor. These mainly include intrinsic parameters, which are the internal parameters of both the camera and the infrared sensor, such as focal length, pixel size, principal point coordinates, and distortion coefficients. These are used to correct lens distortion and convert pixel coordinates into the sensor's own three-dimensional normalized coordinates. Extrinsic parameters are the relative pose parameters between the camera and the infrared sensor, describing their spatial installation position and angular deviations.

[0077] In this embodiment, real-time attitude data and image pixel coordinates acquired by the IMU sensor are obtained, and timestamps are synchronized to ensure data consistency. Using camera intrinsic parameters, distortion correction is performed on the image pixel coordinates, which are then converted to the camera's own three-dimensional normalized coordinates. A rotation matrix is ​​constructed based on the attitude data to transform the camera's normalized coordinates to the geodetic coordinate system, compensating for coordinate deviations caused by the UAV's tilt and rotation. Using infrared sensor extrinsic parameters, the coordinates in the geodetic coordinate system are transformed to the infrared sensor's measurement coordinate system, outputting the final transformed three-dimensional coordinates, i.e., the infrared sensor's measurement coordinate system. By combining attitude data with preset calibration parameters, both inherent sensor installation deviations are corrected, and the effects of attitude fluctuations during UAV flight are offset.

[0078] Step S12: Spatially align the infrared data and image data based on the transformed coordinates to obtain fused data.

[0079] In this embodiment, the transformed infrared measurement coordinate system is used as a unified reference. The pixel coordinates of the image data are mapped to this coordinate system, and the measurement coordinates of the infrared data are matched with the mapped image coordinates, ensuring a one-to-one correspondence between the physical spatial locations described by the two, thus achieving spatial alignment. The aligned image pixel information is then bound to the infrared data information to form fused data of spatial coordinates, image features, and infrared features. Through spatial alignment, the spatial deviation between the infrared data and the image data is eliminated.

[0080] In this embodiment, by performing spatial alignment, the obtained fusion data retains the texture details of the image data and the thermal radiation characteristics of the infrared data, and the two are one-to-one corresponding in space, avoiding data invalidation caused by information misplacement.

[0081] Based on any one of the above embodiments of the present application, the third embodiment of the present application proposes a control method of the unmanned aerial vehicle, which can be referred to the above introduction and will not be described in detail hereinafter. On this basis, the step of analyzing the water mist concentration and the water mist particle size based on the fusion data comprises:

[0082] Step S21, performing image segmentation processing on the fusion data to extract pixel features of the water mist region.

[0083] The pixel features refer to the visual and physical characteristics presented by the pixels of the water mist region in the fusion data, which can be used to distinguish the water mist from the background, mainly including color, texture, gray scale, infrared radiation and other multi-dimensional features.

[0084] In this embodiment, a deep learning method can be used, such as training a model with labeled water mist background samples to learn the features of the water mist in the fusion data and automatically segment the complete water mist region. The pixel features of the water mist region are extracted, such as the visible light features, i.e. calculating the average RGB value, gray mean and variance, texture entropy, edge gradient mean, etc. in the water mist region. The infrared features are calculated, i.e. the infrared radiation intensity mean, radiation value standard deviation, and radiation difference value between the water mist region and the background region, etc. Both the water mist region can be accurately located and its essential features can be extracted, reducing the analysis error caused by background interference.

[0085] Step S22, based on the scattering intensity of the water mist region in the infrared data, combining the preset mapping relationship between the scattering intensity and the particle size, the water mist particle size is obtained by inversion.

[0086] The scattering intensity refers to the strength of the scattering effect of the water mist particles on infrared light, which is the core physical property presented by the water mist region in the infrared data and is directly related to the water mist particle size.

[0087] In this embodiment, the original radiation value of the infrared sensor is used to subtract the infrared radiation reference value of the background to obtain the radiation attenuation amount caused by the scattering of the water mist, which is then converted into the scattering intensity. Through the preset mapping relationship between the scattering intensity and the particle size, the water mist particle size corresponding to the scattering intensity is read. The particle size can be inverted without contacting the water mist, which is suitable for dynamic detection scenarios of the unmanned aerial vehicle and avoids the interference of the sampling process on the distribution of the water mist.

[0088] Step S23, the pixel proportion of the water mist region in the fusion data image is counted, and the water mist concentration at the corresponding spatial position is calculated by combining the field of view angle, pixel resolution and preset installation parameters of the infrared sensor and the image sensor.

[0089] Wherein, the field of view angle is the angle of the space range that the sensor can observe, which is divided into horizontal field of view angle and vertical field of view angle. For example, the infrared sensor has a horizontal field of view angle of 60° and a vertical field of view angle of 45°, which means that the sensor can cover a conical space area of 60°x45° in front. The pixel resolution is the number of pixels of the sensor image and the actual space size corresponding to a single pixel, including the number of image pixels, such as 1920x1080, i.e. 1920 horizontal pixels and 1080 vertical pixels, which describes the detail carrying capacity of the image. The single-pixel space size is calculated by the field of view angle and the observation distance, such as the size corresponding to a single pixel in the horizontal direction = observed horizontal width / horizontal pixel number. The preset installation parameter is the fixed installation information of the infrared sensor and the image sensor on the unmanned aerial vehicle, including the relative position parameter, i.e. the three-dimensional coordinate offset of the two on the unmanned aerial vehicle body. The relative angle parameter, i.e. the installation attitude deviation of the two, clearly shows the difference in the field of view direction.

[0090] In this embodiment, the total number of pixels in the water mist area and the total number of pixels in the fused data image are counted to calculate the pixel ratio. By the field of view angle and the distance, the actual physical volume corresponding to the space position is calculated, and then by the pixel resolution, the water mist pixel number is converted into the actual occupied space of the water mist in the volume to obtain the space ratio, and further obtain the water mist concentration corresponding to the space position. The precise calculation of the water mist concentration is realized.

[0091] In this embodiment, the visual ratio in the image is converted into the water mist concentration in the physical space to realize precise concentration calculation, which provides core data for subsequent distribution fitting and radar threshold adjustment.

[0092] Based on any one of the above embodiments of the present application, the fourth embodiment of the present application proposes a control method of an unmanned aerial vehicle, which can be referred to the above introduction and will not be described in detail hereinafter. On this basis, the water mist concentration and the water mist particle size in the preset continuous detection time period are aligned in time sequence, and then the step of fitting the water mist concentration distribution between the unmanned aerial vehicle and the wall surface is included.

[0093] In step S31, the acquisition time stamps of the infrared data and the image data are obtained, and the water mist concentration and the water mist particle size at the same time stamp are matched one by one to form a time sequence data sequence.

[0094] Wherein, the acquisition time stamp is the data generation time mark synchronously recorded by the infrared sensor and the image sensor when collecting each frame of data. The time sequence data sequence is an ordered data set formed by arranging the water mist concentration and the water mist particle size matched at the same time stamp in the order of acquisition time.

[0095] In the embodiment, the acquisition time stamp corresponding to each frame of data is extracted from the raw data of the infrared sensor and the image sensor. All time stamps are traversed to screen out frame data pairs in which the time stamps in the infrared data and the image data are completely consistent. For each matched frame, the water mist concentration value and the water mist particle size value corresponding to the frame calculated previously are extracted to establish a one-to-one correspondence. All matched time stamp, water mist concentration, and water mist particle size data pairs are sorted in the order of acquisition time stamp. The repeated or abnormal time stamp data are removed to finally form a structured time sequence data sequence. The traceable and analyzable time sequence flow is formed to provide subsequent time-sequentially arranged associated data.

[0096] In step S32, a three-dimensional space coordinate system is established with the distance between the unmanned aerial vehicle and the wall as the z-axis, the horizontal direction as the x-axis, and the vertical direction as the y-axis.

[0097] In the embodiment, the geometric center of the unmanned aerial vehicle is taken as the coordinate origin (0, 0, 0) to ensure the relevance of the sensor data and the coordinate origin. The x-axis is parallel to the horizontal direction of the ground and along the vertical direction of the unmanned aerial vehicle flight, and the right direction is positive. The y-axis is perpendicular to the vertical direction of the ground, and the upward direction is positive, forming a right-handed coordinate system with the x-axis and the z-axis. The z-axis is along the connecting direction of the unmanned aerial vehicle and the wall, and the positive direction is pointing to the wall, and the z-axis value directly corresponds to the real-time distance between the unmanned aerial vehicle and the wall. By establishing the three-dimensional space coordinate system, a unified, accurate, and dynamically adaptive space positioning reference is provided to realize one-to-one binding of the water mist concentration and the three-dimensional space position.

[0098] In step S33, based on the water mist concentration, the water mist particle size, and the corresponding space coordinates in the time sequence data sequence, a three-dimensional Gaussian mixture model is used to fit the water mist concentration distribution of the water mist in the three-dimensional space.

[0099] The three-dimensional Gaussian mixture model is a weighted combination of multiple three-dimensional Gaussian distributions, and each Gaussian distribution corresponds to a local feature region of the data distribution. The model learns the spatial distribution law of the data by using the superposition of multiple Gaussian components to approximate the real three-dimensional distribution, and finally outputs the target variable probability or prediction value at any position in the three-dimensional space.

[0100] In this embodiment, all valid samples are extracted from the time series data sequence, each sample contains three-dimensional coordinates (x_i, y_i, z_i), corresponding water mist concentration C_i, water mist particle size S_i, and abnormal samples are removed. Take (x, y, z) as the core space feature, and take the particle size S_i as the auxiliary feature to improve the fitting accuracy of the model for different particle distribution areas. Select the optimal number of Gaussian distribution components, such as 3-10, corresponding to the possible multiple aggregation areas of water mist. Take the concentration C_i as the target, input the (x, y, z, S_i) features, and iteratively optimize the model parameters through the expectation maximization algorithm to minimize the error between the model prediction value and the actual concentration. After the model training is completed, the entire detection three-dimensional space is grid sampled to generate a dense three-dimensional grid point. The coordinates of each grid point are input into the model to predict the water mist concentration value at that position, and finally a continuous water mist concentration distribution data covering the entire space is formed. Compared with single Gaussian model or linear fitting, three-dimensional Gaussian mixture model can accurately capture the non-uniform distribution of water mist and has higher fitting accuracy.

[0101] Based on any one of the above embodiments of the present application, the fifth embodiment of the present application proposes a control method of the unmanned aerial vehicle, which can be referred to the above introduction and will not be repeated hereinafter. On this basis, the step of adjusting the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor according to the water mist concentration distribution comprises:

[0102] In step S41, based on the fitted water mist concentration distribution, all coordinate points on the detection path of the radar sensor are extracted, the water mist concentration value and the water mist particle size value corresponding to each coordinate point are obtained, and the average concentration and the average particle size on the detection path are obtained through sliding window mean filtering processing.

[0103] Among them, the sliding window mean filtering is a simple and efficient time domain smoothing filtering method, by setting a fixed size window sliding on the data sequence, calculating the mean value of all data in the window as the output value of the current position, so as to weaken the random noise and smooth the data fluctuation, and obtain more stable average concentration and average particle size.

[0104] In this embodiment, based on the radar installation position, detection direction, the coordinate point sequence of the path is generated in the three-dimensional coordinate system. From the fitted water mist concentration distribution data, the water mist concentration value and the water mist particle size value corresponding to the above coordinate points are extracted to form the path data sequence of coordinates, water mist concentration, and water mist particle size. According to the path coordinate point density, the window size is selected to ensure that the window can contain enough data to reflect the local environmental characteristics, while avoiding the trend distortion caused by too large window. The sliding step is set, usually 1 coordinate point, to ensure that the filtered data sequence is consistent with the original path length without data omission. The concentration value and the particle size value in the path data sequence are filtered respectively, and the window slides from the starting position of the sequence. The mean value of all data in the window is calculated each time, such as water mist concentration mean value = window concentration sum / window size, and the water mist particle size mean value is the same. The filtered detection path average concentration sequence and average particle size sequence are output, and each coordinate point corresponds to a smoothed average parameter value. Through the filtering process, the single-point abnormal value interference is eliminated, and the average parameter can better reflect the real water mist environment state on the path.

[0105] Step S42, a preset basic mapping table is called, and the basic mapping table is generated by experimental data training and records the initial signal-to-noise ratio threshold and the initial radar scattering cross-section threshold corresponding to different concentration intervals and different particle sizes.

[0106] In this embodiment, the detection path average concentration value and the average particle size value obtained in the last step are extracted. The intervals to which the two parameters belong are judged respectively. The concentration interval and the particle size interval are used as indexes to query the basic mapping table to extract the corresponding initial signal-to-noise ratio threshold and the initial radar scattering cross-section threshold. By directly calling the preset mapping table, there is no need to calculate the threshold in real time, which meets the real-time requirement of dynamic detection of the unmanned aerial vehicle.

[0107] Step S43, the dynamic correction threshold is calculated based on the real-time flight attitude of the unmanned aerial vehicle and the current signal-to-noise ratio of the radar detection signal.

[0108] Among them, the dynamic correction threshold is the final radar threshold obtained by real-time adjustment based on the initial threshold of the basic mapping table, combined with the real-time flight attitude of the unmanned aerial vehicle and the current signal-to-noise ratio of the radar detection signal, which can dynamically adapt to the flight state and signal change to ensure the radar detection accuracy.

[0109] In the embodiment, the real-time flight attitude data of the unmanned aerial vehicle and the current signal-to-noise ratio of the radar detection signal are collected, the roll angle, the pitch angle and the yaw angle are obtained through the IMU sensor, and the deviation value from the standard attitude is calculated. The attitude deviation threshold is set, if the deviation value is less than or equal to the attitude deviation threshold, the correction amount is 0; if the deviation value is greater than the attitude deviation threshold, the correction amount is calculated according to the formula, the greater the attitude deviation, the greater the absolute value of the correction amount. The signal-to-noise ratio deviation = current signal-to-noise ratio - signal-to-noise ratio standard value is calculated, and the correction amount formula is established to correct the current signal-to-noise ratio. The dynamic correction threshold is updated at a preset frequency. Through the rapid response to the attitude fluctuation of the unmanned aerial vehicle and the change of the signal quality, the failure of the fixed threshold in the complex flight environment is avoided.

[0110] In step S44, the proportional integral algorithm is used to fuse the initial signal-to-noise ratio threshold and the initial radar scattering cross section threshold corresponding to the average concentration and the average particle size with the dynamic correction coefficient, and the target signal-to-noise ratio threshold and the target radar scattering cross section threshold are calculated.

[0111] The proportional integral algorithm is a closed-loop control algorithm based on proportional regulation and integral regulation, which can accurately fuse the initial threshold and the dynamic correction coefficient, and optimize the output through real-time error feedback to finally obtain the target threshold which is stable and adaptive to the current environment.

[0112] In the embodiment, the initial signal-to-noise ratio threshold and the initial radar scattering cross section threshold corresponding to the average concentration and the average particle size are obtained, and the preset proportional coefficient and the integral coefficient are obtained. The proportional calculation outputs the adjustment amount in proportion to the current deviation of the dynamic correction coefficient, the greater the deviation, the stronger the adjustment strength, and the rapid response is realized. The integral calculation accumulates the historical deviation of the dynamic correction coefficient and outputs the compensation amount to eliminate the static deviation that cannot be offset by the proportional link. The initial signal-to-noise ratio threshold and the initial radar scattering cross section threshold are adjusted through the preset proportional coefficient and the integral coefficient to obtain the target signal-to-noise ratio threshold and the target radar scattering cross section threshold. Through the synergistic effect of the proportional and integral, the threshold is prevented from frequent oscillation caused by instantaneous deviation, and the stability of the radar detection state is ensured.

[0113] In step S45, the radar sensor is adjusted according to the target signal-to-noise ratio threshold and the target radar scattering cross section threshold.

[0114] In the embodiment, the target signal-to-noise ratio threshold and the target radar scattering cross section threshold are extracted and converted into a parameter format recognizable by the radar sensor to adjust the radar sensor. It is ensured that the radar can realize accurate detection of the wall target in the current water mist environment and flight state.

[0115] In this embodiment, the initial signal-to-noise ratio threshold and the initial radar scattering cross-section threshold are corrected to obtain a target signal-to-noise ratio threshold and a target radar scattering cross-section threshold, the detection sensitivity of the sensor and the signal recognition standard are dynamically adjusted, and the radar can accurately detect the wall target in the current water mist environment and flight state, and the anti-interference performance and the recognition rate are considered.

[0116] Based on any one of the above embodiments of the present application, the control method of the unmanned aerial vehicle is provided in Embodiment Six of the present application. Please refer to the above description, and the subsequent description will not be repeated. On this basis, please refer to Figure 2 , Figure 2 The flowchart provided in Embodiment Six of the control method of the unmanned aerial vehicle of the present application. After the step of adjusting the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor according to the water mist concentration distribution, it includes:

[0117] Step S401, the flight height of the unmanned aerial vehicle, the working wind speed, and the wall surface material parameters are fused with the water mist concentration distribution to obtain a target water mist concentration distribution.

[0118] The flight height of the unmanned aerial vehicle refers to the vertical height of the unmanned aerial vehicle relative to the ground, which is obtained by the GPS or visual positioning system of the unmanned aerial vehicle. The working wind speed refers to the real-time wind speed and direction in the working environment of the unmanned aerial vehicle, which is measured by the wind speed sensor carried by the unmanned aerial vehicle. The wall surface material parameter refers to the surface material characteristic parameter of the detected target wall, which mainly includes the material type, the surface roughness, and the water absorption rate, which is manually input in advance or automatically recognized by the visual recognition system.

[0119] In this embodiment, the flight height affects the water mist gravity sedimentation effect, the water mist is highly aggregated at low altitude, and the concentration fitting is given a higher weight, the water mist is easily diffused at high altitude, and the weight coefficient is reduced. The greater the working wind speed, the more dispersed the concentration distribution is, and according to the wind direction, the concentration distribution is stretched along the diffusion direction in the three-dimensional coordinate system. According to the wall surface material type, combined with the roughness and the water absorption rate, a local correction factor is fitted. The concentration near the wall is calibrated, and the concentration value in the small value region of the z-axis is corrected. The three correction results are superimposed to generate the final three-dimensional target water mist concentration distribution.

[0120] Step S402, adjusting the signal-to-noise ratio threshold and the radar scattering cross-section threshold of the radar sensor according to the target water mist concentration distribution; wherein the wall surface material parameter is obtained by texture feature recognition of image data, and the working wind speed is obtained by the wind speed sensor carried by the unmanned aerial vehicle.

[0121] In this embodiment, the wall surface material parameters are obtained through texture feature recognition of image data, and the working wind speed is obtained through a wind speed sensor carried by the unmanned aerial vehicle. Based on the target water mist concentration distribution, the three-dimensional space is divided into high, medium and low regions according to the concentration value, and the threshold value is adjusted in different regions. In the high concentration region, the signal-to-noise ratio threshold value is reduced, and the radar scattering cross section threshold value is increased. In the medium concentration region, the signal-to-noise ratio threshold value and the radar scattering cross section threshold value maintain the basic threshold value. In the low concentration region, the signal-to-noise ratio threshold value is increased, and the radar scattering cross section threshold value is reduced. The regional adjustment is based on the real target concentration distribution, and the influence of wind speed and material is taken into account, so as to avoid the insufficient adaptation of the global unified threshold value.

[0122] In this embodiment, the target concentration distribution accurately reflects the water mist state under different heights, different wind speeds and different wall surface materials, and provides more reliable environmental data for regional and refined adjustment of the radar threshold value. The entire technical solution can cope with different working heights, meteorological conditions and wall surface types, and reduce the scene restriction.

[0123] Based on any one of the above embodiments of the application, the seventh embodiment of the application provides a control method of an unmanned aerial vehicle, which can be referred to the above description and will not be described hereinafter. On this basis, the steps of adjusting the signal-to-noise ratio threshold value and the radar scattering cross section threshold value of the radar sensor according to the target water mist concentration distribution, comprising:

[0124] Step S4021, the radar detection path is spatially partitioned to obtain a near-range partition and a long-range partition.

[0125] In this embodiment, the z-axis of the three-dimensional coordinate system, i.e. the distance between the unmanned aerial vehicle and the wall surface, is combined with the water mist diffusion law and the radar signal attenuation characteristic to set the demarcation point. The optimal demarcation point is calibrated through experiments in combination with historical operation data and radar performance parameters. If the radar signal attenuation rate changes obviously at z=6m, and the near-range water mist is concentrated and the long-range water mist is dispersed, the z=6m is set as the demarcation point. z

[0126] Step S4022, the target signal-to-noise ratio threshold value and the target radar scattering cross section threshold value of each spatial partition are calculated according to the target water mist concentration.

[0127] In the embodiment, the near-range partition water mist concentration is easily affected by the wall material, the remote partition water mist concentration is more affected by the wind speed diffusion and signal attenuation, and the concentration correction coefficient and the characteristic correction coefficient of the near-range partition and the remote partition are obtained respectively. The concentration correction coefficient is obtained according to the target water mist concentration, the characteristic correction coefficient of the near-range partition is obtained according to the wall material, and the characteristic correction coefficient of the remote partition is obtained according to the working wind speed. The target signal-to-noise ratio threshold and the target radar scattering cross section threshold of each spatial partition are obtained according to the concentration correction coefficient and the characteristic correction coefficient. According to the concentration characteristics and environmental influence differences of the near-range and the remote, the thresholds are calculated respectively, and the local adaptation failure caused by global adjustment is avoided.

[0128] Step S4023, according to the spatial partition, the corresponding target threshold is respectively issued to the radar sensor to adjust the radar parameters of each spatial partition on the detection path.

[0129] In the embodiment, according to the communication protocol of the radar sensor, the partition threshold configuration instruction is generated, the instruction includes the partition identification, the coordinate range, the target signal-to-noise ratio threshold and the target radar scattering cross section threshold. The instruction is sent to the radar sensor to adjust the radar parameters of each spatial partition on the detection path. The radar parameters of the near-range and the remote partition are independently adapted to the respective environment, and the local detection failure caused by the same configuration is avoided.

[0130] In the embodiment, by performing spatial partition, adjustment of different partitions is realized, and the detection stability, anti-interference ability and scene adaptability in different distance water mist environments are greatly improved.

[0131] Based on any one of the above embodiments of the application, the eighth embodiment of the application provides a control method of the unmanned aerial vehicle, which can be referred to in the above description, and will not be described hereinafter. On this basis, please refer to Figure 3 , Figure 3 The flowchart provided by the eighth embodiment of the control method of the unmanned aerial vehicle of the application. After the step of adjusting the signal-to-noise ratio and the radar scattering cross section of the radar sensor according to the water mist concentration distribution, it includes:

[0132] Step S403, determining the real-time distance between the unmanned aerial vehicle and the wall according to the adjusted radar sensor.

[0133] In the embodiment, the radar transmits 24GHZ electromagnetic waves, the radar receives the echo through the cfar filtering noise and interference, and the correct target point is screened. The signal transmission time and the effective echo receiving time are recorded, and the real-time distance between the unmanned aerial vehicle and the wall is calculated. The real-time distance between the unmanned aerial vehicle and the wall is accurately obtained by transmitting and receiving electromagnetic wave signals.

[0134] Step S404, controlling the unmanned aerial vehicle to perform a wall cleaning task based on the real-time distance.

[0135] In the embodiment, the real-time distance output by the radar sensor is taken as a core feedback signal, preset operation parameters are combined, the flight attitude and the cleaning execution mechanism are dynamically adjusted by the unmanned aerial vehicle system, and accurate and efficient operation of wall cleaning is realized.

[0136] In the embodiment, the unmanned aerial vehicle automatically performs the path according to the real-time distance, without manual intervention, and the operation efficiency is improved.

[0137] The control device of the unmanned aerial vehicle provided in the embodiment of the application adopts the control method of the unmanned aerial vehicle in the above embodiment, and can solve the technical problem that the water mist sprayed by the unmanned aerial vehicle interferes with the distance measurement of the wall. Compared with the prior art, the control device of the unmanned aerial vehicle provided in the embodiment of the application has the same beneficial effects as the control method of the unmanned aerial vehicle provided in the above embodiment, and other technical features in the control device of the unmanned aerial vehicle are the same as the features disclosed in the above embodiment, which will not be described here.

[0138] The control system of the unmanned aerial vehicle provided in the embodiment of the application can solve the technical problem that the water mist sprayed by the unmanned aerial vehicle interferes with the distance measurement of the wall. Compared with the prior art, the control system of the unmanned aerial vehicle provided in the embodiment of the application has the same beneficial effects as the control method of the unmanned aerial vehicle provided in the above embodiment, which will not be described here.

[0139] The application provides a control device of an unmanned aerial vehicle, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the control method of the unmanned aerial vehicle in the above embodiment one.

[0140] Reference will be made to the following description Figure 4 which shows a structural schematic diagram of the control device of the unmanned aerial vehicle suitable for implementing the embodiments of the application. The control device of the unmanned aerial vehicle in the embodiments of the application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDA), tablet computers, vehicle-mounted terminals, and fixed terminals such as digital TVs and desktop computers. Figure 4 The control device of the unmanned aerial vehicle shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the application.

[0141] As Figure 4As shown, the control device of the unmanned aerial vehicle can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the control device of the unmanned aerial vehicle are also stored in the random access memory 1004. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the control device of the unmanned aerial vehicle to communicate wirelessly or wiredly with other devices to exchange data. Although the control device of the unmanned aerial vehicle with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0142] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0143] The control device of the unmanned aerial vehicle provided by the present disclosure adopts the control method of the unmanned aerial vehicle in the above-mentioned embodiments, and can solve the technical problem that the water mist sprayed by the unmanned aerial vehicle can interfere with the distance measurement of the wall surface. Compared with the prior art, the control device of the unmanned aerial vehicle provided by the present disclosure has the same beneficial effects as the control device of the unmanned aerial vehicle provided by the above-mentioned embodiments, and other technical features in the control device of the unmanned aerial vehicle are the same as the features disclosed in the above-mentioned embodiments, which will not be repeated here.

[0144] It should be understood that various parts of the present application can be realized with hardware, software, firmware, or a combination thereof. In the above description of embodiments, specific functional configurations, structures, materials or characteristics can be combined in any appropriate manner in one or more embodiments or examples.

[0145] The above description is merely a specific implementation of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, and all such changes or replacements should be covered within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the scope of protection of the claims.

[0146] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the control method of the unmanned aerial vehicle in the above-described embodiments.

[0147] The computer readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted in any appropriate medium, including but not limited to an electrical wire, an optical cable, a radio frequency (RF), or any appropriate combination of the above.

[0148] The above computer readable storage medium can be included in the control device of the unmanned aerial vehicle; or can exist separately without being assembled into the control device of the unmanned aerial vehicle.

[0149] The computer readable storage medium carries one or more programs, when the one or more programs are executed by the control device of the unmanned aerial vehicle, the control device of the unmanned aerial vehicle: aligns the infrared data and the image data according to coordinates to obtain fusion data, so as to eliminate spatial deviation of the infrared data and the image data; analyzes the water mist concentration and the water mist particle size based on the fusion data; after aligning the water mist concentration and the water mist particle size in a preset continuous detection time period in time sequence, fits the water mist concentration distribution between the unmanned aerial vehicle and the wall surface; and adjusts the signal-to-noise ratio threshold and the radar scattering cross section threshold of the radar sensor according to the water mist concentration distribution.

[0150] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0151] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block in the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by special-purpose hardware-based systems that perform the specified functions or operations, or combinations of special-purpose hardware and computer instructions.

[0152] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not constitute a limitation on the modules themselves.

[0153] The readable storage medium provided in the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the control method of the unmanned aerial vehicle, and can solve the technical problem that the water mist sprayed by the unmanned aerial vehicle interferes with the distance measurement of the wall. Compared with the prior art, the computer readable storage medium provided in the present application has the same beneficial effects as the control method of the unmanned aerial vehicle provided in the above embodiments, and will not be described here.

[0154] The embodiments of the present application provide a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the control method of the unmanned aerial vehicle are implemented.

[0155] The computer program product provided in the present application can solve the technical problem that the water mist sprayed by the unmanned aerial vehicle interferes with the distance measurement of the wall. Compared with the prior art, the computer program product provided in the embodiments of the present application has the same beneficial effects as the control method of the unmanned aerial vehicle provided in the above embodiments, and will not be described here.

[0156] The above is only the preferred embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation based on the content of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent processing scope of the present application.

Claims

1. A control method for an unmanned aerial vehicle (UAV), characterized in that, The control method for the unmanned aerial vehicle includes: Infrared data and image data are aligned according to coordinates to obtain fused data, thereby eliminating spatial deviation between infrared data and image data; Based on the fused data, the water mist concentration and water mist particle size were analyzed. After aligning the water mist concentration and water mist particle size within the preset continuous detection time period according to the time sequence, the water mist concentration distribution between the drone and the wall is fitted. The signal-to-noise ratio threshold and radar cross-section threshold of the radar sensor are adjusted according to the water mist concentration distribution. Based on the adjusted radar sensor, the real-time distance between the drone and the wall is determined; The drone is controlled to perform wall cleaning tasks based on the real-time distance. The step of adjusting the signal-to-noise ratio threshold and radar cross-section threshold of the radar sensor according to the water mist concentration distribution includes: Based on the fitted water mist concentration distribution, all coordinate points on the radar sensor detection path are extracted, and the water mist concentration value and water mist particle size value corresponding to each coordinate point are obtained. The average concentration and average particle size on the detection path are obtained by means filtering through a sliding window. A preset base mapping table is invoked. The base mapping table is generated through training with experimental data and records the initial signal-to-noise ratio threshold and the initial radar cross section threshold corresponding to different concentration ranges and different particle sizes. The dynamic correction threshold is calculated based on the real-time flight attitude of the UAV and the current signal-to-noise ratio of the radar detection signal. The proportional-integral algorithm is used to fuse the initial signal-to-noise ratio thresholds and initial radar cross-section thresholds corresponding to the average concentration and average particle size with the dynamic correction coefficients to calculate the target signal-to-noise ratio threshold and the target radar cross-section threshold. The radar sensor is adjusted based on the target signal-to-noise ratio threshold and the target radar cross section threshold.

2. The control method for a drone as described in claim 1, characterized in that, The step of aligning infrared data and image data according to coordinates includes: Based on the attitude data collected by the IMU sensor on the drone and the preset calibration parameters of the camera and infrared sensor, the pixel coordinates of the image data are converted into the measurement coordinate system of the infrared sensor. Based on the transformed coordinates, the infrared data and image data are spatially aligned to obtain fused data.

3. The control method for a drone as described in claim 1, characterized in that, The step of resolving the water mist concentration and water mist particle size based on the fused data includes: Image segmentation processing is performed on the fused data to extract pixel features of the water mist region; Based on the scattering intensity of the water mist region in infrared data, and combined with the preset mapping relationship between scattering intensity and particle size, the size of the water mist particles is obtained by inversion. The percentage of pixels in the water mist area in the fused data image is statistically analyzed. By combining the field of view, pixel resolution, and preset installation parameters of the infrared sensor and the image sensor, the water mist concentration at the corresponding spatial location is calculated.

4. The control method for a drone as described in claim 1, characterized in that, The step of fitting the water mist concentration distribution between the drone and the wall surface after aligning the water mist concentration and water mist particle size within a preset continuous detection time period according to time sequence includes: The acquisition timestamps of infrared data and image data are obtained, and the water mist concentration and water mist particle size at the same timestamp are matched one by one to form a time-series data sequence. A three-dimensional spatial coordinate system is established with the distance between the drone and the wall as the z-axis, the horizontal direction as the x-axis, and the vertical direction as the y-axis. Based on the water mist concentration, water mist particle size and corresponding spatial coordinates in the time series data, the water mist concentration distribution in three-dimensional space is obtained by fitting a three-dimensional Gaussian mixture model.

5. The control method for a drone as described in claim 1, characterized in that, After the step of adjusting the signal-to-noise ratio threshold and radar cross-section threshold of the radar sensor according to the water mist concentration distribution, the following steps are included: The target water mist concentration distribution is obtained by integrating the drone's flight altitude, operating wind speed, and wall material parameters with the water mist concentration distribution. The signal-to-noise ratio threshold and radar cross-section threshold of the radar sensor are adjusted according to the target water mist concentration distribution; the wall material parameters are obtained by identifying the texture features of the image data, and the operating wind speed is collected by the wind speed sensor carried by the UAV.

6. The control method for a drone as described in claim 5, characterized in that, The steps of adjusting the signal-to-noise ratio threshold and radar cross-section threshold of the radar sensor according to the target water mist concentration distribution include: The radar detection path is spatially partitioned to obtain short-range and long-range partitions; Calculate the target signal-to-noise ratio threshold and target radar cross-section threshold for each spatial zone based on the target water mist concentration; According to the spatial partitions, the corresponding target thresholds are sent to the radar sensors to adjust the radar parameters of each spatial partition along the detection path.

7. A control device for an unmanned aerial vehicle (UAV), characterized in that, The control device of the unmanned aerial vehicle (UAV) includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for the UAV as described in any one of claims 1 to 6.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the control method for the unmanned aerial vehicle as described in any one of claims 1 to 6.

Images