Method for processing unmanned aerial vehicle environment perception data, control terminal and storage medium

Abstract

The application relates to the technical field of unmanned aerial vehicles, in particular to a processing method for unmanned aerial vehicle environment sensing data, a control terminal and a storage medium, wherein the method comprises the following steps: determining high-frequency information, pixel jitter values and light transmittances corresponding to image data collected by an unmanned aerial vehicle; determining noise types existing in a current environment and noise intensities corresponding to the noise types according to the high-frequency information, the pixel jitter values and the light transmittances; determining a target noise reduction algorithm for noise reduction according to the noise types and the noise intensities; and performing noise reduction processing on the image data based on the target noise reduction algorithm. The target noise reduction algorithm of a corresponding scene is selected according to noise characteristics in image data collected by an unmanned aerial vehicle to perform noise reduction processing on the image data, so that the effect of reducing the interference of environmental factors on the image data collected by the unmanned aerial vehicle is achieved, and the problem of how to improve the data collection quality of the unmanned aerial vehicle is solved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a method for processing environmental perception data of UAVs, a control terminal, and a storage medium. Background Technology

[0002] To improve the efficiency of urban safety supervision, drones can be used for automated inspections in some urban supervision projects. Automated inspections of cities using drones can improve the comprehensiveness and effectiveness of urban supervision while reducing labor costs.

[0003] Current solutions for drone data acquisition typically rely on the environmental data collected by the drone to perform inspection tasks. This limits drone operation to good weather conditions with minimal environmental interference, thus preventing data distortion.

[0004] Therefore, when drones operate in adverse weather conditions with significant environmental interference factors such as wind, rain, and fog, the collected data will be significantly distorted, resulting in unstable data collection quality.

[0005] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this invention is to provide a method for processing environmental perception data from unmanned aerial vehicles (UAVs), aiming to solve the problem of how to improve the data acquisition quality of UAVs.

[0007] To achieve the above objectives, the present invention provides a method for processing environmental perception data from unmanned aerial vehicles (UAVs), the method comprising:

[0008] Determine the high-frequency information, pixel jitter value, and transmittance corresponding to the image data collected by the drone;

[0009] Based on the high-frequency information, the pixel jitter value, and the transmittance, the type of noise present in the current environment and the noise intensity corresponding to the noise type are determined.

[0010] Based on the noise type and the noise intensity, a target noise reduction algorithm is determined for noise reduction.

[0011] The image data is denoised based on the target denoising algorithm.

[0012] Optionally, the step of determining the type of noise present in the current environment and the noise intensity corresponding to the noise type based on the high-frequency information, the pixel jitter value, and the transmittance includes:

[0013] The portion of high-frequency information in the image data with a sparsity greater than a preset sparsity threshold is identified as rain noise, and the intensity of rain noise is determined based on its proportion in the image data; and,

[0014] The portion of the pixel jitter value exceeding a preset jitter threshold is identified as wind noise, and the wind noise intensity is determined based on the proportion of wind noise in the image data; and,

[0015] The portion of the light transmittance outside the preset light transmittance range is identified as light noise, and the intensity of light noise is determined based on the proportion of light noise in the image data.

[0016] Optionally, the step of determining the high-frequency information, pixel jitter value, and transmittance corresponding to the image data collected by the UAV includes:

[0017] Based on a preset high-frequency image information extraction algorithm, high-frequency information is extracted from the image data; and,

[0018] The pixel jitter value is determined based on the coordinate change between pixel coordinate sets of the same pixel region between at least two consecutively acquired image data sets, and / or feature points are extracted between at least two consecutively acquired image data sets, and the pixel jitter value is determined based on the feature change between the feature points; and,

[0019] The signal-to-noise ratio, contrast, and / or visibility of the image data are obtained, and the transmittance is determined based on the signal-to-noise ratio, the contrast, and / or the visibility.

[0020] Optionally, the step of determining the target noise reduction algorithm for noise reduction based on the noise type and the noise intensity includes:

[0021] The noise reduction parameters in the preset noise reduction algorithm are adjusted according to the noise type and the noise intensity, and the adjusted preset noise reduction algorithm is determined as the target noise reduction algorithm; or...

[0022] Based on the noise type and the noise intensity, a target noise reduction algorithm is selected from the available noise reduction algorithms.

[0023] Optionally, adjusting the noise reduction parameters in the preset noise reduction algorithm according to the noise type and the noise intensity includes at least one of the following:

[0024] When the noise type is rain noise, the high-frequency information filling value of the image data is determined according to the rain noise intensity corresponding to the rain noise, so as to improve the sparsity of high-frequency information in the image data based on the high-frequency information filling value.

[0025] When the noise type is wind noise, the pixel stability value of the image data is determined according to the wind noise intensity corresponding to the wind noise, so as to reduce the pixel jitter in the image data based on the pixel stability value;

[0026] When the noise type is light noise, the target signal-to-noise ratio, target contrast and / or target visibility of the image data are determined according to the light noise intensity corresponding to the light noise, so as to improve the transmittance of the image data based on the target signal-to-noise ratio, the target contrast and / or the target visibility.

[0027] Optionally, the target denoising algorithm includes a dehazing algorithm, and the step of performing denoising processing on the image data based on the target denoising algorithm includes:

[0028] The image data is denoised based on the dehazing algorithm to obtain dehazed image data;

[0029] After the step of denoising the image data based on the target denoising algorithm, the method further includes:

[0030] The visible boundary value, gradient mean, and / or saturated pixel in the dehazed image data are determined, and the dehazing result of the dehazed image data is determined based on the visible boundary value, the gradient mean, and / or the saturated pixel. When the dehazing result meets the preset conditions, the UAV is controlled to perform an inspection task based on the dehazed image data.

[0031] Alternatively, the mean squared error, peak signal-to-noise ratio, and / or structural similarity between the dehazed image data and the image data can be determined, and the dehazing threshold can be determined based on the mean squared error, the peak signal-to-noise ratio, and / or the structural similarity. When the dehazing threshold is greater than or equal to the dehazing threshold, the UAV can be controlled to perform an inspection task based on the dehazed image data.

[0032] Optionally, the drone includes a laser sensor, and after the step of denoising the image data based on the target denoising algorithm, the method further includes:

[0033] The laser sensor acquires point cloud data collected at multiple historical moments, wherein the point cloud data and the image data are collected from the same data acquisition object;

[0034] Based on the point cloud data, determine the overlapping regions in the denoised image data that satisfy the preset geometric registration relationship;

[0035] The point cloud data and the image data in the overlapping area are fused to obtain fused data, which is then used to identify the data acquisition object.

[0036] Optionally, before the step of performing noise reduction processing on the image data based on the target noise reduction algorithm, the method further includes:

[0037] Obtain the image dimensions of the image data;

[0038] The target filtering window size corresponding to the image data is determined based on the image size;

[0039] Based on the target filtering window size, the image data is smoothed using a dark channel prior dehazing algorithm.

[0040] In addition, to achieve the above objectives, the present invention also provides a control terminal, the control terminal comprising: a memory, a processor, and a processing program for UAV environmental perception data stored in the memory and executable on the processor, wherein when the UAV environmental perception data processing program is executed by the processor, it implements the steps of the UAV environmental perception data processing method described above.

[0041] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a processing program for UAV environmental perception data, wherein the UAV environmental perception data processing program, when executed by a processor, implements the steps of the UAV environmental perception data processing method described above.

[0042] This invention provides a method for processing environmental perception data from a UAV, a control terminal, and a storage medium. By analyzing high-frequency information, pixel jitter values, and transmittance in the image data collected by the UAV, the method determines the noise type and corresponding noise intensity in the UAV's environment. Based on the noise type and intensity, a target noise reduction algorithm is determined, and finally, the image data is denoised using the selected algorithm. This reduces the interference of environmental factors on the image data collected by the UAV. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the hardware operating environment of the control terminal involved in an embodiment of the present invention;

[0044] Figure 2 This is a flowchart illustrating the first embodiment of the method for processing environmental perception data from an unmanned aerial vehicle (UAV) according to the present invention.

[0045] Figure 3 This is a flowchart illustrating a second embodiment of the method for processing UAV environmental perception data according to the present invention.

[0046] Figure 4 This is a flowchart illustrating a third embodiment of the method for processing environmental perception data from an unmanned aerial vehicle (UAV) according to the present invention.

[0047] Figure 5 This is a flowchart illustrating the fourth embodiment of the method for processing UAV environmental perception data according to the present invention.

[0048] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0050] As one implementation scheme, Figure 1 This is a schematic diagram of the hardware operating environment of the control terminal involved in the embodiment of the present invention.

[0051] like Figure 1 As shown, the control terminal may include: a processor 1001, such as a CPU; a memory 1005; a user interface 1003; a network interface 1004; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1 The control terminal architecture shown does not constitute a limitation on the control terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0053] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a program for processing UAV environmental perception data. The operating system is a program that manages and controls the hardware and software resources of the terminal, the UAV environmental perception data processing program, and the operation of other software or programs.

[0054] exist Figure 1In the control terminal shown, the user interface 1003 is mainly used to connect to the terminal and communicate with the terminal; the network interface 1004 is mainly used to communicate with the backend server; and the processor 1001 can be used to call the processing program for UAV environmental perception data stored in the memory 1005.

[0055] In this embodiment, the control terminal includes: a memory 1005, a processor 1001, and a processing program for UAV environmental perception data stored in the memory and executable on the processor, wherein:

[0056] When processor 1001 calls the processing program for UAV environmental perception data stored in memory 1005, it performs the following operations:

[0057] Determine the high-frequency information, pixel jitter value, and transmittance corresponding to the image data collected by the drone;

[0058] Based on the high-frequency information, the pixel jitter value, and the transmittance, the type of noise present in the current environment and the noise intensity corresponding to the noise type are determined.

[0059] Based on the noise type and the noise intensity, a target noise reduction algorithm is determined for noise reduction.

[0060] The image data is denoised based on the target denoising algorithm.

[0061] When processor 1001 calls the processing program for UAV environmental perception data stored in memory 1005, it performs the following operations:

[0062] The portion of high-frequency information in the image data with a sparsity greater than a preset sparsity threshold is identified as rain noise, and the intensity of rain noise is determined based on its proportion in the image data; and,

[0063] The portion of the pixel jitter value exceeding a preset jitter threshold is identified as wind noise, and the wind noise intensity is determined based on the proportion of wind noise in the image data; and,

[0064] The portion of the light transmittance outside the preset light transmittance range is identified as light noise, and the intensity of light noise is determined based on the proportion of light noise in the image data.

[0065] When processor 1001 calls the processing program for UAV environmental perception data stored in memory 1005, it performs the following operations:

[0066] Based on a preset high-frequency image information extraction algorithm, high-frequency information is extracted from the image data; and,

[0067] The pixel jitter value is determined based on the coordinate change between pixel coordinate sets of the same pixel region between at least two consecutively acquired image data sets, and / or feature points are extracted between at least two consecutively acquired image data sets, and the pixel jitter value is determined based on the feature change between the feature points; and,

[0068] The signal-to-noise ratio, contrast, and / or visibility of the image data are obtained, and the transmittance is determined based on the signal-to-noise ratio, the contrast, and / or the visibility.

[0069] When processor 1001 calls the processing program for UAV environmental perception data stored in memory 1005, it performs the following operations:

[0070] The noise reduction parameters in the preset noise reduction algorithm are adjusted according to the noise type and the noise intensity, and the adjusted preset noise reduction algorithm is determined as the target noise reduction algorithm; or...

[0071] Based on the noise type and the noise intensity, a target noise reduction algorithm is selected from the available noise reduction algorithms.

[0072] When processor 1001 calls the processing program for UAV environmental perception data stored in memory 1005, it performs the following operations:

[0073] When the noise type is rain noise, the high-frequency information filling value of the image data is determined according to the rain noise intensity corresponding to the rain noise, so as to improve the sparsity of high-frequency information in the image data based on the high-frequency information filling value.

[0074] When the noise type is wind noise, the pixel stability value of the image data is determined according to the wind noise intensity corresponding to the wind noise, so as to reduce the pixel jitter in the image data based on the pixel stability value;

[0075] When the noise type is light noise, the target signal-to-noise ratio, target contrast and / or target visibility of the image data are determined according to the light noise intensity corresponding to the light noise, so as to improve the transmittance of the image data based on the target signal-to-noise ratio, the target contrast and / or the target visibility.

[0076] When processor 1001 calls the processing program for UAV environmental perception data stored in memory 1005, it performs the following operations:

[0077] The image data is denoised based on the dehazing algorithm to obtain dehazed image data;

[0078] The visible boundary value, gradient mean, and / or saturated pixel in the dehazed image data are determined, and the dehazing result of the dehazed image data is determined based on the visible boundary value, the gradient mean, and / or the saturated pixel. When the dehazing result meets the preset conditions, the UAV is controlled to perform an inspection task based on the dehazed image data.

[0079] Alternatively, the mean squared error, peak signal-to-noise ratio, and / or structural similarity between the dehazed image data and the image data can be determined, and the dehazing threshold can be determined based on the mean squared error, the peak signal-to-noise ratio, and / or the structural similarity. When the dehazing threshold is greater than or equal to the dehazing threshold, the UAV can be controlled to perform an inspection task based on the dehazed image data.

[0080] When processor 1001 calls the processing program for UAV environmental perception data stored in memory 1005, it performs the following operations:

[0081] The laser sensor acquires point cloud data collected at multiple historical moments, wherein the point cloud data and the image data are collected from the same data acquisition object;

[0082] Based on the point cloud data, determine the overlapping regions in the denoised image data that satisfy the preset geometric registration relationship;

[0083] The point cloud data and the image data in the overlapping area are fused to obtain fused data, which is then used to identify the data acquisition object.

[0084] When processor 1001 calls the processing program for UAV environmental perception data stored in memory 1005, it performs the following operations:

[0085] Obtain the image dimensions of the image data;

[0086] The target filtering window size corresponding to the image data is determined based on the image size;

[0087] Based on the target filtering window size, the image data is smoothed using a dark channel prior dehazing algorithm.

[0088] Based on the hardware architecture of the control terminal based on UAV control technology described above, an embodiment of the UAV environmental perception data processing method of the present invention is proposed.

[0089] Reference Figure 2 In the first embodiment, the method for processing the UAV environmental perception data includes the following steps:

[0090] Step S10: Determine the high-frequency information, pixel jitter value, and transmittance corresponding to the image data collected by the UAV;

[0091] In this embodiment, the drone is equipped with an image acquisition device. When the drone is performing an inspection task, it will acquire image data at certain time intervals. After acquiring the image data, the high-frequency information, pixel jitter value and transmittance corresponding to the image data are determined.

[0092] Image information can be considered as a combination of high-frequency and low-frequency information. The main structural information of an image exists in the low-frequency part, while image details exist in the high-frequency part. High-frequency information represents the details in the image data. Pixel jitter represents the portion of the image data that changes between two consecutively acquired images. Transmittance represents the transmittance of visible light in the image data.

[0093] For drones, when it is raining, the rain has a certain sparsity in the image texture. Therefore, the feature part that represents rain mainly exists in the high-frequency part of the image. So, the high-frequency information in the image data can be used to determine whether the image data collected by the drone is raining.

[0094] When a drone is in windy weather, the wind will cause the drone and the lens to shake, resulting in a deviation in the image of the object. Therefore, the pixel jitter value in the image can be used to determine whether the drone is in windy weather.

[0095] When a drone is in a poorly lit or unevenly lit environment, the exposure in the image data will be too high or too low. Therefore, the light transmittance in the image can be used to determine whether the lighting in the drone's environment is good. For example, when a drone is in a poorly lit environment such as a foggy day, the light transmittance will be low.

[0096] Step S20: Based on the high-frequency information, the pixel jitter value, and the transmittance, determine the type of noise present in the current environment and the noise intensity corresponding to the noise type;

[0097] In this embodiment, after determining the above information, the noise type and corresponding noise intensity in the drone's environment are determined based on high-frequency information, pixel jitter value, and transmittance. Since there can be one or more noise types in the drone's environment, when these three noise characteristics—high-frequency information, pixel jitter value, and transmittance—meet certain preset conditions, it can be determined that the drone's environment contains a noise type corresponding to that noise characteristic. Furthermore, the magnitude of the noise characteristic is positively correlated with the noise intensity corresponding to its noise type; therefore, the noise intensity can be determined based on the magnitude of the noise characteristic value.

[0098] Optionally, the portion of high-frequency information in the image data with a sparsity greater than a preset sparsity threshold is identified as rain-type noise. Sparsity is a mathematical quantification of the proportion of high-frequency information in the image data relative to the high, medium, and low-frequency information of the entire image data. When the sparsity exceeds the preset sparsity threshold, rain-type noise is determined to exist in the image data. Furthermore, a first mapping relationship is established between sparsity and rain-type noise intensity. When the sparsity exceeds the sparsity threshold, the rain-type noise intensity corresponding to the rain-type noise in the image can be calculated based on the first mapping relationship.

[0099] Optionally, high-frequency information can be extracted using a pre-defined high-frequency information extraction algorithm, mainly including two methods. The first method involves performing a Fourier transform on the image, transforming it from the spatial domain to the frequency domain, then performing a filtering operation to obtain low-frequency information. Finally, subtracting the obtained low-frequency information from the original image yields the corresponding high-frequency information. The second method involves directly filtering the image in the spatial domain to obtain low-frequency information, with the remaining steps similar to the first method. Finally, subtracting the obtained low-frequency information from the original image yields the corresponding high-frequency information.

[0100] Optionally, the portion of pixel jitter values ​​exceeding a preset jitter threshold is identified as wind noise. Furthermore, a second mapping relationship is established between pixel jitter values ​​and wind noise intensity. When a pixel jitter value exceeds the jitter threshold, the wind noise intensity corresponding to the wind noise in the image can be calculated based on this second mapping relationship.

[0101] Optionally, the pixel jitter value can be determined in three ways. The first is to determine the pixel jitter value based on the coordinate change between the pixel coordinate sets of the same pixel region between at least two consecutively acquired image data sets, where a larger coordinate change results in a larger pixel jitter value. The second is to extract feature points between at least two consecutively acquired image data sets and determine the pixel jitter value based on the feature changes between these feature points, where a more significant feature change results in a larger pixel jitter value. The third method combines the aforementioned coordinate changes and feature changes to jointly determine the pixel jitter value. This method involves a larger computational load and is suitable for scenarios with high noise reduction accuracy requirements.

[0102] Optionally, the portion of the transmittance outside a preset transmittance range is defined as optical noise. When the transmittance is below the lower limit of the transmittance range, it is determined that the drone may be in weather conditions with poor light transmission, such as foggy weather; while when the transmittance is above the upper limit of the transmittance range, it is determined that the drone may be in a scene with excessively high light intensity, resulting in abnormal transmittance. Furthermore, a third mapping relationship is established between transmittance and optical noise intensity. When the transmittance is outside the transmittance range, the intensity of optical noise corresponding to the optical noise in the image can be calculated based on the third mapping relationship.

[0103] Optionally, since excessively bright, dim, or uneven lighting can all cause changes in parameters such as signal-to-noise ratio, contrast, and visibility, transmittance can be determined based on the signal-to-noise ratio, contrast, and / or visibility in the image data.

[0104] Furthermore, it should be noted that since there may be more than one type of noise in the environment where the drone is located, the noise type determined based on the above noise characteristics may also be one or more. Therefore, it is understood that the above optional implementation schemes can be selected individually or in multiple ways.

[0105] Step S30: Determine the target noise reduction algorithm for noise reduction based on the noise type and the noise intensity;

[0106] In this embodiment, after determining the noise type and the corresponding noise intensity, a target noise reduction algorithm for noise reduction is determined based on the noise type and noise intensity.

[0107] Optionally, the target noise reduction algorithm can be determined in two ways: one is to adjust the noise reduction parameters in the preset noise reduction algorithm according to the noise type and noise intensity, and determine the adjusted preset noise reduction algorithm as the target noise reduction algorithm; the other is to select a suitable target noise reduction algorithm from the optional noise reduction algorithms that meets the current environment of the UAV according to the noise type and noise intensity.

[0108] Optionally, when the determination method is to adjust the noise reduction parameters in the preset noise reduction algorithm, the noise reduction parameters are different for different types of noise.

[0109] Specifically, when the noise type is rain noise, the high-frequency information filling value of the image data is determined according to the intensity of the rain noise. The high-frequency information filling value is used to fill the part of the image data that is judged to be rain noise, thereby improving the sparsity of high-frequency information in the image data based on the high-frequency information filling value, so as to achieve the effect of removing rain noise.

[0110] When the noise type is wind noise, the pixel stability value of the image data is determined based on the wind noise intensity. The portion of the image data identified as wind noise is stabilized by the pixel stability value, thereby reducing the degree of pixel jitter in the image data.

[0111] When the noise type is light noise, the target signal-to-noise ratio, target contrast and / or target visibility of the image data are determined according to the light noise intensity corresponding to the light noise, so as to optimize the image data based on the target signal-to-noise ratio, the target contrast and / or the target visibility, thereby improving the light transmittance of the image data.

[0112] Optionally, when the determination method is to select a target noise reduction algorithm, the target noise reduction algorithm includes three types: defogging algorithm, rain noise reduction algorithm, and wind noise reduction algorithm, which are used to remove light noise, rain noise, and wind noise, respectively. Target noise reduction algorithms with different noise reduction capabilities are set according to different noise intensities, and one or more target noise reduction algorithms can be selected.

[0113] For example, when it is determined that the image data contains rain noise R and wind noise W, where the intensity of rain noise Rz = 10 and the intensity of wind noise Wz = 8, the selected target noise reduction algorithms are: Rz10 noise reduction algorithm and Wz8 noise reduction algorithm.

[0114] Step S40: Perform noise reduction processing on the image data based on the target noise reduction algorithm.

[0115] In this embodiment, after the target noise reduction algorithm is determined, the image data is denoised according to the target noise reduction algorithm, thereby eliminating noise in the image data collected by the UAV in a complex environment, so that the UAV can perform inspection tasks based on the denoised image data and achieve higher positioning and navigation accuracy.

[0116] In the technical solution provided in this embodiment, the noise type and corresponding noise intensity in the environment where the UAV is located are determined by using high-frequency information, pixel jitter value, and transmittance in the image data collected by the UAV. Based on the noise type and intensity, a target noise reduction algorithm is determined, and finally, the image data is denoised using the selected target noise reduction algorithm. This reduces the interference of environmental factors on the image data collected by the UAV.

[0117] Reference Figure 3 In the second embodiment, based on any embodiment, step S40 includes:

[0118] Step S41: Perform noise reduction processing on the image data based on the dehazing algorithm to obtain dehazed image data;

[0119] After step S40, the method further includes:

[0120] Step S51: Determine the visible boundary value, gradient mean, and / or saturated pixels in the dehazed image data, and determine the dehazing result of the dehazed image data based on the visible boundary value, gradient mean, and / or saturated pixels. When the dehazing result meets the preset conditions, control the UAV to perform an inspection task based on the dehazed image data.

[0121] Alternatively, in step S52, the mean square error, peak signal-to-noise ratio, and / or structural similarity between the dehazed image data and the image data are determined, and the dehazing threshold is determined based on the mean square error, the peak signal-to-noise ratio, and / or the structural similarity. When the dehazing threshold is greater than or equal to the dehazing threshold, the UAV is controlled to perform an inspection task based on the dehazed image data.

[0122] Optionally, the target noise reduction algorithm in this embodiment is a defogging algorithm. Since the defogging effect of the defogging algorithm may be inconsistent in foggy weather with different concentrations, in order to enable the defogging algorithm to adapt to data noise reduction in different scenarios, in this embodiment, the defogging image data obtained based on the defogging algorithm is evaluated from two aspects: qualitative analysis and quantitative evaluation, so as to determine whether the defogging algorithm is suitable for the current scenario of the drone.

[0123] Specifically, if we analyze the dehazing algorithm from a qualitative perspective, this analysis method has no reference object; that is, it directly evaluates the image after dehazing. The evaluation steps are as follows: determine the visible boundary values, gradient mean, and / or saturated pixels in the dehazed image data, and determine the dehazing result of the dehazed image data based on the visible boundary values, the gradient mean, and / or the saturated pixels. The visible edge ratio is used to evaluate the dehazing algorithm's ability to restore image edges, the gradient mean is used to evaluate the dehazing algorithm's ability to restore contrast, and the saturated pixels are used to evaluate the ratio of black and white pixels in the dehazed image. Then, when the dehazing result meets preset conditions, the drone is controlled to perform an inspection task based on the dehazed image data. Optionally, the preset conditions can be one or more of the following: the visible boundary value is greater than or equal to a preset boundary threshold, the gradient mean is greater than or equal to a preset gradient threshold, and the number of saturated pixels is greater than or equal to a preset pixel count threshold. When the preset condition is met, it is determined that the defogging algorithm matches the fog noise in the drone-collected image of the current scene, and the drone is then controlled to perform subsequent inspection tasks based on the defogging image data.

[0124] Specifically, if we analyze dehazing algorithms from a quantitative evaluation perspective, this analysis method has a reference object: comparing the original image data before dehazing with the dehazed image data after dehazing, and then analyzing the results of the comparison. The steps are as follows: First, determine the mean square error, peak signal-to-noise ratio (PSNR), and / or structural similarity between the dehazed image data and the original image data (these parameters can be obtained according to a preset image data algorithm, and will not be elaborated here). The mean square error represents the magnitude of the difference between the image data before and after dehazing; the larger the mean square error, the greater the difference. The PSNR represents the degree of distortion between the image data before and after dehazing; the larger the PSNR, the more severe the distortion. The structural similarity represents the similarity between the image data before and after dehazing; the greater the structural similarity, the more similar the two images are.

[0125] Then, based on the mean squared error, the peak signal-to-noise ratio, and / or the structural similarity, the dehazing threshold is determined. The higher the dehazing threshold, the higher the accuracy of the dehazed image data obtained after dehazing. When the dehazing threshold is greater than a preset dehazing threshold, it is determined that the dehazing algorithm matches the fog noise in the drone-collected image of the current scene, and the drone is controlled to perform subsequent inspection tasks based on the dehazed image data.

[0126] Understandably, whether it is qualitative analysis or quantitative evaluation, when the conclusion is that it does not meet the noise reduction effect standard, the noise reduction parameters and / or the target noise reduction algorithm are readjusted to perform noise reduction processing on the image data.

[0127] In the technical solution provided in this embodiment, the dehazing algorithm is analyzed from both qualitative and quantitative perspectives to determine whether the dehazing algorithm is suitable for the current scene of the drone. This ensures that the image data after noise reduction will not differ too much from the original image data, thereby improving the robustness and stability of the dehazing effect.

[0128] Reference Figure 4 In the third embodiment, based on any embodiment, step S40 is followed by:

[0129] Step S60: Obtain point cloud data collected by the laser sensor at multiple historical moments, wherein the point cloud data and the image data are collected from the same data acquisition object;

[0130] Step S70: Based on the point cloud data, determine the overlapping regions in the denoised image data that satisfy the preset geometric registration relationship;

[0131] Step S80: The point cloud data and the image data in the overlapping area are fused to obtain fused data, so as to identify the data collection object based on the fused data.

[0132] Optionally, in this embodiment, since the buildings in the city and the surrounding environment of the drone may change during the urban inspection process, the recognition range and recognition content of the drone will also change. In order to ensure the recognition accuracy in this scenario, a geometric registration method is introduced to fuse multi-source data collected by different sensors on the drone.

[0133] Specifically, the drone is equipped with a laser sensor to acquire point cloud data collected from multiple historical moments. Based on a preset geometric registration relationship, points in overlapping areas are found among the point clouds from multiple historical moments. Points in non-overlapping areas are considered difference points, and the geometric surfaces formed by these difference points are called difference surfaces. The point cloud data in the overlapping areas and the image data are fused to obtain fused data. The data acquisition object (i.e., building) is identified based on the fused data.

[0134] In addition, to reduce the registration difficulty caused by large differences in resolution, measurement scale, and field of view of cross-source point clouds, a multi-source data registration algorithm based on wavelet transform, voxel centroid neighborhood, and iterative nearest neighbor (ICP) can be introduced to register point cloud data and image data.

[0135] Specifically, the topological relationship of the point cloud is first constructed to improve the point cloud search efficiency. Redundant point cloud data is removed by reducing the density of voxel centroid nearest neighbor points, point cloud pass-through filtering, and outlier removal. The target point cloud data collected under different spatiotemporal conditions are compared based on the ICP algorithm. Then, the spectral RGB and HSV images are decomposed into high-frequency and low-frequency parts through wavelet decomposition. The low-frequency and high-frequency parts are fused separately using appropriate fusion rules to obtain a fused image. This fused image can be used as a supplement to the point cloud feature information. Finally, the target point cloud after removing redundant data is fused with the image to realize the recognition of data collected under different spatiotemporal conditions.

[0136] In the technical solution provided in this embodiment, after the image data is denoised, the denoised image data is registered with the point cloud data, so that the UAV can ensure the accuracy of the identification of the data collection object during the city inspection process.

[0137] Reference Figure 5 In the fourth embodiment, based on any embodiment, before step S40, the method further includes:

[0138] Step S90: Obtain the image size of the image data;

[0139] Step S100: Determine the target filtering window size corresponding to the image data based on the image size;

[0140] Step S110: Based on the target filtering window size, the image data is smoothed using a dark channel prior dehazing algorithm.

[0141] Optionally, in this embodiment, since the halo effect can have a negative impact on the image, using the same size filter window for different image data will affect the filtering effect. Therefore, an adaptive threshold filtering algorithm is used to smooth the image data. Specifically, based on the dark channel prior dehazing principle, the window size is adjusted according to the input image size to smooth the image.

[0142] In the technical solution provided in this embodiment, the size of the filter window is determined according to the size of the acquired image data. Based on this size, the image data is smoothed, thereby reducing the negative impact of halo effect on the image and improving the image processing effect.

[0143] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the control terminal to implement the process steps of the embodiments of the above methods.

[0144] Therefore, the present invention also provides a computer-readable storage medium storing a processing program for UAV environmental perception data, wherein when the UAV environmental perception data processing program is executed by a processor, it implements the various steps of the UAV environmental perception data processing method described in the above embodiments.

[0145] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0146] It should be noted that, since the storage medium provided in the embodiments of this application is the storage medium used to implement the methods of the embodiments of this application, those skilled in the art can understand the specific structure and variations of the storage medium based on the methods described in the embodiments of this application, and therefore will not be repeated here. All storage media used in the methods of the embodiments of this application fall within the scope of protection of this application.

[0147] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0149] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0151] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0152] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0153] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for processing environmental perception data from unmanned aerial vehicles (UAVs), characterized in that, The method comprises: determining high-frequency information, pixel jitter value and light transmittance corresponding to image data collected by the unmanned aerial vehicle; judging whether the image data collected by the unmanned aerial vehicle is in a rainy day according to the high-frequency information in the image data, determining that the part of the high-frequency information in the image data with a sparsity greater than a preset sparsity threshold is rain noise, and determining rain noise intensity according to the proportion of the rain noise in the image data; and judging whether the unmanned aerial vehicle is in a windy day according to the pixel jitter value in the image, determining that the part of the pixel jitter value greater than a preset jitter threshold is wind noise, and determining wind noise intensity according to the proportion of the wind noise in the image data; and judging whether the light in the environment where the unmanned aerial vehicle is located is good according to the light transmittance in the image, determining that the part of the light transmittance outside a preset light transmittance interval is light noise, and determining light noise intensity according to the proportion of the light noise in the image data; adjusting a noise reduction parameter in a preset noise reduction algorithm according to the noise type and the noise intensity, and determining the adjusted preset noise reduction algorithm as a target noise reduction algorithm; or selecting a target noise reduction algorithm from a selectable noise reduction algorithm according to the noise type and the noise intensity; performing noise reduction processing on the image data based on the target noise reduction algorithm; wherein adjusting the noise reduction parameter in the preset noise reduction algorithm according to the noise type and the noise intensity comprises at least one of the following: when the noise type is rain noise, determining a high-frequency information filling value of the image data according to the rain noise intensity corresponding to the rain noise, so as to improve the sparsity of the high-frequency information in the image data based on the high-frequency information filling value; when the noise type is wind noise, determining a pixel stability value of the image data according to the wind noise intensity corresponding to the wind noise, so as to reduce the pixel jitter degree in the image data based on the pixel stability value; when the noise type is light noise, determining a target signal-to-noise ratio, a target contrast and / or a target visibility of the image data according to the light noise intensity corresponding to the light noise, so as to improve the light transmittance of the image data based on the target signal-to-noise ratio, the target contrast and / or the target visibility.

2. The method of claim 1, wherein, The step of determining high-frequency information, pixel jitter value and light transmittance corresponding to image data collected by the unmanned aerial vehicle comprises: extracting the high-frequency information in the image data based on a preset image high-frequency information extraction algorithm; and determining the pixel jitter value according to the coordinate change amount between the pixel coordinate sets of the same pixel region between at least two continuously periodically collected image data, and / or extracting feature points between at least two continuously periodically collected image data, and determining the pixel jitter value according to the feature change amount between the feature points; and obtaining a signal-to-noise ratio, a contrast and / or a visibility of the image data, and determining the light transmittance according to the signal-to-noise ratio, the contrast and / or the visibility.

3. The method of claim 1, wherein, The target denoising algorithm includes a defogging algorithm, and the step of performing denoising processing on the image data based on the target denoising algorithm includes: performing denoising processing on the image data based on the defogging algorithm to obtain defogging image data; after the step of performing denoising processing on the image data based on the target denoising algorithm, the method further includes: determining a visible boundary value, a gradient mean value, and / or a saturated pixel point in the defogging image data, and determining a defogging result of the defogging image data according to the visible boundary value, the gradient mean value, and / or the saturated pixel point, and when the defogging result meets a preset condition, controlling the unmanned aerial vehicle to perform an inspection task based on the defogging image data; or, determining a mean square error, a peak signal-to-noise ratio, and / or a structural similarity between the defogging image data and the image data, and determining a defogging threshold value according to the mean square error, the peak signal-to-noise ratio, and / or the structural similarity, and when the defogging threshold value is greater than or equal to a defogging threshold value, controlling the unmanned aerial vehicle to perform an inspection task based on the defogging image data.

4. The method of claim 1, wherein, The unmanned aerial vehicle includes a laser sensor, and after the step of performing denoising processing on the image data based on the target denoising algorithm, the method further includes: acquiring point cloud data collected by the laser sensor at multiple historical time points, wherein the data collection object of the point cloud data and the image data is the same; based on the point cloud data, determining an overlapping area in the denoised image data that meets a preset geometric registration relationship; performing data fusion on the point cloud data and the image data in the overlapping area to obtain fusion data, so as to identify the data collection object according to the fusion data.

5. The method of claim 1, wherein, Before the step of performing denoising processing on the image data based on the target denoising algorithm, the method further includes: acquiring an image size of the image data; determining a target filter window size corresponding to the image data according to the image size; based on the target filter window size, performing smoothing processing on the image data by a dark channel prior defogging algorithm.

6. A control terminal, characterized by comprising: The control terminal includes a memory, a processor, and a program for processing unmanned aerial vehicle environment sensing data stored on the memory and executable on the processor, and when the program for processing unmanned aerial vehicle environment sensing data is executed by the processor, the steps of the method for processing unmanned aerial vehicle environment sensing data according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for processing unmanned aerial vehicle environment sensing data, and when the program for processing unmanned aerial vehicle environment sensing data is executed by the processor, the steps of the method for processing unmanned aerial vehicle environment sensing data according to any one of claims 1 to 5 are implemented.

Images