Low-altitude unmanned aerial vehicle monitoring method and system based on image processing

Abstract

The invention relates to the technical field of image data processing, in particular to a low-altitude unmanned aerial vehicle monitoring method and system based on image processing, and the method comprises the steps: obtaining continuous frame images shot by a low-altitude unmanned aerial vehicle at each time point and corresponding angular velocities; identifying the images of which the angular velocities are greater than a set threshold value as target images, and constructing a target image set; for the pixel points in each target image, calculating a mean value of displacement changes between the pixel points and the corresponding pixel points in the adjacent image, and taking the mean value as a displacement change rate; calculating the mutation degree according to the difference between the pixel point and the neighborhood gray average value of the pixel point; obtaining the abnormal degree of the pixel in combination with frequency domain analysis; and classifying pixels in the target image by integrating the three types of features, and performing enhancement processing on the image by adopting different correction modes according to a classification result. The problem that the image processing efficiency and the image content restoration accuracy are not high is solved.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology. More specifically, the present invention relates to a low-altitude unmanned aerial vehicle monitoring method and system based on image processing. Background Art

[0002] With the gradual opening of low-altitude airspace and the rapid development of drone technology, low-altitude drones have been widely used in many fields, such as power inspection, agricultural plant protection, environmental monitoring, security patrols, traffic monitoring, and disaster investigation. In particular, low-altitude drones have become a crucial information collection and processing tool in scenarios such as smart cities, intelligent transportation, and border security. However, in practical applications, low-altitude drones generally face problems such as image blur, unclear target recognition, severe image jitter interference, and distorted monitoring images. These problems are mainly caused by factors such as high-speed movement, sudden turns, and complex airflow disturbances, which seriously affect the stability and accuracy of drone image monitoring systems.

[0003] Traditional image stabilization and restoration methods primarily rely on hardware compensation techniques (such as three-axis gimbal stabilization), image registration algorithms, image deblurring algorithms, or deep learning image enhancement techniques. However, hardware compensation devices often increase the weight and power consumption of drones, limiting their deployment capabilities on small, lightweight UAV platforms. Existing image processing algorithms, such as single-frame blind deblurring and registration-based multi-frame image fusion, are insufficient in complex environments such as those with large angular velocity variations, strong noise interference, and continuous object occlusion, easily leading to misjudgments or loss of image details.

[0004] Although some methods for image blur analysis have attempted to analyze the degree of blur through image gradients, edge amplitudes, or frequency domain features, thereby achieving image quality classification and deblurring, these methods generally fail to comprehensively consider the characteristics of image grayscale mutations, the stability of neighborhood textures, and the dynamic changes in the image frequency domain energy structure. This results in weak robustness in blur recognition and an inability to effectively adapt to complex dynamic scenes with overlapping blur sources. This, in turn, leads to a lack of prior constraints for image correction, affecting image processing efficiency and the accuracy of image content restoration. Summary of the Invention

[0005] In order to solve the problem in the above background technology that image correction lacks prior constraints, which affects the image processing efficiency and the accuracy of image content restoration, the present invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a low-altitude UAV monitoring method based on image processing, comprising: obtaining continuous frame images and angular velocities taken by a low-altitude UAV at various time points; determining an angular velocity greater than a set angular velocity threshold as a target image to obtain a target image set; taking the average of the displacement changes between any pixel point of any target image in the target image set and the corresponding pixel point in a set adjacent target image as the displacement change rate of any pixel point; obtaining the degree of mutation of any pixel point, wherein the degree of mutation is inversely correlated with the grayscale mean of all pixel points in a set neighborhood window centered on the any pixel point, and is positively correlated with the grayscale mean of any pixel point; obtaining the degree of abnormality of any pixel point, classifying the target image based on the displacement change rate, degree of mutation and degree of abnormality of all pixel points in the target image, and correcting the target image based on the classification result.

[0007] The above technical solution classifies the target image based on the displacement change rate, mutation degree and abnormality degree of all pixels in the target image, and takes targeted correction measures for different categories, thereby significantly improving the clarity, stability and processing accuracy of the overall image, effectively enhancing the intelligent perception and post-processing capabilities of low-altitude UAV images, and is particularly suitable for image quality optimization in scenes with intense motion.

[0008] Furthermore, the mutation degree r i , Where H i is the grayscale mean of all pixels in the set neighborhood window centered on the i-th pixel, h i is the grayscale mean of the i-th pixel, and α is a preset hyperparameter.

[0009] Furthermore, the abnormality degree is specifically as follows: Fourier transform is performed on the area of the set neighborhood window centered on any pixel point to obtain a frequency domain image; the frequency domain image is centralized and divided into a high-frequency area and a low-frequency area by a preset high-pass mask; the sum of the squares of the amplitudes of all frequency domain coefficients in the high-frequency area is taken as the high-frequency energy, the high-frequency energy is normalized, and then the sum is added with a preset value and the inverse is taken to obtain the abnormality degree.

[0010] The above technical solution extracts the frequency domain information of the image by performing Fourier transform on the neighborhood window centered on any pixel point, and uses centralization processing combined with high-pass masking to effectively divide the high-frequency area and the low-frequency area, so that the high-frequency features such as edges and textures in the image are highlighted. The high-frequency energy is calculated by summing the amplitude squares of the frequency domain coefficients in the high-frequency area, and after normalization, a preset value is introduced to sum and then take the inverse. This not only enhances the sensitivity to local texture details, but also suppresses the instability caused by the minimum value at the numerical level. The final abnormality degree can more accurately reflect the abnormal fluctuations or distorted areas of the texture structure in the image, which helps to improve the robustness and accuracy of image blur detection and classification.

[0011] Furthermore, the target image is classified as follows: if the displacement change rate, mutation degree and abnormality degree are all greater than or equal to or less than a set threshold, the average of the displacement change rate, mutation degree and abnormality degree is used as the fuzzy coefficient; if any one of the displacement change rate, mutation degree and abnormality degree is greater than the set threshold, the parameter greater than the set threshold is used as the fuzzy coefficient; if two of the displacement change rate, mutation degree and abnormality degree are greater than the set threshold, the average of the two parameters greater than the set threshold is used as the fuzzy coefficient; the fuzzy coefficient of each pixel point in the target image is clustered to obtain multiple clusters, each cluster corresponding to a category.

[0012] This technical solution comprehensively considers the displacement change rate, degree of mutation, and degree of anomaly of each pixel, and dynamically generates a fuzzy coefficient based on the relationship between these parameters and a set threshold. This makes the judgment of blur level more detailed and flexible, accurately reflecting the differences in motion blur, edge changes, and texture structure between different regions in the image. Furthermore, by clustering the fuzzy coefficients of all pixels, the spatial distribution of blurred features in the image is systematically divided, resulting in clear classification of blurred regions. This facilitates the hierarchical processing of subsequent inpainting methods and improves the accuracy and efficiency of image reconstruction.

[0013] Furthermore, the clustering adopts K-means clustering algorithm.

[0014] Furthermore, the classification results include: first-level blur, second-level blur and third-level blur; the correction of the target image is specifically: if it is first-level blur, the Laplacian operator and unsharp mask are used for correction; if it is second-level blur, the deformable convolutional neural network is used for correction; if it is third-level blur, the physical motion model inversion and GAN are used for correction.

[0015] The above technical solution divides the target image into three levels according to the degree of blur and adopts differentiated correction strategies for different levels of blur types. It can achieve more accurate and efficient image quality restoration, improve the targetedness and intelligence level of the image deblurring process, and demonstrate stronger robustness and image restoration capabilities in complex dynamic scenes.

[0016] Furthermore, the displacement change rate, mutation degree and abnormality degree are normalized.

[0017] The above technical solution can unify image features of different dimensions and numerical ranges onto a comparable scale by normalizing the displacement change rate, mutation degree and abnormality degree, thereby avoiding classification bias or correction misjudgment caused by differences in eigenvalue distribution, improving the accuracy and stability of subsequent fuzzy coefficient calculation and cluster analysis, making the judgment of the blur degree of each pixel in the image more reasonable, further enhancing the robustness of fuzzy classification and the targetedness of the repair strategy, and effectively improving the overall intelligence level and consistency of image quality correction.

[0018] Furthermore, a high-speed camera is used to obtain continuous frame images taken by the low-altitude UAV at various time points.

[0019] Furthermore, the angular velocity is acquired using a gyroscope.

[0020] In a second aspect, the present invention provides a low-altitude UAV monitoring system based on image processing, comprising a memory and a processor, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, any one of the above-described low-altitude UAV monitoring methods based on image processing is implemented.

[0021] The beneficial effects of the present invention are:

[0022] By integrating image processing with flight status information, the present invention achieves accurate identification and targeted repair of blurred areas in low-altitude drone images, effectively improving the stability of image quality under conditions of high-speed motion or drastic posture changes. Fuzzy information is extracted by comprehensively utilizing displacement changes, image grayscale mutations, and frequency domain features, and clustering algorithms are combined to achieve classification. Different repair strategies are matched according to the fuzziness level to further improve image clarity and detail recovery capabilities. The feature fusion effect is enhanced through normalization processing, and dynamic analysis is supported by high-speed camera and gyro data, achieving a higher level of automation and intelligence, significantly enhancing the reliability and practicality of drone monitoring images in complex motion environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart schematically illustrating a low-altitude UAV monitoring method based on image processing according to an embodiment of the present invention;

[0024] Figure 2 Schematically illustrates a structural block diagram of a low-altitude UAV monitoring system based on image processing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] An embodiment of a low-altitude drone monitoring method based on image processing.

[0026] like Figure 1 As shown in FIG, a flow chart of a low-altitude UAV monitoring method based on image processing according to an embodiment of the present invention includes the following steps:

[0027] S1: Obtain continuous frame images and angular velocities taken by a low-altitude UAV at each time point; determine that the images with angular velocities greater than a set angular velocity threshold are target images to obtain a target image set.

[0028] In one embodiment, a high-speed camera can be used to capture images of a low-altitude drone at various points in flight to obtain a continuous frame image sequence. High-speed cameras, with their high frame rate and high resolution, can effectively capture the changes in image detail produced by low-altitude drones during high-speed or intense maneuvers, ensuring image continuity and dynamic integrity, and providing a high-quality data foundation for subsequent image analysis.

[0029] The system also uses a high-precision gyroscope to collect real-time angular velocity information from low-altitude drones during flight. As a key dynamic parameter reflecting a drone's rotational range and maneuverability, angular velocity can be used to accurately identify whether a drone is undergoing drastic attitude changes. The collected angular velocity is compared with a preset angular velocity threshold. If the current angular velocity value exceeds the threshold, the drone may be turning, pitching, rolling, or undergoing other high-speed motions, potentially causing blurring, jitter, and distortion in the image.

[0030] Furthermore, images with angular velocities greater than a set angular velocity threshold are identified as target images, and all images meeting these criteria form the target image set. This approach effectively screens keyframes most susceptible to image degradation and serves as the focus of subsequent image processing, quality assessment, and blur restoration. This avoids redundant computations when the image is stable, improving overall algorithm efficiency and resource utilization.

[0031] S2: Calculate the displacement change, mutation degree and abnormality degree of any pixel point of any target image in the target image set.

[0032] In one embodiment, the displacement change rate of any pixel in any target image in the target image set is calculated as the average of the displacement changes between the corresponding pixel in a set of adjacent target images. The displacement change rate of each pixel in the target image is obtained by calculating the average of the displacement changes in adjacent image frames. This effectively reflects the motion trend and stability of the pixel in consecutive frames, thereby capturing dynamic changes in the image caused by factors such as drone movement, jitter, or image blur. Compared to methods based solely on single-frame image feature extraction, this solution improves the ability to depict image change patterns through information fusion across multiple frames, making the recognition of unclear information such as blurred areas and fast-moving areas more accurate.

[0033] In one embodiment, the mutation degree r of the i-th pixel is calculated i , Where H i is the grayscale mean of all pixels in the set neighborhood window centered on the i-th pixel, h i is the grayscale mean of the i-th pixel, α is a preset hyperparameter; the neighborhood window is set to 5×5;

[0034] The degree of abnormality is specifically determined by performing a Fourier transform on a region within a predetermined neighborhood window centered on any pixel to obtain a frequency domain image; centering the frequency domain image and dividing it into a high-frequency region and a low-frequency region using a preset high-pass mask; and calculating the sum of the squared amplitudes of all frequency domain coefficients in the high-frequency region as the high-frequency energy. This high-frequency energy is normalized, summed with a preset value, and the inverse is taken to obtain the degree of abnormality. The preset value is 0.01, but can be set to any desired value.

[0035] By performing Fourier transform on the neighborhood area centered on any pixel point, its characteristic distribution in the frequency domain is obtained, effectively introducing the frequency information of the local texture and detail changes of the image. The introduction of centralization processing and high-pass mask makes it possible to highlight the areas representing high-frequency features in the frequency domain image, thereby focusing on important information such as edges, details and mutation structures. Calculating the sum of the squares of the amplitudes of the frequency domain coefficients in the high-frequency area as high-frequency energy helps to fully reflect the texture complexity and change intensity of the local area. Normalizing the high-frequency energy and summing it with the preset value and then taking the inverse, not only avoids the risk of instability caused by the denominator being zero, but also enhances the ability to compare the numerical values of the abnormality levels of different image areas. The final abnormality level index has strong sensitivity and stability, and can more accurately reflect areas with potential blur, distortion or motion abnormalities in the image.

[0036] S3: Classify the target image based on the displacement change rate, mutation degree and abnormality degree of all pixels in the target image.

[0037] In one embodiment, the displacement change rate, mutation degree and abnormality degree are normalized. Exemplarily, maximum-minimum normalization may be used.

[0038] The target image is classified as follows: if the displacement change rate, mutation degree and abnormality degree are all greater than or equal to or less than the set threshold, the average of the displacement change rate, mutation degree and abnormality degree is used as the fuzzy coefficient; if any one of the displacement change rate, mutation degree and abnormality degree is greater than the set threshold, the parameter greater than the set threshold is used as the fuzzy coefficient; if two of the displacement change rate, mutation degree and abnormality degree are greater than the set threshold, the average of the two parameters greater than the set threshold is used as the fuzzy coefficient;

[0039] By comprehensively considering the displacement change rate, degree of mutation, and degree of anomaly of each pixel in the image, the blur level is flexibly determined and the blur coefficient is calculated accordingly, achieving precise quantification of the image blur state. This approach effectively distinguishes different blur types and degrees in complex image degradation scenarios, avoiding misclassification issues caused by single-metric judgments and improving classification accuracy and robustness. Furthermore, the dynamic generation of the blur coefficient makes subsequent clustering and correction processing more targeted, helping to improve the efficiency and effectiveness of overall image clarity restoration.

[0040] Clustering the fuzzy coefficients of each pixel in the target image to obtain multiple clusters, each cluster corresponding to a category. The clustering adopts the K-means clustering algorithm;

[0041] S4: Correct the target image based on the classification results.

[0042] In one embodiment, the classification results include: first-level blur, second-level blur and third-level blur; the target image is corrected based on the classification results, specifically: if it is first-level blur, the Laplacian operator and unsharp mask are used for correction; if it is second-level blur, the deformable convolutional neural network is used for correction; if it is third-level blur, the physical motion model inversion and GAN are used for correction.

[0043] By classifying the target image into multiple levels based on blur and applying differentiated image correction methods for each level, more refined and adaptable image quality optimization is achieved. First-level blurred images can be restored to clarity using traditional edge enhancement operators and sharpening methods. Second-level blurred images incorporate a deep learning model to handle more complex blur features. Third-level blurred images utilize a generative adversarial network combined with physical motion laws for deep reconstruction, improving image restoration capabilities even under extreme blur conditions.

[0044] The solution of the present invention combines continuous drone images, angular velocity information and multiple image feature calculation methods to achieve accurate identification and classification of blurred areas in images, and adopts differentiated image restoration strategies based on the degree of blur. It can effectively improve the clarity and usability of images obtained by low-altitude drones during high-speed flight or rapid turning, enhance the stable monitoring capability of key target areas, and reduce the interference of image blur on subsequent identification, tracking and monitoring tasks.

[0045] Example of low-altitude drone monitoring system based on image processing:

[0046] like Figure 2 As shown in FIG, a structural block diagram of a low-altitude UAV monitoring system based on image processing according to an embodiment of the present invention includes a processor and a memory.

[0047] The present invention also provides a low-altitude UAV monitoring system based on image processing. Figure 2 As shown, the system includes a processor and a memory, and the memory stores computer program instructions. When the computer program instructions are executed by the processor, the low-altitude UAV monitoring method based on image processing according to the present invention is implemented.

[0048] The image processing-based low-altitude UAV monitoring system also includes other components familiar to those skilled in the art, such as a communication interface. The settings and functions of these components are known in the art and will not be described in detail here.

[0049] In the present invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, the computer-readable storage medium can be any suitable magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc., or any other medium that can be used to store the required information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible or connectable to a device. Any application or module described in the present invention can be implemented using computer-readable / executable instructions stored or otherwise retained by such a computer-readable medium.

[0050] In the description of this specification, "multiple" and "several" mean at least two, such as two, three or more, unless otherwise clearly defined.

[0051] While several embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous modifications, variations, and alternatives will occur to those skilled in the art without departing from the concept and spirit of the present invention. It should be understood that various alternatives to the embodiments of the present invention described herein may be employed in practicing the present invention.

Claims

1. A low-altitude UAV monitoring method based on image processing, characterized in that: include: Obtain continuous frame images and angular velocity taken by low-altitude UAV at each time point; Determine the image with an angular velocity greater than a set angular velocity threshold as a target image to obtain a target image set; The average of the displacement changes between any pixel point of any target image in the target image set and the corresponding pixel point in the set adjacent target image is used as the displacement change rate of the any pixel point; Obtaining a mutation degree of any pixel point, wherein the mutation degree is negatively correlated with the grayscale mean of all pixels within a set neighborhood window centered on the any pixel point, and positively correlated with the grayscale mean of the any pixel point; The abnormality degree of any pixel point is obtained, the target image is classified based on the displacement change rate, mutation degree and abnormality degree of all pixels in the target image, and the target image is corrected based on the classification result.

2. The low-altitude UAV monitoring method based on image processing according to claim 1 is characterized in that: The mutation degree r i , Where H i is the grayscale mean of all pixels in the set neighborhood window centered on the i-th pixel, h i is the grayscale mean of the i-th pixel, and α is a preset hyperparameter.

3. The low-altitude UAV monitoring method based on image processing according to claim 1 is characterized in that: The degree of abnormality is specifically: Performing Fourier transform on the area of the set neighborhood window centered at any pixel point to obtain a frequency domain image; Performing centralization processing on the frequency domain image, and dividing the frequency domain image into a high-frequency region and a low-frequency region by using a preset high-pass mask; The sum of the squares of the amplitudes of all frequency domain coefficients in the high-frequency area is taken as high-frequency energy, the high-frequency energy is normalized, and then the sum is added to a preset value and the inverse is taken to obtain the abnormality degree.

4. The low-altitude UAV monitoring method based on image processing according to claim 1 is characterized in that: The target image is classified as follows: If the displacement change rate, mutation degree and abnormality degree are all greater than or equal to or less than the set threshold, the average of the displacement change rate, mutation degree and abnormality degree is used as the fuzzy coefficient; if any one of the displacement change rate, mutation degree and abnormality degree is greater than the set threshold, the parameter greater than the set threshold is used as the fuzzy coefficient; if two of the displacement change rate, mutation degree and abnormality degree are greater than the set threshold, the average of the two parameters greater than the set threshold is used as the fuzzy coefficient; The fuzzy coefficients of each pixel in the target image are clustered to obtain multiple clusters, each of which corresponds to a category.

5. The low-altitude UAV monitoring method based on image processing according to claim 1 is characterized in that: The clustering adopts K-means clustering algorithm.

6. The low-altitude UAV monitoring method based on image processing according to claim 1 is characterized in that: The classification results include: first-level fuzzy, second-level fuzzy and third-level fuzzy; The target image is corrected as follows: if it is a first-level blur, the Laplacian operator and the unsharp mask are used for correction; if it is a second-level blur, the deformable convolutional neural network is used for correction; if it is a third-level blur, the physical motion model inversion and GAN are used for correction.

7. The low-altitude UAV monitoring method based on image processing according to claim 1 is characterized in that: The displacement change rate, mutation degree and abnormality degree are normalized.

8. The low-altitude UAV monitoring method based on image processing according to claim 1 is characterized in that: A high-speed camera is used to obtain continuous frame images taken by the low-altitude UAV at various time points.

9. The low-altitude UAV monitoring method based on image processing according to claim 1 is characterized in that: The angular velocity is obtained using a gyroscope.

10. The low-altitude UAV monitoring system based on image processing is characterized by: The invention comprises a memory and a processor, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the low-altitude UAV monitoring method based on image processing according to any one of claims 1 to 9 is implemented.

Images