Unmanned aerial vehicle crop condition monitoring system and method under multi-spectral image fusion
By using multispectral image fusion technology, agricultural monitoring images were collected and stitched together using drones, which solved the problems of radiation distortion and spatial registration error in drone-based agricultural monitoring systems, and achieved high-precision and real-time agricultural condition identification and visualization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INNER MONGOLIA UNIV OF TECH
- Filing Date
- 2025-09-16
- Publication Date
- 2026-05-12
AI Technical Summary
Existing UAV agricultural monitoring systems suffer from insufficient accuracy and poor real-time performance due to radiation distortion and spatial registration errors during multispectral data fusion. Furthermore, the automation level of multispectral image processing is low, and the visualization of agricultural information is not intuitive.
The UAV agricultural monitoring system and method based on multispectral image fusion uses a multispectral camera to collect remote sensing images of the target agricultural area, introduces a radiometric correction mechanism for preprocessing, and stitches the images together with geographic coordinates to generate an agricultural monitoring map, thereby realizing visualized agricultural monitoring.
It improves the spatiotemporal resolution and accuracy of agricultural condition identification in multispectral visual detection, solves the problems of insufficient accuracy and poor real-time performance in agricultural condition monitoring, and provides intuitive guidance for agricultural condition decision-making.
Smart Images
Figure CN121191027B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual inspection technology, specifically to an unmanned aerial vehicle (UAV) agricultural monitoring system and method based on multispectral image fusion. Background Technology
[0002] For a long time, agricultural monitoring has mainly relied on manual inspections or satellite remote sensing. The former is inefficient and unable to cover large areas of farmland, while the latter, although capable of large-scale monitoring, suffers from insufficient spatiotemporal resolution and high data acquisition costs. In recent years, the rapid development of drone technology has provided a new technological path for agricultural monitoring. In particular, the combination of multispectral imaging technology and drones has enabled the acquisition of farmland information to break through the spatiotemporal limitations of traditional monitoring methods. However, existing drone-based agricultural monitoring systems still face technical bottlenecks such as insufficient fusion of multi-source data, low automation in image processing, and unintuitive visualization of agricultural information. In terms of data acquisition, conventional monospectral or RGB images cannot fully reflect the physiological state of crops; in terms of data processing, key technologies such as radiometric correction, geometric registration, and temporal analysis of multispectral images still need optimization; and in terms of information application, how to transform professional remote sensing information into intuitive and easy-to-understand agricultural decision-making guidance is an urgent problem to be solved in the field of agricultural remote sensing. Summary of the Invention
[0003] This application provides a UAV agricultural monitoring system and method based on multispectral image fusion, aiming to solve the technical problems of insufficient accuracy and poor real-time performance of traditional visual inspection technology in multispectral data fusion due to radiometric distortion and spatial registration errors. It achieves the technical effect of improving the spatiotemporal resolution of multispectral visual inspection and enhancing the accuracy of agricultural condition identification by using multispectral image radiometric correction and accurate geographic coordinate stitching.
[0004] In view of the above problems, this application provides a UAV agricultural monitoring system and method based on multispectral image fusion.
[0005] The first aspect disclosed in this application provides a UAV agricultural monitoring system based on multispectral image fusion. This system includes: an image acquisition module for acquiring a target remote sensing image set of a target agricultural area using a multispectral camera mounted on a UAV under the constraints of a predetermined flight strategy; an image extraction module for extracting a first remote sensing image from the target remote sensing image set, wherein the first remote sensing image corresponds to a first geographic coordinate; an image preprocessing module for preprocessing the first remote sensing image using a radiometric correction mechanism to obtain a first target image; an image stitching module for stitching the first target image using the first geographic coordinate as a stitching reference to obtain a target agricultural monitoring map of the target agricultural area; and an agricultural visualization monitoring module for performing agricultural visualization monitoring of the target agricultural area based on the target agricultural monitoring map.
[0006] Another aspect of this application discloses a method for UAV agricultural monitoring using multispectral image fusion. The method includes: acquiring a target remote sensing image set of a target agricultural area using a multispectral camera mounted on a UAV under the constraints of a predetermined flight strategy; extracting a first remote sensing image from the target remote sensing image set, wherein the first remote sensing image corresponds to a first geographic coordinate; preprocessing the first remote sensing image using a radiometric correction mechanism to obtain a first target image; stitching the first target image using the first geographic coordinate as a stitching reference to obtain a target agricultural monitoring map of the target agricultural area; and performing agricultural condition visualization monitoring of the target agricultural area based on the target agricultural monitoring map.
[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:
[0008] In the image acquisition module, under the constraints of a predetermined flight strategy, a set of target remote sensing images of the target agricultural area is acquired using a multispectral camera mounted on a drone. In the image extraction module, a first remote sensing image is extracted from the target remote sensing image set, where the first remote sensing image corresponds to a first geographic coordinate. In the image preprocessing module, a radiometric correction mechanism is introduced to preprocess the first remote sensing image to obtain a first target image. In the image stitching module, the first target image is stitched together using the first geographic coordinate as the stitching reference to obtain a target agricultural condition monitoring map of the target agricultural area. In the agricultural condition visualization monitoring module, agricultural condition visualization monitoring of the target agricultural area is performed based on the target agricultural condition monitoring map. This solves the technical problems of insufficient accuracy and poor real-time performance in agricultural condition monitoring caused by radiometric distortion and spatial registration errors in traditional visual inspection technologies during multispectral data fusion. It achieves the technical effect of improving the spatiotemporal resolution of multispectral visual inspection and enhancing the accuracy of agricultural condition identification through accurate stitching of multispectral image radiometric correction and geographic coordinates.
[0009] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0010] Figure 1 This application provides a schematic diagram of a UAV agricultural monitoring system based on multispectral image fusion.
[0011] Figure 2 This application provides a flowchart illustrating a method for monitoring agricultural conditions using unmanned aerial vehicles (UAVs) based on multispectral image fusion.
[0012] Figure labeling: Image acquisition module 11, Image extraction module 12, Image preprocessing module 13, Image stitching module 14, Agricultural condition visualization monitoring module 15. Detailed Implementation
[0013] This application provides a UAV agricultural monitoring system and method based on multispectral image fusion, which solves the technical problems of insufficient accuracy and poor real-time performance of traditional visual inspection technology due to radiometric distortion and spatial registration errors when fusion of multispectral data. It achieves the technical effect of improving the spatiotemporal resolution of multispectral visual inspection and enhancing the accuracy of agricultural condition identification by using multispectral image radiometric correction and accurate stitching of geographic coordinates.
[0014] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.
[0015] Example 1, as Figure 1 As shown in the embodiment of this application, a UAV agricultural monitoring system based on multispectral image fusion is provided. The system includes:
[0016] The image acquisition module 11 is used to acquire a set of remote sensing images of the target agricultural area by using a multispectral camera mounted on a UAV under the constraints of a predetermined flight strategy.
[0017] Specifically, in the image acquisition module 11, under the guidance of a prescribed flight strategy, a drone (such as a DJI Mavic 3 multispectral drone) carrying a multispectral camera conducts aerial patrols over the target agricultural area. This flight strategy includes predetermined parameters such as flight altitude, flight path, and speed, ensuring that the drone can cover the entire monitoring area and guaranteeing the uniformity and accuracy of data acquisition. Through the multispectral camera, the drone acquires remote sensing image data from different spectral bands. This image data reflects the reflectance characteristics of crops in different spectral bands, providing basic data for subsequent analyses such as crop health assessment. These acquired remote sensing images form a target remote sensing image set, containing various information about the target agricultural area, serving as the basis for further processing and analysis.
[0018] Furthermore, the predetermined flight strategy includes a predetermined flight altitude and a predetermined flight route, and the predetermined flight altitude includes a first altitude and a second altitude, wherein the first altitude is a vertical height of 40 meters and the second altitude is a vertical height of 120 meters.
[0019] In a preferred embodiment, the predetermined flight strategy includes pre-set flight altitude and flight path. Two flight altitudes are available: the first is a vertical altitude of 40 meters, typically suitable for lower flight requirements, enabling detailed acquisition of ground and crop data; the second is a vertical altitude of 120 meters, suitable for large-area coverage missions, allowing for rapid data collection over a larger area. These different flight altitude settings allow the UAV to be flexibly adjusted according to specific needs, ensuring both high-resolution detailed data and effective coverage over a wide area. The flight path is pre-planned based on existing navigation systems (such as GPS, automatic flight control systems, GIS, etc.) to ensure the UAV covers the entire target agricultural area and effectively collects the required remote sensing data.
[0020] Furthermore, the image acquisition module includes:
[0021] Remote sensing video of the target is acquired by the multispectral camera; lens detection and segmentation are performed on the remote sensing video of the target to obtain target segmentation results; key images of the first segmented video segment in the target segmentation results are selected as first images; and the target remote sensing image set is constructed based on the first images.
[0022] In a preferred embodiment, a multispectral camera mounted on a drone captures aerial images of the target agricultural area, obtaining remote sensing video of the area. This video contains spectral information reflected from different wavelengths, providing data on crop growth status, soil characteristics, and other relevant information. Subsequently, by comparing the displacement index between adjacent image frames, the acquired remote sensing video is subjected to lens detection and segmentation to determine different scenes or perspectives within the video. When a change in scene is detected, i.e., the displacement index exceeds a predetermined threshold, the system segments the video based on these changes, forming a target segmentation result. This target segmentation result includes multiple segmented video clips. Subsequently, in each segmented video fragment, the quality metrics of each image frame of the first segmented video fragment are statistically analyzed, including image sharpness, signal-to-noise ratio (SNR), contrast, and brightness. Sharpness can be measured by image sharpness. For example, the Laplacian variance method measures sharpness by calculating the variation of each pixel in the image. The sharper the image, the more obvious the edge changes, and the larger the variance value. Blurry images have smaller variance values. Therefore, after calculating the Laplacian operator for the image, the variance of the result can be calculated to measure the image sharpness. The SNR can be determined by calculating the mean square error between the image and its noise component. A higher SNR indicates better image quality. Contrast can be determined by the brightness distribution of the image. For example, calculating the histogram of the image's grayscale values and then calculating the standard deviation of the histogram reflects the dispersion of grayscale values in the image. A larger standard deviation indicates higher image contrast. Brightness can be determined by calculating the brightness value of each pixel in the image. Brightness is usually the weighted average of the image's RGB values or the pixel values of a grayscale image. A higher average brightness value indicates a brighter image, and a lower average brightness value indicates a darker image. Typically, the value closest to the ideal brightness range is chosen. Then, based on these quality indicators, the highest quality image frame from the first segmented video segment is selected. This image frame is the key image for that segment and can serve as the first image for that segment, participating in the construction of the subsequent target remote sensing image set. The same process is repeated for other segmented video segments, resulting in multiple images. Finally, the selected first image and other images are added to a set to obtain the target remote sensing image set. This target remote sensing image set will be further processed and analyzed for subsequent crop health assessments and other agricultural diagnostic work, ensuring the overall effectiveness of agricultural monitoring.
[0023] Furthermore, the target remote sensing video is subjected to shot detection and segmentation to obtain the target segmentation result, including:
[0024] Extract a first image frame from the target remote sensing video, and the first image frame has a first adjacent image frame; compare the first image frame with the first adjacent image frame to obtain a first displacement index; if the first displacement index reaches a predetermined threshold, then segment the first image frame and the first adjacent image frame to obtain the target segmentation result.
[0025] In one feasible implementation, a first image frame is extracted from the target remote sensing video. This first image frame, the first frame of the video, serves as a representative image of the current monitoring point. Simultaneously, adjacent image frames are extracted, typically the next frame in the video (i.e., the second adjacent frame), for inter-frame comparison. Then, an appropriate feature point extraction algorithm is selected, such as SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Feature Transform), or ORB (Oriented Fast and Rotated BRIEF). The descriptor of each feature point, such as the texture and color of the surrounding area, is obtained using the selected algorithm. The similarity between the feature point descriptors in the first image frame and those in the first adjacent image frame is then measured using methods such as Euclidean distance, Hamming distance, or cosine similarity. The feature point pair with the smallest distance is selected as the matching point, or a matching threshold (such as minimum distance) is set to filter out high-quality matching pairs. Afterward, for each pair of matching points, the coordinates of each pair are obtained, and the displacement of each pair is calculated using Euclidean distance. The displacements of all matching points are then averaged to obtain a first displacement index. Then, the calculated first displacement index is compared with a predetermined displacement index threshold. This predetermined displacement index threshold is a pre-set standard used to determine whether the changes between image frames are sufficiently significant. When the displacement index is greater than or equal to the threshold, it indicates that there is sufficient change between the first image frame and its first adjacent image frame. In this case, the first image frame is separated from its first adjacent image frame. After the segmentation of the first image frame and its first adjacent image frame is completed, the first adjacent image frame is used as the second image frame, and the above operation is repeated until all image frames have been compared. Finally, the segmented target remote sensing video is summarized to obtain the target segmentation result. Through this process, significantly changing image regions can be effectively identified from continuous video frames, and image frames with large changes can be segmented, thereby improving the processing efficiency of image data and ensuring that key changes can be accurately captured under different crop growth states or environmental conditions, thus providing high-quality image data for agricultural monitoring.
[0026] The image extraction module 12 is used to extract the first remote sensing image from the target remote sensing image set, wherein the first remote sensing image corresponds to the first geographic coordinates.
[0027] Specifically, in the image extraction module 12, an image is first selected from the collected target remote sensing image set as the first remote sensing image. This image is the first image extracted from multiple remote sensing images, representing the image data of an agricultural area at a certain time or location. The first remote sensing image contains the first geographic coordinates, as well as the crop and environmental information at the location corresponding to the first geographic coordinates, which can provide a reliable spatial reference for subsequent analysis.
[0028] The image preprocessing module 13 is used to introduce a radiometric correction mechanism to preprocess the first remote sensing image to obtain the first target image.
[0029] Specifically, in the image preprocessing module 13, a radiometric correction mechanism is introduced to preprocess the first remote sensing image to ensure the accuracy and comparability of the remote sensing image data. The purpose of radiometric correction is to eliminate deviations in image radiometric values caused by sensor performance, weather conditions, or other external factors, making the spectral information of the image more accurately reflect the actual situation of the target ground. In this process, the system uses a predetermined reference image as a reference and performs radiometric correction on each pixel in the first remote sensing image according to a histogram matching algorithm, thereby obtaining the first target image. This first target image has higher accuracy and can provide a more reliable data foundation for subsequent data analysis and agricultural monitoring.
[0030] Furthermore, the image preprocessing module includes:
[0031] According to the radiometric correction mechanism, the first remote sensing image is converted from the RGB color space to the HSL color space to obtain the first converted image; the first converted image is analyzed to determine a predetermined reference image; using the predetermined reference image as a reference, the first remote sensing image is radiometrically corrected using a histogram matching algorithm to obtain the first target image.
[0032] In a preferred embodiment, during radiometric correction, the first remote sensing image is first converted from the RGB color space (i.e., red, green, and blue channels) to the HSL color space (i.e., hue, saturation, and brightness) to obtain the first converted image. In the RGB color space, colors are represented by three independent color channels, while the HSL color space describes colors through hue, saturation, and brightness components, allowing for better separation of color and illumination information. The conversion process is based on the RGB-to-HSL conversion formula. This conversion effectively reduces the impact of different lighting conditions on color, making the image colors more stable and accurate, facilitating subsequent radiometric correction. Subsequently, the converted first image is analyzed to extract key features, such as brightness features. These extracted features are then used as constraints to filter the target remote sensing image set, determining a reference image and defining it as a predetermined baseline image. Subsequently, using a predetermined reference image, a histogram matching algorithm is employed to perform radiometric correction on the first remote sensing image. The core of this algorithm is to compare the grayscale distribution (i.e., the brightness value distribution of pixels) of the images and adjust the histogram of the input image to be as close as possible to the histogram of the reference image. In this way, the brightness and contrast in the image can be adjusted, eliminating radiometric deviations caused by lighting conditions, sensor differences, or other environmental factors, making the corrected image more consistent with the true reflective characteristics of ground features. The image obtained after histogram matching is the first target image. This image, having undergone radiometric correction, accurately reflects the reflective characteristics of the target area, eliminating unnecessary noise and lighting errors, ensuring that subsequent analysis can be based on more accurate and standardized image data.
[0033] Further, analyzing the first transformed image to determine the predetermined reference image includes:
[0034] A first pixel set of the first converted image is constructed, and a first color feature of the first pixel in the first pixel set is obtained; the average value of the first brightness value in the first color feature is taken as the first target brightness of the first remote sensing image; the target remote sensing image set is filtered and statistically analyzed with the first target brightness as a constraint to obtain the number of first images corresponding to the first target brightness; the brightness list of the first target brightness is obtained by sorting the first image number in descending order; any remote sensing image corresponding to the first brightness in the brightness list is taken as the predetermined reference image.
[0035] In one feasible implementation, all pixels are extracted from the first converted image to form a first pixel set. For each pixel, its color features, including brightness features, are extracted. These brightness features reflect the light intensity information of each pixel in the image. After obtaining the brightness features of all pixels in the first pixel set, the average brightness value of all pixels is calculated as the first target brightness. This average value represents a standard value for the overall brightness in the image, reflecting the overall illumination situation. This value is used for subsequent screening and calibration to ensure that subsequent image processing and analysis are performed according to a unified brightness standard. Subsequently, the entire target remote sensing image set is screened using the first target brightness as a constraint. Specifically, the system checks the brightness value of each image in the image set and filters out images with a brightness greater than or equal to the first target brightness, thus obtaining the number of images corresponding to the first target brightness. These images have a high degree of conformity with the target brightness and are suitable for subsequent analysis. Subsequently, the selected images are sorted in descending order of brightness to obtain a brightness list of the first target brightness. Then, the remote sensing image corresponding to the brightness of the first image is taken from the brightness list as the predetermined reference image. This image represents an ideal brightness standard and serves as a reference image for subsequent radiometric correction. The reference image has high brightness consistency and standardization characteristics, and can be used as the benchmark data for subsequent analysis to improve the accuracy and consistency of remote sensing image analysis.
[0036] Furthermore, using the predetermined reference image as a reference, the first remote sensing image is radiometrically corrected using a histogram matching algorithm to obtain the first target image, including:
[0037] Obtain a reference histogram of the predetermined reference image; obtain a first histogram of the first remote sensing image; compare the reference histogram and the first histogram to obtain a first matching degree; perform correction processing on the first remote sensing image with the first matching degree reaching a predetermined matching degree limit as the target, specifically including: sequentially curveforming the reference histogram and the first histogram to obtain a reference curve and a first curve respectively; calculating the first Fletcher distance between the reference curve and the first curve; and using the first Fletcher distance to represent the first matching degree.
[0038] In one feasible implementation, a predetermined reference image is analyzed at the pixel level. The image typically consists of multiple pixels, each with a specific brightness value (a single value in a grayscale image, and brightness values for three channels: red, green, and blue in a color image). By traversing each pixel in the image, the brightness value of that pixel is obtained, and the number of pixels with the same brightness value is accumulated and recorded in the corresponding brightness level, thereby extracting the reference histogram of the predetermined reference image. This reference histogram reflects the ideal image brightness distribution. Similarly, a first histogram is extracted from the first remote sensing image to be corrected. This first histogram represents the pixel distribution of each brightness level in the image and is used for comparison with the reference image. Subsequently, the reference histogram and the first histogram are compared, and their matching degree is calculated. Specifically, for the reference histogram and the first histogram, the cumulative frequency corresponding to each brightness value is obtained by accumulating the pixel frequency of each brightness level with all frequencies lower than that level. These cumulative frequencies form a gradually increasing curve, thus obtaining the reference curve and the first curve. These two curves reflect the distribution pattern of each brightness value in the corresponding image. Next, for each pair of corresponding points between the baseline curve and the first curve, their difference is calculated, i.e., the absolute difference in the frequency values of the two curves at that point. Then, the first Fretcher distance is obtained by calculating the square root of the sum of the squares of all difference values. The smaller this first Fretcher distance, the smaller the difference between the baseline curve and the first curve, and the higher the degree of matching. Conversely, the larger the difference between the two curves, the lower the degree of matching, and further correction is needed. Then, the calculated first Fretcher distance is used as the first matching degree and compared with a predetermined matching degree limit to determine whether it meets the predetermined matching degree limit (i.e., whether it is less than or equal to the predetermined matching degree limit). If it does not meet this predetermined matching degree limit, it indicates a significant difference between the first remote sensing image and the predetermined baseline image. In this case, the brightness of all pixels can be shifted by adding or subtracting a constant value to adjust the overall brightness of the image until an ideal match is achieved. After correction is completed, i.e., when the first matching degree meets the predetermined matching degree limit, the current first remote sensing image is output as the first target image. Through this process, the system can accurately correct the brightness distribution of the first remote sensing image, making it as consistent as possible with the standard brightness distribution of the predetermined reference image. This ensures the consistency of image brightness between different images, enhances the comparability of images, and provides stable basic data for subsequent agricultural analysis and image processing.
[0039] The image stitching module 14 is used to stitch the first target image with the first geographic coordinates as the stitching reference to obtain the target agricultural monitoring map of the target agricultural area.
[0040] Specifically, in the image stitching module 14, a first geographic coordinate is used as the stitching reference. This coordinate corresponds to the geographical location of the image, ensuring the accuracy of the spatial information of the stitched image. Based on the first geographic coordinate, the system aligns multiple images, starting with the first target image, ensuring their correct positioning in geographic space. Subsequently, the images are precisely stitched together according to their geographic coordinates, eliminating edge overlaps or misalignments. During this process, the images are transformed and registered using their spatial characteristics, ensuring a perfect geographical match. Finally, after stitching, a complete agricultural monitoring map of the target agricultural area is obtained. This map displays the overall agricultural situation of the area, containing multi-dimensional remote sensing data from different locations and time points, providing a comprehensive perspective and analytical basis for farmland management.
[0041] The agricultural condition visualization monitoring module 15 is used to perform agricultural condition visualization monitoring on the target agricultural area based on the target agricultural condition monitoring map.
[0042] Specifically, in the agricultural condition visualization monitoring module 15, after the target agricultural condition monitoring map is stitched together, the system performs agricultural condition visualization monitoring on the target agricultural area based on this map. Visual monitoring typically includes annotations of different colors, icons, or layers. Each level of vegetation index will have a corresponding color or graphic label, allowing agricultural managers to clearly see which areas require special attention. This process transforms complex remote sensing data into easily understandable visual information, supporting agricultural managers to make quick and accurate decisions, such as adjusting irrigation and fertilization, thereby improving the efficiency and accuracy of agricultural production.
[0043] Furthermore, the agricultural condition visualization monitoring module also includes:
[0044] The ratio of the difference between the reflectance of the near-infrared band and the red band to their sum is calculated and denoted as the normalized vegetation index (NDI). The ratio of the reflectance of the near-infrared band to the red band is calculated and denoted as the ratio vegetation index. The difference between the reflectance of the near-infrared band and the red band is calculated and denoted as the difference vegetation index. A soil brightness adjustment factor is introduced, and the reflectance of the near-infrared band and the red band are combined to obtain the soil-regulated vegetation index. The mean of the normalized vegetation index, the ratio vegetation index, the difference vegetation index, and the soil-regulated vegetation index is taken as the target vegetation index. The target vegetation index is marked on the target agricultural monitoring map.
[0045] In a preferred embodiment, the Normalized Difference Vegetation Index (NDVI) is obtained by calculating the ratio of the difference between the reflectance of the near-infrared (NIR) band and the red (R) band to their sum. This NDVI measures plant health and reflects the chlorophyll content of the plant. The index ranges from -1 to 1, with values closer to 1 indicating healthier plants. The Ratio Vegetation Index (RVI) is obtained by calculating the ratio of the reflectance of the near-infrared band to that of the red band. This RVI is used to assess plant health, especially when the reflectance value is relatively high; a higher RVI value generally indicates higher plant cover and better health. The Difference Vegetation Index (DVI) is obtained by calculating the difference between the reflectance of the near-infrared band and the red band. This DVI is another simple vegetation index suitable for preliminary assessment of vegetation growth status; a larger DVI value generally indicates better plant growth. To reduce the influence of soil background on vegetation indices, a soil brightness adjustment factor is introduced. This factor is typically set to 0.5. It is calculated by subtracting the reflectance difference between the near-infrared and red bands, summing the reflectance of the near-infrared and red bands to the soil brightness adjustment factor, then dividing the reflectance difference by the sum, and multiplying the result by 1 plus the soil brightness adjustment factor to obtain the Soil-Adjusted Vegetation Index (SAVI). This SAVI more accurately reflects plant health and reduces interference from soil reflection. After obtaining the Normalized Difference Vegetation Index (NDI), Ratio Vegetation Index (RDI), Difference Vegetation Index (DDI), and SAVI, the mean of these four indices is used as the target vegetation index. This target vegetation index integrates the characteristics of multiple vegetation indices, enabling a more comprehensive assessment of the agricultural conditions in the target area. Finally, the calculated target vegetation indices are graphically plotted on the target crop monitoring map. Different colors or shades represent different vegetation index values, allowing agricultural managers to visually observe the health status of vegetation in different areas. For example, a higher target vegetation index might be marked in green, indicating healthy vegetation in the area; while a lower target vegetation index might be marked in yellow or red, indicating potential problems such as water shortage or pests. Through this process, the system can generate accurate target vegetation indices, providing more comprehensive and detailed crop monitoring results. This helps agricultural managers monitor crop health in real time, allocate resources rationally, and implement effective agricultural management measures, thereby improving agricultural production efficiency and crop quality.
[0046] In summary, the UAV agricultural monitoring system based on multispectral image fusion provided in this application has the following technical effects:
[0047] The system comprises: an image acquisition module 11, used to acquire a target remote sensing image set of the target agricultural area using a multispectral camera mounted on a UAV under the constraints of a predetermined flight strategy; an image extraction module 12, used to extract a first remote sensing image from the target remote sensing image set, wherein the first remote sensing image corresponds to a first geographic coordinate; an image preprocessing module 13, used to preprocess the first remote sensing image by introducing a radiometric correction mechanism to obtain a first target image; an image stitching module 14, used to stitch the first target image using the first geographic coordinate as the stitching reference to obtain a target agricultural condition monitoring map of the target agricultural area; and an agricultural condition visualization monitoring module 15, used to perform agricultural condition visualization monitoring of the target agricultural area based on the target agricultural condition monitoring map. Through these steps, the system solves the technical problems of insufficient accuracy and poor real-time performance in agricultural condition monitoring caused by radiometric distortion and spatial registration errors in traditional visual inspection technologies during multispectral data fusion. It achieves the technical effect of improving the spatiotemporal resolution of multispectral visual inspection and enhancing the accuracy of agricultural condition identification through accurate stitching of multispectral image radiometric correction and geographic coordinates.
[0048] Example 2, based on the same inventive concept as the UAV agricultural monitoring system under multispectral image fusion in the previous examples, such as... Figure 2 As shown in the embodiments of this application, a method for monitoring agricultural conditions using unmanned aerial vehicles (UAVs) based on multispectral image fusion is provided. This method includes:
[0049] Under the constraints of a predetermined flight strategy, a set of target remote sensing images of the target agricultural area is acquired by a multispectral camera mounted on a drone; a first remote sensing image is extracted from the target remote sensing image set, wherein the first remote sensing image corresponds to a first geographic coordinate; a radiometric correction mechanism is introduced to preprocess the first remote sensing image to obtain a first target image; the first target image is stitched together using the first geographic coordinate as a stitching reference to obtain a target agricultural condition monitoring map of the target agricultural area; and the target agricultural condition is visualized and monitored in the target agricultural area based on the target agricultural condition monitoring map.
[0050] Furthermore, the method includes:
[0051] The predetermined flight strategy includes a predetermined flight altitude and a predetermined flight route, and the predetermined flight altitude includes a first altitude and a second altitude, wherein the first altitude is a vertical height of 40 meters and the second altitude is a vertical height of 120 meters.
[0052] Furthermore, the method includes:
[0053] Remote sensing video of the target is acquired by the multispectral camera; lens detection and segmentation are performed on the remote sensing video of the target to obtain target segmentation results; key images of the first segmented video segment in the target segmentation results are selected as first images; and the target remote sensing image set is constructed based on the first images.
[0054] Furthermore, the method includes:
[0055] Extract a first image frame from the target remote sensing video, and the first image frame has a first adjacent image frame; compare the first image frame with the first adjacent image frame to obtain a first displacement index; if the first displacement index reaches a predetermined threshold, then segment the first image frame and the first adjacent image frame to obtain the target segmentation result.
[0056] Furthermore, the method includes:
[0057] According to the radiometric correction mechanism, the first remote sensing image is converted from the RGB color space to the HSL color space to obtain the first converted image; the first converted image is analyzed to determine a predetermined reference image; using the predetermined reference image as a reference, the first remote sensing image is radiometrically corrected using a histogram matching algorithm to obtain the first target image.
[0058] Furthermore, the method includes:
[0059] A first pixel set of the first converted image is constructed, and a first color feature of the first pixel in the first pixel set is obtained; the average value of the first brightness value in the first color feature is taken as the first target brightness of the first remote sensing image; the target remote sensing image set is filtered and statistically analyzed with the first target brightness as a constraint to obtain the number of first images corresponding to the first target brightness; the brightness list of the first target brightness is obtained by sorting the first image number in descending order; any remote sensing image corresponding to the first brightness in the brightness list is taken as the predetermined reference image.
[0060] Furthermore, the method includes:
[0061] Obtain a reference histogram of the predetermined reference image; obtain a first histogram of the first remote sensing image; compare the reference histogram and the first histogram to obtain a first matching degree; perform correction processing on the first remote sensing image with the first matching degree reaching a predetermined matching degree limit as the target, specifically including: sequentially curveforming the reference histogram and the first histogram to obtain a reference curve and a first curve respectively; calculating the first Fletcher distance between the reference curve and the first curve; and using the first Fletcher distance to represent the first matching degree.
[0062] Furthermore, the method includes:
[0063] The ratio of the difference between the reflectance of the near-infrared band and the red band to their sum is calculated and denoted as the normalized vegetation index (NDI). The ratio of the reflectance of the near-infrared band to the red band is calculated and denoted as the ratio vegetation index. The difference between the reflectance of the near-infrared band and the red band is calculated and denoted as the difference vegetation index. A soil brightness adjustment factor is introduced, and the reflectance of the near-infrared band and the red band are combined to obtain the soil-regulated vegetation index. The mean of the normalized vegetation index, the ratio vegetation index, the difference vegetation index, and the soil-regulated vegetation index is taken as the target vegetation index. The target vegetation index is marked on the target agricultural monitoring map.
[0064] In summary, any step of the method described above can be stored as a computer instruction or program in an unrestricted computer memory, and can be called and identified by an unrestricted computer processor to implement any method in the embodiments of this application, without any additional restrictions.
[0065] Furthermore, the "first" or "second" mentioned above may not only represent a sequential relationship, but may also represent a specific concept, and / or refer to the individual or collective selection of multiple elements. Clearly, those skilled in the art can make various modifications and variations to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.
Claims
1. A UAV agricultural monitoring system based on multispectral image fusion, characterized in that, include: The image acquisition module is used to acquire a set of remote sensing images of the target agricultural area using a multispectral camera mounted on a drone, under the constraints of a predetermined flight strategy. The image extraction module is used to extract the first remote sensing image from the target remote sensing image set, wherein the first remote sensing image corresponds to the first geographic coordinates; The image preprocessing module is used to preprocess the first remote sensing image by introducing a radiometric correction mechanism to obtain the first target image; The image stitching module is used to stitch the first target image with the first geographic coordinates as the stitching reference to obtain the target agricultural monitoring map of the target agricultural area. The agricultural condition visualization monitoring module is used to perform agricultural condition visualization monitoring of the target agricultural area based on the target agricultural condition monitoring map. The image preprocessing module includes: According to the radiometric correction mechanism, the first remote sensing image is converted from the RGB color space to the HSL color space to obtain the first converted image; Analyze the first converted image to determine the predetermined reference image; Using the predetermined reference image as a reference, the first remote sensing image is radiometrically corrected using a histogram matching algorithm to obtain the first target image; Analyzing the first transformed image to determine the predetermined reference image includes: Construct a first pixel set of the first converted image, and obtain the first color feature of the first pixel in the first pixel set; The average value of the first brightness value in the first color feature is taken as the first target brightness of the first remote sensing image; The target remote sensing image set is filtered and statistically analyzed using the first target brightness as a constraint to obtain the first number of images corresponding to the first target brightness. A brightness list of the first target brightness is obtained by sorting the first image count in descending order; Take any remote sensing image corresponding to the first brightness in the brightness list as the predetermined reference image; Using the predetermined reference image as a reference, the first remote sensing image is radiometrically corrected using a histogram matching algorithm to obtain the first target image, including: Obtain the reference histogram of the predetermined reference image; Obtain the first histogram of the first remote sensing image; The first matching degree between the baseline histogram and the first histogram is obtained by comparison; The correction processing of the first remote sensing image is performed with the goal of achieving a predetermined matching degree limit, specifically including: The baseline histogram and the first histogram are sequentially curve-transformed to obtain the baseline curve and the first curve, respectively; The first Frazer distance between the reference curve and the first curve is calculated; The first matching degree is represented by the first Fraser distance.
2. The UAV agricultural monitoring system based on multispectral image fusion as described in claim 1, characterized in that, The predetermined flight strategy includes a predetermined flight altitude and a predetermined flight route, and the predetermined flight altitude includes a first altitude and a second altitude, wherein the first altitude is a vertical height of 40 meters and the second altitude is a vertical height of 120 meters.
3. The UAV agricultural monitoring system based on multispectral image fusion as described in claim 1, characterized in that, The image acquisition module includes: The target remote sensing video was acquired using the multispectral camera; The target remote sensing video is subjected to lens detection and segmentation to obtain the target segmentation result; Select key images from the first segmented video segment in the target segmentation result as the first image; The target remote sensing image set is constructed based on the first image.
4. The UAV agricultural monitoring system based on multispectral image fusion as described in claim 3, characterized in that, The target remote sensing video is subjected to shot detection and segmentation to obtain the target segmentation result, including: Extract the first image frame from the target remote sensing video, and the first image frame has a first adjacent image frame; The first displacement index is obtained by comparing the first image frame with the first adjacent image frame; If the first displacement index reaches a predetermined threshold, the first image frame and the first adjacent image frame are segmented to obtain the target segmentation result.
5. The UAV agricultural monitoring system based on multispectral image fusion as described in claim 1, characterized in that, The agricultural condition visualization monitoring module also includes: The ratio of the difference in reflectance between the near-infrared band and the red band to their sum is denoted as the normalized vegetation index. Calculate the ratio of reflectance in the near-infrared band to that in the red band, and denot it as the ratio vegetation index; The difference in reflectance between the near-infrared band and the red band is calculated and denoted as the difference vegetation index. By introducing a soil brightness adjustment factor and coordinating near-infrared reflectance and red reflectance, a soil-regulated vegetation index was obtained. The average of the normalized vegetation index, the ratio vegetation index, the difference vegetation index, and the soil-regulated vegetation index is taken as the target vegetation index. The target vegetation index is marked on the target agricultural monitoring map.
6. A method for monitoring agricultural conditions using unmanned aerial vehicles (UAVs) based on multispectral image fusion, characterized in that, The method is executed by the UAV agricultural monitoring system based on multispectral image fusion as described in any one of claims 1 to 5, and includes: Under the constraints of a predetermined flight strategy, a set of remote sensing images of the target agricultural area is acquired by using a multispectral camera mounted on a drone. Extract the first remote sensing image from the target remote sensing image set, wherein the first remote sensing image corresponds to the first geographic coordinates; A radiometric correction mechanism is introduced to preprocess the first remote sensing image to obtain the first target image; Using the first geographic coordinates as the stitching reference, the first target image is stitched together to obtain the target agricultural monitoring map of the target agricultural area. The target agricultural area is monitored visually based on the target agricultural condition monitoring map.