A vegetation detection method and system based on a UAV

Abstract

The application provides a kind of vegetation detection method and system based on unmanned aerial vehicle, belongs to unmanned aerial vehicle vegetation detection technical field, its method includes: through unmanned aerial vehicle carries remote sensing, high definition, infrared multi-source equipment, obtains target area initial image and fuses to obtain complete image;Complete image is carried out cluster analysis, and after layering cluster chart, the cluster boundary line is adjusted based on boundary common point, and the image after adjustment is obtained;The vegetation features and confidence of the image after adjustment are extracted, a feature-confidence matrix is constructed, and the abnormal sub-region is screened and the abnormal factor is determined by linear fitting and trend synchronous analysis, the reference image is called to generate a replacement image to complete abnormal replacement, and the detection result is output.The application integrates multi-source equipment information, optimizes cluster boundary and abnormal detection logic, improves the comprehensiveness, accuracy and reliability of vegetation detection, and is suitable for ecological monitoring, agricultural management and other scenes.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) vegetation detection technology, and in particular to a UAV-based vegetation detection method and system. Background Technology

[0002] With the increasing demand for vegetation monitoring in fields such as ecological environment monitoring and agricultural production management, traditional vegetation monitoring methods, such as manual field surveys and single-device remote sensing monitoring, are no longer sufficient to meet the requirements of accurate and efficient monitoring. Manual field surveys are time-consuming and labor-intensive, have limited coverage, and are greatly affected by factors such as terrain and climate, making it impossible to achieve large-scale rapid detection. Single-device remote sensing monitoring can only obtain partial information about vegetation. For example, ordinary high-definition equipment can only obtain information about the appearance and morphology of vegetation, and infrared equipment can only reflect the temperature information of vegetation. Although remote sensing equipment can obtain multispectral information, its resolution is limited, resulting in incomplete image information and thus affecting the accuracy of the detection results.

[0003] Therefore, this invention proposes a vegetation detection method and system based on unmanned aerial vehicles (UAVs). Summary of the Invention

[0004] This invention provides a vegetation detection method and system based on unmanned aerial vehicles (UAVs) to solve the aforementioned technical problems.

[0005] This invention provides a vegetation detection method based on unmanned aerial vehicles (UAVs), comprising: Step 1: Acquire several initial images of the target area using the multi-source devices carried by the UAV, and fuse all the initial images from the same source devices to obtain a complete image. The multi-source devices include: remote sensing devices, high-definition devices, and infrared devices. Step 2: Perform cluster analysis on each complete image, stack all cluster maps to lock the cluster boundary lines, and adjust the cluster boundary lines according to the common points of the boundary lines to obtain the adjusted image; Step 3: Perform feature extraction and anomaly analysis on the adjusted image, retrieve reference images from all initial images for anomaly replacement, obtain the detection results of the target region, and output them.

[0006] Preferably, step 2 includes: Determine the feature complexity of each pixel in each complete image, and perform cluster analysis on all pixels in the corresponding complete image based on the feature complexity and pixel value of each pixel to obtain the cluster map of each complete image. All clustering diagrams are stacked, with any one clustering diagram as the first diagram and the remaining clustering diagrams as the second diagram. The clustering ambiguity vectors of the boundary points of each clustering boundary line in the first diagram and the corresponding points in each second diagram are extracted. The boundary points are then arranged in the order of the clustering boundary lines to construct the confusion matrix of the corresponding clustering boundary lines. Extract the self-multidimensional matrix of the boundary shared points in the first image and the single row vector of the corresponding boundary shared points acting on each second image, and construct a multidimensional auxiliary matrix based on the single row vector. The number of dimensions of the self-multidimensional matrix is ​​the frequency of the corresponding boundary shared points being used, and the number of dimensions of the multidimensional auxiliary matrix is ​​the number of second images. Based on the confusion matrix of the cluster boundary line and the self-multidimensional matrix and multidimensional auxiliary matrix of the boundary common points in the confusion matrix, the corresponding boundary points are pre-analyzed and the adjustment line of the cluster boundary line is obtained, resulting in the third figure. Using the aforementioned adjustment line as a reference, each second figure is adjusted to obtain the corresponding fourth figure; The third and fourth images are the adjusted images.

[0007] Preferably, step 3 includes: The vegetation features and feature confidence scores of each sub-region enclosed by the adjusted boundary line are extracted from the third image and each fourth image to construct the feature-confidence matrix of the target region. Fitting analysis is performed on each column vector of the feature-confidence matrix, and trend synchronization analysis is performed on the first row vector and each of the remaining row vectors to screen out abnormal sub-regions and determine the abnormal factors of the abnormal sub-regions. The region number of the abnormal sub-region is locked, and based on the abnormal factor and the device type of the first and second images, the number of initial images with the same region code captured from each source device is determined and the retrieved images are regarded as reference images. A new image is obtained by aligning the pixels of the reference image of each source device, and a replacement image is obtained by comprehensive analysis based on the new images of all source devices. Then, the vegetation features of the corresponding sub-region are replaced based on the replacement features of the replacement image.

[0008] Preferably, clustering analysis is performed on all pixels in the corresponding complete image based on the feature complexity and pixel value of each pixel, including: Select a preset number of neighboring pixels around the corresponding pixel, calculate the Pearson correlation coefficient between the feature complexity of the corresponding pixel and each neighboring pixel, take the mean of all correlation coefficients as the neighborhood feature complexity correlation degree, and use the neighborhood feature complexity correlation degree as a correction factor to perform weighted correction on the feature complexity of the corresponding pixel to obtain the corrected complexity. Based on the pixel value distribution of the complete image, it is divided into dense vegetation sub-regions, sparse vegetation sub-regions, and non-vegetation sub-regions. The corrected complexity of each pixel and the regional weight of its sub-region are used as joint features. Density clustering algorithm is used for clustering, and the neighborhood radius of the cluster is adjusted according to the correlation of the neighborhood feature complexity, so as to obtain the clustering map of each complete image.

[0009] Preferably, pre-analyzing the corresponding boundary points and obtaining the adjustment lines for the cluster boundary lines includes: Extract the deviation sequence based on feature complexity and the mapping difference sequence based on pixel value of the same boundary point in the complete images of different devices, and combine the device attributes of the multi-source devices carried by the UAV to determine the cross-device correlation coefficient; When the boundary point is the intersection of vegetation and non-vegetation, a nonlinear regression model is constructed to calculate the adjustment offset using the ambiguity vector of the confusion matrix, the cross-device correlation coefficient, and the frequency of its own multidimensional matrix as input. When the boundary point is the intersection of different vegetation types, the decision tree model is constructed to calculate the adjustment offset by taking the ambiguity vector of the confusion matrix, the cross-device correlation coefficient, and the row vector of the multidimensional auxiliary matrix as input. The adjustment offsets of all boundary points are arranged according to the topological order of the cluster boundary lines. The actual geographical contour of the target area is used as a constraint to perform geometric correction on the arranged offsets. The adjustment line of the cluster boundary lines is obtained by fitting a Bézier curve.

[0010] Preferably, the process of screening abnormal sub-regions and determining the abnormal factors of abnormal sub-regions includes: Determine the confidence interval range for each column vector, match the fitting strategy for the corresponding column vector from the range-strategy lookup table, and perform fitting processing on the corresponding column vector based on the fitting strategy to obtain the fitting residual of the corresponding column vector. If the fitting residual exceeds n1 times the fitting standard deviation of the corresponding interval, then the sub-region of the corresponding column vector is marked as the first candidate sub-region. Taking the first row vector of the feature-confidence matrix as a reference, calculate the trend synchronization coefficient with each of the remaining row vectors, and correct the corresponding trend synchronization coefficient with the average confidence of each row vector as the weight to obtain the weighted synchronization coefficient. If the weighted synchronization coefficient is lower than a preset threshold, the corresponding sub-region is marked as the second candidate sub-region; The intersection of the first candidate sub-region and the second candidate sub-region is used to obtain the abnormal sub-region; For each anomalous sub-region, the vegetation feature type of the column with the largest residual in the corresponding feature-confidence matrix is ​​extracted. Triples are constructed by combining the deviation direction of the fitting residual and the deviation degree of the weighted synchronization coefficient, which serve as the anomalous factor for the corresponding anomalous sub-region.

[0011] Preferably, a replacement image is obtained by comprehensive analysis of new images from all source devices, including: Based on different vegetation feature types, match the corresponding target feature association channels in new images from the same source device; Predict the importance priority of each target feature associated channel, generate the channel priority coefficient of each channel, and predict the vegetation spatial focus distribution of each new image in the target feature associated channel to generate the feature space priority map of the corresponding new image; perform local brightness extraction on each target feature associated channel to obtain the local brightness distribution features in the channel, and perform light detection on the target feature associated channel to identify the vegetation sub-region labels covered by different light types in the channel and the proportion of the corresponding sub-regions in the channel, and calculate the brightness confidence probability of the corresponding channel. Calculate the mean and variance of the significant values ​​of the associated channel priorities of all target features, and correct the original first weight based on the channel priority coefficient in combination with the brightness confidence probability to obtain the brightness-adapted channel priority weight. Based on the feature space priority map and the brightness confidence probability, the original second weight based on the feature space priority map is corrected to obtain the feature space weight after brightness adaptation. The final integrated weight is obtained by weighting and fusing the channel priority weight and feature space weight with the overall confidence of each new image as the base weight and the degree of deviation in the anomaly factor. Based on the final integrated weight, the target feature association channels of the new images from all source devices are weighted and fused to obtain the desired image. The pixel gradient difference between the desired image and the surrounding normal sub-regions is calculated, and the Gaussian fusion algorithm is used to perform gradient transition processing on the edge regions of the initial image to obtain the replacement image.

[0012] This invention provides a vegetation detection system based on unmanned aerial vehicles (UAVs), comprising: The fusion processing module is used to acquire several initial images of the target area based on the multi-source devices carried by the UAV, and to fuse all the initial images from the same source device to obtain a complete image. The multi-source devices include: remote sensing devices, high-definition devices, and infrared devices. The clustering and adjustment module is used to perform clustering analysis on each complete image, stack all cluster maps to lock the cluster boundary lines, and adjust the cluster boundary lines according to the common points of the boundary lines to obtain the adjusted image. The anomaly replacement module is used to extract features and perform anomaly analysis on the adjusted image, retrieve reference images from all initial images for anomaly replacement, obtain the detection results of the target region, and output them.

[0013] Compared with the prior art, the beneficial effects of this application are as follows: Initial images of the target area are collected by drones equipped with remote sensing, high-definition, and infrared multi-source devices. The images are then fused from the same source devices to obtain a comprehensive and complete image. The image region division is then optimized through cluster analysis and boundary adjustment. Finally, feature extraction, anomaly analysis, and reference image replacement are combined to achieve accurate detection, which significantly improves the comprehensiveness, accuracy, and reliability of vegetation detection and provides high-quality data support for subsequent vegetation management and decision-making.

[0014] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0015] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a vegetation detection method based on a drone in an embodiment of the present invention; Figure 2 This is a structural diagram of a vegetation detection system based on a drone, according to an embodiment of the present invention. Detailed Implementation

[0017] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0018] This invention provides a vegetation detection method based on unmanned aerial vehicles (UAVs), such as... Figure 1 As shown, it includes: Step 1: Acquire several initial images of the target area using the multi-source devices carried by the UAV, and fuse all the initial images from the same source devices to obtain a complete image. The multi-source devices include: remote sensing devices, high-definition devices, and infrared devices. Step 2: Perform cluster analysis on each complete image, stack all cluster maps to lock the cluster boundary lines, and adjust the cluster boundary lines according to the common points of the boundary lines to obtain the adjusted image; Step 3: Perform feature extraction and anomaly analysis on the adjusted image, retrieve reference images from all initial images for anomaly replacement, obtain the detection results of the target region, and output them.

[0019] In this embodiment, the multi-source device refers to a collection of various devices mounted on a drone for collecting different types of image information of the target area. These devices are fixed to the underside of the drone using a customized bracket. The remote sensing device's shooting interval is set to 0.5 seconds, the high-definition device's shooting interval to 0.3 seconds, and the infrared device's shooting interval to 1 second. The flight altitude is set to 50-200 meters depending on the detection area, and the flight speed is controlled between 3-8 m / s. The remote sensing device uses an MS600 multispectral remote sensing camera, which can acquire multispectral images within the 40-900nm spectral range, reflecting physiological information such as chlorophyll content and water content of vegetation. The high-definition device uses a 4K high-definition industrial camera with a resolution of 48 million pixels to clearly capture details such as the appearance and leaf texture of vegetation. The infrared device uses an infrared thermal imaging camera with a spectral range of 8-14μm to acquire temperature distribution information of vegetation for determining its growth status.

[0020] In this embodiment, the initial image refers to the original image that is captured in a single shot by a multi-source device during its flight over the target area along a preset route, without any processing.

[0021] In this embodiment, fusion processing refers to the process of integrating several initial images with overlapping areas taken by the same type of equipment (from the same source equipment) into a single image that completely covers the target area through techniques such as feature point matching and image stitching. For example, for 30 initial images taken by a remote sensing device, the SIFT feature point matching algorithm is used to extract feature points (such as vegetation edge points, plot corner points, etc.) from each image, calculate the matching relationship of feature points between different images to obtain a homography matrix, and then perform projection transformation on the images based on the homography matrix to eliminate geometric distortion between images. Finally, a linear fusion algorithm is used to process the pixel values ​​of the overlapping areas to obtain a complete remote sensing image that fully covers 1,000 acres of farmland.

[0022] In this embodiment, a complete image refers to an image that, after fusion processing, completely covers the target area without obvious stitching gaps and contains complete information; it serves as the foundational data for subsequent clustering analysis. For example, after fusion processing, three complete images are obtained: a remote sensing complete image, a high-resolution complete image, and an infrared complete image. The resolution of each image is adjusted to 4096×2160 pixels, clearly presenting the overall distribution, detailed features, and temperature distribution of vegetation within the target area.

[0023] In this embodiment, cluster analysis refers to the process of dividing pixels into different categories based on the feature differences (such as feature complexity, pixel value, etc.) of pixels in an image, so that the feature similarity of pixels within the same category is high, and the feature differences of pixels between different categories are large. Its purpose is to divide different vegetation areas or vegetation and non-vegetation areas.

[0024] In this embodiment, the cluster boundary line refers to the dividing line between pixels of different categories after cluster analysis, used to define the range of different regions. For example, between the tree region and the shrub region obtained by clustering, there is a continuous cluster boundary line. This boundary line accurately delineates the distribution range of trees and shrubs, and the pixels on both sides of the boundary line belong to the tree category and the shrub category, respectively.

[0025] In this embodiment, a shared boundary point refers to a pixel in different clustering maps that is located at the same geographical location and is part of the cluster boundary line when multiple clustering layers are overlaid. This type of point is a key reference point for adjusting the cluster boundary line. For example, when the clustering map of a complete remote sensing image and the clustering map of a high-resolution complete image are overlaid, the pixel with coordinates (1200, 800) is a boundary point of both tree areas and non-vegetation areas in both clustering maps; this pixel is a shared boundary point.

[0026] In this embodiment, feature extraction refers to extracting information that characterizes vegetation attributes from the adjusted image, such as color features, texture features, spectral features, and temperature features, to provide data support for anomaly analysis. For example, multispectral features of vegetation, such as red band reflectance, green band reflectance, and near-infrared band reflectance, are extracted from the adjusted remote sensing image; texture features of vegetation are extracted from the adjusted high-resolution image; and temperature features of vegetation (such as the temperature value of pixels) are extracted from the adjusted infrared image. Specifically, a CNN network is used for feature extraction. The adjusted image is input into the network, and features are extracted and transformed through convolutional layers and pooling layers. The output feature vector of the fully connected layer is used as the vegetation feature, and the dimension of the feature vector is set to 256.

[0027] In this embodiment, anomaly analysis refers to the process of identifying abnormal sub-regions whose vegetation characteristics differ significantly from normal regions by comparing and analyzing the vegetation characteristics and confidence levels of each sub-region, and determining the relevant factors that cause the anomalies, namely, the anomalous factors.

[0028] In this embodiment, the reference image refers to the image retrieved from the initial images taken by the same source device that matches the code of the anomalous sub-region and is used for anomalous replacement. Its selection must be based on the anomalous factor and device type to ensure the reliability of the replacement information. For example, if the anomalous factor of the anomalous sub-region is mainly related to spectral features, and the first image is from a remote sensing device and the second image is from a high-definition device, then 10 initial images taken by the remote sensing device that match the code of the anomalous sub-region will be retrieved as reference images, while 5 initial images from the high-definition device will be retrieved to assist in verification.

[0029] In this embodiment, anomaly replacement refers to the process of generating a replacement image based on a reference image, and replacing the abnormal vegetation features of the abnormal sub-region with the normal vegetation features in the replacement image, so that the detection results more accurately reflect the actual vegetation situation of the target area. For example, the retrieved remote sensing reference image and high-resolution reference image are pixel-aligned and comprehensively analyzed to generate a replacement image containing normal spectral and texture features. The features in this image are then used to replace the pest-related abnormal features of the abnormal sub-region, resulting in a corrected detection result.

[0030] The beneficial effects of the above technical solution are as follows: by using a drone equipped with remote sensing, high-definition, and infrared multi-source equipment to collect initial images of the target area, and then using image fusion of the same source equipment to obtain a comprehensive and complete image, the image region division is optimized through cluster analysis and boundary adjustment, and finally, accurate detection is achieved by combining feature extraction, anomaly analysis and reference image replacement, which significantly improves the comprehensiveness, accuracy and reliability of vegetation detection, and provides high-quality data support for subsequent vegetation management and decision-making.

[0031] This invention provides a vegetation detection method based on unmanned aerial vehicles (UAVs), step 2 of which includes: Determine the feature complexity of each pixel in each complete image, and perform cluster analysis on all pixels in the corresponding complete image based on the feature complexity and pixel value of each pixel to obtain the cluster map of each complete image. All clustering diagrams are stacked, with any one clustering diagram as the first diagram and the remaining clustering diagrams as the second diagram. The clustering ambiguity vectors of the boundary points of each clustering boundary line in the first diagram and the corresponding points in each second diagram are extracted. The boundary points are then arranged in the order of the clustering boundary lines to construct the confusion matrix of the corresponding clustering boundary lines. Extract the self-multidimensional matrix of the boundary shared points in the first image and the single row vector of the corresponding boundary shared points acting on each second image, and construct a multidimensional auxiliary matrix based on the single row vector. The number of dimensions of the self-multidimensional matrix is ​​the frequency of the corresponding boundary shared points being used, and the number of dimensions of the multidimensional auxiliary matrix is ​​the number of second images. Based on the confusion matrix of the cluster boundary line and the self-multidimensional matrix and multidimensional auxiliary matrix of the boundary common points in the confusion matrix, the corresponding boundary points are pre-analyzed and the adjustment line of the cluster boundary line is obtained, resulting in the third figure. Using the aforementioned adjustment line as a reference, each second figure is adjusted to obtain the corresponding fourth figure; The third and fourth images are the adjusted images.

[0032] In this embodiment, when calculating the feature complexity of a pixel, the gray-level co-occurrence matrix algorithm is used. The gray level is set to 256, the distance d=1, and the angle θ is set to 0°, 45°, 90°, and 135° respectively. The entropy value at the four angles is calculated, and the arithmetic mean of the four entropy values ​​is taken as the feature complexity of the pixel.

[0033] In this embodiment, after cluster analysis is completed, a unique color value or identifier code is assigned to each category. All pixels in the image are traversed, and each pixel is assigned a corresponding color or identifier based on its category, generating a cluster map. The resolution of the cluster map is consistent with the complete image to facilitate subsequent layering processing. For example, after performing cluster analysis on a high-resolution complete image, a cluster map is obtained, where tree areas are labeled dark green, shrub areas are labeled light green, herbaceous areas are labeled yellow-green, and non-vegetation areas are labeled gray. The boundary lines of each category are marked with solid black lines.

[0034] In this embodiment, the first image refers to any one clustering image selected from all clustering images as a benchmark. This benchmark is used to extract boundary point information and construct a confusion matrix. The selection can be randomized or based on device priority, such as remote sensing devices having the highest priority. For example, among remote sensing clustering images, high-definition clustering images, and infrared clustering images, the remote sensing clustering image is selected as the first image. Its clustering boundary lines are used as a benchmark for comparative analysis with the other two clustering images.

[0035] In this embodiment, the clustering ambiguity vector refers to a vector used to characterize the degree and type of difference when the cluster category of a boundary point in the first image differs from the cluster category of the corresponding point in the second image. Its dimension is determined by the number of cluster categories. For example, assuming the cluster categories are divided into 4 types: trees, shrubs, herbs, and non-vegetation, and the cluster category of a boundary point in the first image is trees (coded as 1), and the cluster category of the corresponding point in the second image is shrubs (coded as 2), then the clustering ambiguity vector is [0, 1, -1, 0], where the absolute value of the non-zero elements represents the degree of difference, and the sign represents the direction of the category difference.

[0036] In this embodiment, the confusion matrix refers to a matrix constructed by arranging the boundary points in a clockwise order according to the cluster boundary lines of the first graph, with the clustering ambiguity vector of each boundary point as the row vector. It is used to comprehensively reflect the differences in cluster boundaries between the first graph and the second graph. For example, if a certain cluster boundary line in the first graph contains 100 boundary points, corresponding to two second graphs, each second graph corresponds to a confusion matrix. Each confusion matrix is ​​100 rows and 4 columns, where 4 represents the number of cluster categories, and the matrix elements are the clustering ambiguity vector values ​​of each boundary point. This matrix allows for a direct view of which boundary points exhibit clustering differences and the degree of these differences.

[0037] In this embodiment, the self-multidimensional matrix refers to a multidimensional matrix constructed using the feature information of the boundary shared points in the first image, such as feature complexity, pixel value, and cluster category code, as elements. Its dimension number equals the frequency with which the boundary shared point is shared by different second images, and it is used to characterize the self-feature association of the boundary shared points. For example, if a boundary shared point is shared by 3 second images (frequency 3), the feature complexity, pixel value, and cluster category code of that point are selected to construct a 3x3 self-multidimensional matrix. The matrix elements are the feature parameter values ​​of that point corresponding to different second images, i.e.: Each row corresponds to a second image, and each column corresponds to a feature parameter.

[0038] In this embodiment, a single row vector refers to the feature parameter vector corresponding to a boundary common point in a certain second image. It contains information such as the feature complexity, pixel value, and cluster category of the point in the second image and is the basic unit for constructing a multi-dimensional auxiliary matrix. For example, if the feature complexity of a boundary common point in a certain second image is 1.6, the pixel value is 190, the cluster category code is 2, and the shooting angle is 30°, then its single row vector is [1.6, 190, 2, 30°].

[0039] In this embodiment, the multidimensional auxiliary matrix refers to a matrix constructed using the single row vector of the boundary common point in each second graph as the row vector. Its dimension is equal to the number of second graphs. It is used to assist in analyzing the feature differences of the boundary common point in different second graphs and to provide a basis for boundary line adjustment.

[0040] In this embodiment, the adjustment line refers to the new boundary line obtained by offsetting, geometrically correcting, and curve fitting the boundary points of the first cluster boundary line based on data such as the confusion matrix, its own multidimensional matrix, and multidimensional auxiliary matrix. Its accuracy is higher than that of the original cluster boundary line.

[0041] In this embodiment, the third image refers to the adjusted image obtained by replacing the clustering boundary lines with adjustment lines based on the first image. It is the result of boundary optimization of the first image, retaining other image information of the first image and only updating the boundary lines.

[0042] In this embodiment, the fourth image refers to the adjusted image obtained by using the adjustment line as a reference and making auxiliary adjustments to the cluster boundary lines of each second image, so that the cluster boundary lines of the second image are consistent with the adjustment line, improving the consistency of cluster images from different devices. Specifically, the second image and the third image are image registered, the deviation between the original cluster boundary lines of the second image and the adjustment line is calculated, and for boundary points with deviations exceeding the allowable range, offset adjustments are made according to the position of the adjustment line. After adjustment, the category region information and other image features of the second image are retained, and the fourth image is generated. For example, if the second image is a high-definition cluster image, the boundary points in the high-definition cluster image with large deviations from the adjustment line are adjusted based on the adjustment line of the third image, so that the cluster boundary lines of the high-definition cluster image coincide with the adjustment line or the deviation is within the allowable range (≤1 pixel), thus obtaining the fourth image.

[0043] The beneficial effects of the above technical solution are: by refining the clustering analysis process, fully considering the feature complexity and pixel value of the pixels, and constructing a confusion matrix, a self-multidimensional matrix and a multidimensional auxiliary matrix by combining the stacked clustering graph, the third and fourth graphs are obtained for the first and second graphs respectively, realizing the accurate adjustment of the clustering boundary line, laying a solid image foundation for the accurate identification of subsequent abnormal sub-regions, and improving the reliability of the entire vegetation detection process.

[0044] This invention provides a vegetation detection method based on unmanned aerial vehicles (UAVs), step 3 of which includes: The vegetation features and feature confidence scores of each sub-region enclosed by the adjusted boundary line are extracted from the third image and each fourth image to construct the feature-confidence matrix of the target region. Fitting analysis is performed on each column vector of the feature-confidence matrix, and trend synchronization analysis is performed on the first row vector and each of the remaining row vectors to screen out abnormal sub-regions and determine the abnormal factors of the abnormal sub-regions. The region number of the abnormal sub-region is locked, and based on the abnormal factor and the device type of the first and second images, the number of initial images with the same region code captured from each source device is determined and the retrieved images are regarded as reference images. A new image is obtained by aligning the pixels of the reference image of each source device, and a replacement image is obtained by comprehensive analysis based on the new images of all source devices. Then, the vegetation features of the corresponding sub-region are replaced based on the replacement features of the replacement image.

[0045] In this embodiment, the feature-confidence matrix refers to a two-dimensional matrix that records the vegetation features of each sub-region within the target area and their corresponding confidence scores. The rows of the matrix correspond to the adjusted images from different devices, and the columns correspond to each sub-region. Each matrix element contains the vegetation feature vector of the corresponding sub-region in the corresponding row image and the confidence score of the feature extraction result. Assuming there is one third image and two fourth images, and the target area is divided into 15 sub-regions, the feature-confidence matrix is ​​3 rows and 15 columns. The element in the second row and the eighth column is: {[0.3,0.6,0.9,0.4],0.88}, indicating that the vegetation feature vector of the eighth sub-region in the second fourth image is [0.3,0.6,0.9,0.4], and the confidence score of the feature extraction result is 0.88. It should be noted that the confidence score ranges from 0 to 1, and the closer it is to 1, the more reliable the feature extraction. Specifically, the third image and all fourth images are sorted according to device type, the number of rows in the matrix is ​​determined, the target area is divided into several sub-regions, and the size of the sub-regions is set according to the target area range and detection accuracy, such as 10×10 pixels as a sub-region. The number of columns in the matrix is ​​determined, and the vegetation feature vector and confidence score of each sub-region are extracted from each adjusted image. The vegetation feature vector is extracted using the AlexNet network with a dimension of 256, and the confidence score is 1 / (1+exp(loss)), where loss is the cross-entropy loss function value.

[0046] In this embodiment, trend synchronization analysis refers to using the vegetation feature vectors of each sub-region in the first row vector of the feature-confidence matrix (i.e., the vegetation feature vectors of each sub-region in the third image) as a reference benchmark, and calculating the degree of trend synchronization between the vegetation feature vectors of each remaining row vector (i.e., the vegetation feature vectors of each sub-region in the fourth image) and the reference vector. The corrected synchronization coefficient is used to determine whether the vegetation feature trends of the same sub-region in images from different devices are consistent. For example, if the vegetation feature trend corresponding to the first row vector is: red band reflectance gradually decreases and green band reflectance gradually increases, and the trend corresponding to the row vector in a certain fourth image is: red band reflectance gradually decreases and green band reflectance gradually increases, the trend synchronization coefficient between the two is 0.95. After correction by an average confidence level of 0.85, the weighted synchronization coefficient is 0.90, indicating a high degree of trend consistency. Here, the weighted synchronization coefficient = synchronization coefficient × average confidence level.

[0047] In this embodiment, the region number refers to the unique identifier code assigned to each sub-region of the target region. For example, if the target region is divided into 100 sub-regions, they are numbered 001, 002, ..., 100 in order from left to right and from top to bottom.

[0048] In this embodiment, the reference image with the highest clarity and confidence is used as the base image. The SIFT feature point matching algorithm is used to extract the feature points of other reference images and the base image, calculate the homography matrix, and perform projection transformation on other reference images according to the homography matrix to achieve pixel alignment. The pixel deviation of the aligned image is ≤0.5 pixels.

[0049] In this embodiment, the new image refers to an image with uniform position and complete information obtained after multiple reference images from the same source device have undergone pixel alignment processing. That is, after the alignment processing is completed, the corresponding pixels of all reference images are weighted and averaged, with the weight being the confidence level of each reference image, to obtain the pixel value of the new image.

[0050] The beneficial effects of the above technical solution are: by constructing a feature-confidence matrix, combining fitting analysis and trend synchronization analysis to achieve accurate screening of abnormal sub-regions, identifying abnormal factors and retrieving corresponding reference images, generating replacement images through pixel alignment and comprehensive analysis to complete the abnormal replacement, ensuring that the detection results can truly reflect the actual vegetation situation in the target area, and improving the accuracy and practicality of vegetation detection.

[0051] This invention provides a drone-based vegetation detection method that performs cluster analysis on all pixels in a complete image based on the feature complexity and pixel value of each pixel, including: Select a preset number of neighboring pixels around the corresponding pixel, calculate the Pearson correlation coefficient between the feature complexity of the corresponding pixel and each neighboring pixel, take the mean of all correlation coefficients as the neighborhood feature complexity correlation degree, and use the neighborhood feature complexity correlation degree as a correction factor to perform weighted correction on the feature complexity of the corresponding pixel to obtain the corrected complexity. Based on the pixel value distribution of the complete image, it is divided into dense vegetation sub-regions, sparse vegetation sub-regions, and non-vegetation sub-regions. The corrected complexity of each pixel and the regional weight of its sub-region are used as joint features. Density clustering algorithm is used for clustering, and the neighborhood radius of the cluster is adjusted according to the correlation of the neighborhood feature complexity, so as to obtain the clustering map of each complete image.

[0052] In this embodiment, neighboring pixels refer to the surrounding pixels contained within a pixel window of a preset size, centered on the target pixel. These pixels are used to calculate the feature complexity correlation between the target pixel and its surrounding pixels. The preset window size can be adaptively adjusted according to the image resolution. For example, if the target pixel coordinates are (800, 600), the image resolution is 4096×2160, and the preset window size is 5×5, then the neighboring pixels are 24 pixels (excluding the target pixel itself) centered at (800, 600), with an x-coordinate range of 798-802 and a y-coordinate range of 598-602.

[0053] In this embodiment, the Pearson correlation coefficient is a statistic used to measure the degree of linear correlation between two variables (the feature complexity of the target pixel and the feature complexity of the neighboring pixels). The value range is [-1, 1]. The closer the absolute value of the correlation coefficient is to 1, the stronger the linear correlation between the two; the closer it is to 0, the weaker the correlation.

[0054] In this embodiment, the neighborhood feature complexity correlation degree refers to the arithmetic mean of the Pearson correlation coefficients between the target pixel and all its neighboring pixels.

[0055] In this embodiment, the corrected complexity = original feature complexity × (e1 + correlation degree) / (e1 + 1), where e1 is an adjustment coefficient, set to 0.5 to 1.0 according to the overall distribution of image feature complexity. Through extensive experimental verification, the correction effect is best when e1 = 0.8, which not only retains the core information of the original feature complexity, but also effectively integrates the neighborhood correlation information.

[0056] In this embodiment, a dense vegetation sub-region refers to a sub-region with high vegetation coverage and pixel values ​​concentrated in the range of vegetation characteristic pixel values, such as a sub-region in an RGB image where the green band pixel value is 150-255 and accounts for ≥70%. A sparse vegetation sub-region refers to a sub-region with low vegetation coverage and the percentage of pixels whose pixel values ​​conform to the range of vegetation characteristic pixel values, based on the pixel value distribution of the complete image. A non-vegetated sub-region refers to a sub-region with almost no vegetation coverage and the percentage of pixels whose pixel values ​​conform to the range of vegetation characteristic pixel values, based on the pixel value distribution of the complete image.

[0057] In this embodiment, the regional weights are determined by the analytic hierarchy process (AHP). A judgment matrix is ​​constructed for the sub-region type (dense vegetation, sparse vegetation, and no vegetation) and the clustering accuracy. The weight value of each sub-region type is calculated. For example, the regional weight of the dense vegetation sub-region is 1.0, the regional weight of the sparse vegetation sub-region is 0.7, and the regional weight of the no vegetation sub-region is 0.3.

[0058] In this embodiment, the joint feature refers to the comprehensive feature vector formed by combining the corrected complexity of a pixel with the regional weight of its sub-region. That is, the corrected complexity and regional weight are arranged in the order of corrected complexity first and regional weight second to form a two-dimensional joint feature vector. For example, if a pixel belongs to a dense vegetation sub-region, the corrected complexity is 1.4 and the regional weight is 1.0, then its joint feature is [1.4, 1.0]; another pixel belongs to a sparse vegetation sub-region, the corrected complexity is 0.9 and the regional weight is 0.7, then its joint feature is [0.9, 0.7].

[0059] In this embodiment, the DBSCAN algorithm is used. The minimum number of samples (MinPts) of the DBSCAN algorithm is set according to the dimension of the joint features. MinPts=5 for two-dimensional joint features and MinPts=8 for three-dimensional joint features. The neighborhood radius (Eps) is adjusted by the correlation degree of the neighborhood features. The higher the correlation degree, the smaller Eps is, and Eps=0.1×(2-correlation degree) to ensure that pixels of the same type can be effectively clustered and pixels of different types can be accurately distinguished.

[0060] The beneficial effects of the above technical solution are as follows: by introducing the correlation of neighborhood feature complexity to correct the feature complexity of pixels, combining the pixel value distribution to divide different types of sub-regions and assigning region weights, and constructing joint features to perform density clustering with adaptive neighborhood radius, the accuracy and rationality of clustering analysis are significantly improved, providing a high-quality clustering foundation for subsequent boundary line adjustment and anomaly detection.

[0061] This invention provides a vegetation detection method based on unmanned aerial vehicles (UAVs), which pre-analyzes corresponding boundary points and obtains adjustment lines for the cluster boundary lines, including: Extract the deviation sequence based on feature complexity and the mapping difference sequence based on pixel value of the same boundary point in the complete images of different devices, and combine the device attributes of the multi-source devices carried by the UAV to determine the cross-device correlation coefficient; When the boundary point is the intersection of vegetation and non-vegetation, a nonlinear regression model is constructed to calculate the adjustment offset using the ambiguity vector of the confusion matrix, the cross-device correlation coefficient, and the frequency of its own multidimensional matrix as input. When the boundary point is the intersection of different vegetation types, the decision tree model is constructed to calculate the adjustment offset by taking the ambiguity vector of the confusion matrix, the cross-device correlation coefficient, and the row vector of the multidimensional auxiliary matrix as input. The adjustment offsets of all boundary points are arranged according to the topological order of the cluster boundary lines. The actual geographical contour of the target area is used as a constraint to perform geometric correction on the arranged offsets. The adjustment line of the cluster boundary lines is obtained by fitting a Bézier curve.

[0062] In this embodiment, if the feature complexity of a boundary point in the complete images of remote sensing, high-definition, and infrared devices is 1.8, 1.7, and 1.5 respectively, then its deviation sequence is [1.8-1.7, 1.8-1.5, 1.7-1.5] = [0.1, 0.3, 0.2]. Taking the feature complexity of the remote sensing device as the benchmark, if the normalized pixel values ​​of a boundary point in the complete images of remote sensing, high-definition, and infrared devices are 0.85, 0.82, and 0.78 respectively, then taking the remote sensing device as the benchmark, its mapping difference sequence is: [0.85-0.82, 0.85-0.78, 0.82-0.78] = [0.03, 0.07, 0.04].

[0063] In this embodiment, device attributes refer to the inherent characteristic parameters of the multi-source devices carried by the UAV, including spectral range, resolution, imaging principle, exposure time, focal length, etc. It should be noted that device attributes are obtained from the device's factory parameter manual, and some parameters (such as actual exposure time and focal length) can be read in real time by the UAV's device control module and stored in the system database for use when calculating cross-device correlation coefficients.

[0064] In this embodiment, the formula for calculating the cross-device correlation coefficient K is as follows: ; Where w1, w2, w3, w4, w5, and w6 are the dynamic weights of the parameters in each dimension, and w1+w2+w3+w4+w5+w6=1; S1 is the spectral matching degree, which characterizes the degree of overlap and adaptation of the device's spectral range; R1 is the resolution adaptation degree, which quantifies the matching level of the device's image resolution; P1 is the consistency of imaging principles, which reflects the fit of the device's imaging physical mechanism; and T1 is the temporal synchronization, which reflects the degree of coordination of the device's shooting sequence. This is a correction item for the equipment's operating status, used to compensate for the impact of the equipment's real-time operating status on the correlation. The correction coefficient ranges from 0.05 to 0.15 and is dynamically calibrated according to the detection scenario; D1 represents the feature complexity deviation consistency, which quantifies the stability of the feature complexity deviation sequence of boundary points between different devices; M1 represents the pixel value mapping consistency, which quantifies the average difference of the pixel value mapping difference sequence of boundary points between different devices. This is a data consistency correction coefficient, ranging from 0.03 to 0.08, dynamically calibrated based on the degree of dispersion between the deviation sequence and the mapping difference sequence.

[0065] Where αs is the sensitive band adaptation coefficient. If the overlapping interval includes the core sensitive band of vegetation, the value is 1.2; if it only includes some sensitive bands, the value is 1.0; and if it does not include any sensitive bands, the value is 0.8. It should be noted that the length of the spectral overlap interval is the wavelength span of the intersection of the spectral ranges of the two devices, and the length of the vegetation sensitive band interval refers to the intersection length of the core sensitive band of vegetation detection and the spectral overlap interval of the two devices.

[0066] Where r01 and r02 are the image resolutions of the two devices, respectively; ; For example, high-definition equipment (visible light optical imaging) and remote sensing equipment (multispectral optical imaging) are imaging from the same source, so P1=0.7; infrared equipment (thermal imaging) and high-definition equipment (optical imaging) are imaging from different sources, so P1=0.3.

[0067] t is the time difference between the two devices capturing images of the same location in the target area; t0 is the timing synchronization threshold; k is the attenuation coefficient, which is fixed at 1.2. In this embodiment, the default values ​​for w1, w2, w3, w4, w5, and w6 are 0.3, 0.15, 0.15, 0.1, 0.15, and 0.15, respectively. Then, w5 increases to 0.15, w1 decreases to 0.2, and the remaining weights are adjusted proportionally. If Then w6 increases to 0.25, and w2 decreases to 0.05.

[0068] Where SNR0 and SNR are the normalized values ​​of the actual signal-to-noise ratio of the equipment and the normalized values ​​of the rated signal-to-noise ratio of the equipment, respectively; Exp=1 and Exp are the values ​​under ideal stable conditions and the actual exposure stability of the equipment, respectively.

[0069] ,in, This represents the deviation sequence of the same boundary point in complete images from different devices based on feature complexity. is the coefficient of variation of the biased sequence; D0 is the feature complexity bias threshold, which defaults to 0.5 and can be adjusted according to the vegetation type. This is the arithmetic mean of the biased sequence; This is the minimum value, taking the value 0.01; The attenuation coefficient is set to 1.5, which controls the rate at which the degree of deviation dispersion affects the correlation. ,in, This is the upper limit of the pixel value mapping difference, fixed at 0.5, representing the reasonable mapping difference threshold for boundary point pixel values; k is the maximum value of the mapping difference sequence, reflecting the extreme differences in pixel values ​​between devices; k4 is the scaling factor, fixed at 2.0, which converts the average mapping difference into a consistency index in the range of 0 to 1.

[0070] .

[0071] In this embodiment, the nonlinear regression model refers to a mathematical model used to calculate the adjustment offset at the boundary between vegetation and non-vegetation areas. It takes the ambiguity vector of the confusion matrix, the cross-device correlation coefficient, and the frequency of its own multidimensional matrix as inputs, and obtains the adjustment offset through nonlinear fitting. This model can handle complex nonlinear relationships between input parameters and offsets. For example, using a quadratic polynomial nonlinear regression model, the input parameters are the magnitude of the ambiguity vector (x1), the cross-device correlation coefficient (x2), and the frequency of its own multidimensional matrix (x3). The model equation is: Where y is the adjustment offset, and a0 to a9 are model parameters, which are obtained through training with a large amount of sample data. It should be noted that sample data (including input parameters and actual measured adjustment offsets) of the boundary between vegetation and non-vegetation in different scenarios are collected, with no less than 1000 sets of samples; the least squares method is used to fit the sample data to determine the model parameters.

[0072] In this embodiment, a C4.5 decision tree model is used. The input features include elements of the ambiguity vector, cross-device correlation coefficients, and elements of the row vectors of the multi-dimensional auxiliary matrix, totaling 10 input features. The adjustment offset is divided into 5 intervals, such as [-0.5,-0.3), [-0.3,-0.1), [-0.1,0.1), (0.1,0.3], (0.3,0.5], as output categories. The decision tree is trained using sample data to obtain a model that can predict the adjustment offset interval. Specifically, sample data of the boundary points of different vegetation types in different scenarios are collected, i.e., the input features and the actual adjustment offset, with no less than 1500 sets of samples. The sample data is divided into a training set and a test set in a 7:3 ratio. The C4.5 decision tree model is trained using the training set, and the classification accuracy of the model is calculated using the test set. An accuracy of ≥80% is considered acceptable. After training, new boundary point parameters are input, the model outputs the adjustment offset interval, and the median value of the interval is taken as the final adjustment offset.

[0073] In this embodiment, the adjustment offset refers to the pixel distance and direction that the boundary point needs to be adjusted. It is used to correct the deviation of the original cluster boundary line. The distance is in pixels, and the direction is represented by positive and negative values. For example, a positive offset in the x-direction indicates adjustment to the right, and a negative offset indicates adjustment to the left; a positive offset in the y-direction indicates adjustment downwards, and a negative offset indicates adjustment upwards. For example, if the x-direction adjustment offset of a boundary point is 0.2 pixels and the y-direction adjustment offset is -0.1 pixels, it means that the boundary point needs to be adjusted 0.2 pixels to the right and 0.1 pixels upwards.

[0074] In this embodiment, the topological order refers to the connection order of boundary points on the cluster boundary line, that is, the continuous order formed by arranging boundary points in a clockwise or counterclockwise direction.

[0075] In this embodiment, geometric correction refers to correcting the arranged offset sequence by using the actual geographic contour of the target area as a constraint. Specifically, it involves: obtaining the actual geographic contour data of the target area, such as the boundary coordinates of the plots obtained by GPS mapping, and converting them into image pixel coordinates; calculating the deviation between the adjusted boundary line and the actual geographic contour; and using affine transformation to correct the offset sequence so that the deviation between the adjusted boundary line and the actual geographic contour does not exceed 1 pixel, thus achieving geometric correction. The boundary points after geometric correction are a series of discrete pixels. By fitting a Bézier curve, these discrete points are connected to form a smooth curve.

[0076] The beneficial effects of the above technical solution are: it designs differentiated adjustment offset calculation models for different types of boundary points, combines multi-dimensional parameters such as deviation sequence and cross-device correlation coefficient, and obtains accurate adjustment lines through topological order arrangement, geometric correction and Bézier curve fitting, which significantly improves the accuracy, rationality and smoothness of cluster boundary lines, and provides a reliable regional division basis for subsequent anomaly detection and replacement.

[0077] This invention provides a vegetation detection method based on unmanned aerial vehicles (UAVs), which filters abnormal sub-regions and determines the anomalous factors of these sub-regions, including: Determine the confidence interval range for each column vector, match the fitting strategy for the corresponding column vector from the range-strategy lookup table, and perform fitting processing on the corresponding column vector based on the fitting strategy to obtain the fitting residual of the corresponding column vector. If the fitting residual exceeds n1 times the fitting standard deviation of the corresponding interval, then the sub-region of the corresponding column vector is marked as the first candidate sub-region. Taking the first row vector of the feature-confidence matrix as a reference, calculate the trend synchronization coefficient with each of the remaining row vectors, and correct the corresponding trend synchronization coefficient with the average confidence of each row vector as the weight to obtain the weighted synchronization coefficient. If the weighted synchronization coefficient is lower than a preset threshold, the corresponding sub-region is marked as the second candidate sub-region; The intersection of the first candidate sub-region and the second candidate sub-region is used to obtain the abnormal sub-region; For each anomalous sub-region, the vegetation feature type of the column with the largest residual in the corresponding feature-confidence matrix is ​​extracted. Triples are constructed by combining the deviation direction of the fitting residual and the deviation degree of the weighted synchronization coefficient, which serve as the anomalous factor for the corresponding anomalous sub-region.

[0078] In this embodiment, the confidence interval range refers to the numerical distribution range of each column vector in the feature-confidence matrix, that is, the interval between the minimum and maximum confidence values ​​in that column vector. The range-strategy mapping table refers to the pre-established mapping relationship table between the confidence interval range and the linear fitting strategy, and some of its contents are shown in Table 1: Table 1 Scope-Strategy Comparison Table

[0079] In this embodiment, the fitting residual refers to the difference between the actual confidence value of each column vector and the fitted value obtained by fitting the model. It is used to measure the degree of deviation between the actual value and the fitted model and is a key indicator for screening the first candidate sub-region. For example, if the actual confidence value sequence of a column vector is [0.85, 0.88, 0.90, 0.92], and the fitted value sequence obtained by fitting using the least squares method is [0.84, 0.87, 0.91, 0.93], then the corresponding fitting residual sequence is [0.01, 0.01, -0.01, -0.01].

[0080] In this embodiment, n1 refers to the threshold multiple used to determine whether the fitting residual is abnormal, and the value of n1 ranges from 2 to 3. The optimal value is determined through experimental verification. Different n1 values ​​(2.0, 2.2, 2.5, 2.8, 3.0) are selected to test a large number of sample data, and the recognition accuracy and false positive rate of abnormal sub-regions are calculated. The n1 value with the highest accuracy and the lowest false positive rate is selected as the final set value. In this embodiment, n1=2.5.

[0081] In this embodiment, if the target region is divided into 20 sub-regions, after linear fitting analysis, the fitting residuals of 3 sub-regions exceed 2.5 times the fitting standard deviation of the corresponding interval. These 3 sub-regions are the first candidate sub-regions, and their region numbers are 005, 012, and 017.

[0082] In this embodiment, the average confidence level refers to the arithmetic mean of the confidence levels of all elements in a row vector of the feature-confidence matrix. It is used to measure the reliability of the overall feature extraction result of the image and is the core weight parameter for correcting the trend synchronization coefficient. Its value range is [0,1].

[0083] In this embodiment, the ROC curve analysis method is used to determine the preset threshold. The ROC curve is plotted with the weighted synchronization coefficient as the abscissa and the anomaly recognition accuracy as the ordinate. The weighted synchronization coefficient corresponding to the largest area under the curve is selected as the preset threshold. In this embodiment, the preset threshold is verified to be 0.6 through experiments. At this time, the anomaly recognition accuracy reaches 89% and the false judgment rate is only 7%.

[0084] In this embodiment, if the target area is divided into 18 sub-regions, after trend synchronization analysis, the weighted synchronization coefficients of sub-regions 004, 009, 013, and 016 are 0.55, 0.52, 0.48, and 0.57, respectively, all of which are lower than the preset threshold of 0.6. These four sub-regions are marked as second candidate sub-regions.

[0085] In this embodiment, the region numbers of the first candidate sub-region and the second candidate sub-region are stored as two sets respectively. The set of abnormal sub-region numbers is obtained by taking the intersection of the sets. If the set is empty, a prompt message indicating that there are no abnormal sub-regions in the target area is output. If the set is not empty, the detailed information of the abnormal sub-region is extracted, which includes the region number, the corresponding device image, feature data, etc., for subsequent construction of abnormal factors.

[0086] In this embodiment, the column with the largest residual refers to the column with the largest absolute value of the fitted residual in the column vector of the feature-confidence matrix corresponding to the abnormal sub-region. For example, the fitted residual sequence of the column vector corresponding to the abnormal sub-region 009 is [0.03, 0.06, -0.09, 0.04], where the residual with the largest absolute value is -0.09, corresponding to the 3rd column (the fourth figure of the infrared device). The vegetation feature type corresponding to this column is temperature feature, indicating that the anomaly is mainly reflected in the temperature feature.

[0087] In this embodiment, the deviation direction refers to the positive or negative direction of the fitted residual corresponding to the column with the largest residual. It is used to characterize the bias of the anomaly. A positive deviation (residual > 0) indicates that the actual confidence level is higher than the fitted value, and a negative deviation (residual < 0) indicates that the actual confidence level is lower than the fitted value. For example, if the fitted residual of the column with the largest residual is -0.09, the deviation direction is negative, indicating that the confidence level of the temperature feature of the abnormal sub-region in the infrared image corresponding to this column is lower than the normal fitting level; if the residual is 0.08, the deviation direction is positive, and the confidence level is higher than the normal level.

[0088] In this embodiment, .

[0089] In this embodiment, if the triplet of the abnormal sub-region 009 is (temperature feature, negative deviation, 0.133), it means that the core of the abnormality of the sub-region is the abnormal temperature feature, the fitting residual shows a negative deviation, and the degree to which the feature trend deviates from the preset threshold is 0.133.

[0090] The beneficial effects of the above technical solution are as follows: by using a confidence interval-adapted differential fitting strategy, combined with fitting residual analysis, the first candidate sub-region with anomaly in confidence is screened. At the same time, based on the weighted correction of the trend synchronization coefficient and the average confidence, the second candidate sub-region with anomaly in characteristic trend is screened. The intersection of the two candidate sets is used to achieve accurate localization of the abnormal sub-region. Furthermore, a triplet abnormal factor is constructed to comprehensively represent the abnormal information, which significantly improves the accuracy, reliability, and specificity of anomaly detection, and provides accurate and comprehensive basis for subsequent reference image retrieval and anomaly replacement.

[0091] This invention provides a vegetation detection method based on unmanned aerial vehicles (UAVs), which relies on comprehensive analysis of new images from all source devices to obtain replacement images, including: Based on different vegetation feature types, match the corresponding target feature association channels in new images from the same source device; Predict the importance priority of each target feature associated channel, generate the channel priority coefficient of each channel, and predict the vegetation spatial focus distribution of each new image in the target feature associated channel to generate the feature space priority map of the corresponding new image; perform local brightness extraction on each target feature associated channel to obtain the local brightness distribution features in the channel, and perform light detection on the target feature associated channel to identify the vegetation sub-region labels covered by different light types in the channel and the proportion of the corresponding sub-regions in the channel, and calculate the brightness confidence probability of the corresponding channel. Calculate the mean and variance of the significant values ​​of the associated channel priorities of all target features, and correct the original first weight based on the channel priority coefficient in combination with the brightness confidence probability to obtain the brightness-adapted channel priority weight. Based on the feature space priority map and the brightness confidence probability, the original second weight based on the feature space priority map is corrected to obtain the feature space weight after brightness adaptation. The final integrated weight is obtained by weighting and fusing the channel priority weight and feature space weight with the overall confidence of each new image as the base weight and the degree of deviation in the anomaly factor. Based on the final integrated weight, the target feature association channels of the new images from all source devices are weighted and fused to obtain the desired image. The pixel gradient difference between the desired image and the surrounding normal sub-regions is calculated, and the Gaussian fusion algorithm is used to perform gradient transition processing on the edge regions of the initial image to obtain the replacement image.

[0092] In this embodiment, a pre-established vegetation feature type-equipment type-target channel association table is used to match the corresponding target feature association channel based on the vegetation feature type and equipment type in the anomaly factors. Table 2 shows a partial content of the association table: Table 2. Vegetation Feature Type - Equipment Type - Target Channel Association Table

[0093] The channel priority coefficients are determined using the Analytic Hierarchy Process (AHP). The steps are as follows: Construct a channel importance judgment matrix, and invite 3-5 experts in the field of vegetation detection to compare and score the importance of each target feature associated channel in pairs (1-9 scale method, where 1 indicates equal importance and 9 indicates extreme importance). Calculate the largest eigenvalue and the corresponding eigenvector of the judgment matrix; The feature vectors are normalized to obtain the channel priority coefficients; Perform a consistency check (consistency ratio CR < 0.1) to ensure that the coefficient allocation is reasonable. If the check fails, reconstruct the judgment matrix.

[0094] For example, the vegetation feature type of the abnormal factor is spectral feature, and the target feature association channel is the near-infrared channel and green band channel of the remote sensing equipment. Since the near-infrared channel is more sensitive to the vegetation growth status, its channel priority coefficient is set to 0.7, and the green band channel is 0.3.

[0095] In this embodiment, the following steps are used to generate the feature space priority map: Gaussian filtering (3×3 Gaussian kernel size, standard deviation of 1.5) is applied to the target feature association channel image to smooth noise; the local variance of each pixel (the variance of pixel values ​​within a 5×5 window centered on that pixel) is calculated. The smaller the local variance, the more stable the feature and the higher the degree of focus. The local variance is normalized (to 0-255) and then inverted, i.e., the smaller the local variance, the larger the pixel value, to obtain the feature space priority map.

[0096] In this embodiment, the target feature associated channel image is divided into 16×16 pixel local blocks, and the following parameters are calculated for each local block: Luminance mean: The arithmetic mean of the luminance values ​​of all pixels within a local block; Luminance variance: the average of the sum of squares of the differences between the pixel luminance values ​​and the mean within a local block; Peak brightness: The maximum brightness value within a local block; These parameters are integrated into a local brightness distribution feature vector, which is used for subsequent light type identification and brightness confidence probability calculation.

[0097] For example, in the target feature association channel of the near-infrared channel of a remote sensing device, the average brightness of local region A is 185, the variance is 28, and the peak value is 230, indicating that the brightness of this region is relatively high and uniformly distributed; the average brightness of local region B is 120, the variance is 75, and the peak value is 190, indicating that the brightness of this region is relatively low and fluctuates greatly.

[0098] In this embodiment, light detection is achieved based on a machine learning algorithm, and the steps are as follows: Collect channel image samples of different light types (natural light, reflected light, stray light), with ≥500 samples for each type, and extract the local brightness distribution feature vector of the samples; A light type classification model was trained using the Support Vector Machine (SVM) algorithm, with the kernel function set as the Radial Basis Function (RBF). The local brightness distribution feature vector of the target feature associated channel is input into the trained model, and the light type recognition result is output with an accuracy of ≥85%.

[0099] In this embodiment, the target feature associated channel image is annotated at the pixel level based on the light detection results: First, the vegetation area is determined based on the previous clustering results; For each pixel within the vegetation area, a corresponding vegetation sub-region label is assigned based on the light type of the local block to which it belongs; a label image is generated, and the label uses an 8-bit integer encoding (e.g., L1-V=1, L2-V=2, L3-V=3) to facilitate subsequent statistical analysis.

[0100] For example, vegetation sub-regions covered by natural light are labeled L1-V; vegetation sub-regions covered by reflected light are labeled L2-V; and vegetation sub-regions covered by stray light are labeled L3-V, where L represents the light type and V represents the vegetation region.

[0101] In this embodiment, the confidence weights for different light types are preset and determined based on experimental data: natural light 0.9, reflected light 0.6, stray light 0.3, and other light 0.2. The pixel percentage of each vegetation sub-region label within the statistical target feature association channel is calculated as follows: the percentage of a certain label = the number of pixels of that label / the total number of pixels in the vegetation region. The confidence probability of brightness is calculated using a weighted summation formula: Confidence probability of brightness = Σ (proportion of a certain label × confidence weight of that label).

[0102] Priority significance value = Channel priority coefficient × (1 + Channel feature entropy), where the channel feature entropy value is calculated using the entropy calculation formula in information theory. , where p(x) is the probability distribution of the channel pixel values.

[0103] In this embodiment, the channel priority coefficient is directly used as the first weight.

[0104] The channel priority weight after brightness adaptation = (the first weight of the corresponding channel × the brightness confidence probability of the corresponding channel) / the sum of the first weights of all channels × the brightness confidence probabilities.

[0105] In this embodiment, the second weight of each pixel is equal to the pixel value of that pixel / the sum of the pixel values ​​of all pixels in the feature space priority map, ensuring that the sum of the second weights of all pixels is 1. The feature space weight of each pixel after brightness adaptation is equal to the second weight of that pixel × the brightness confidence probability.

[0106] In this embodiment, the overall confidence level = mean confidence level of features × (1 - variance of feature confidence level / ... ),in, The variance benchmark is determined to be 0.5 by statistically analyzing the feature confidence variance of a large number of normal images. If the feature confidence variance > Variance = This is to avoid excessive variance leading to low overall confidence.

[0107] In this embodiment, the preset channel priority weight, feature space weight, and overall confidence level weight coefficients are determined to be 0.4, 0.3, and 0.3 through experiments. The final integrated weight is calculated using a linear weighting formula: Final integrated weight = Channel priority weight × 0.4 + Feature space weight × 0.3 + Overall confidence level × 0.3. The weight coefficients are adjusted through the system configuration file to adapt to different detection scenarios.

[0108] In this embodiment, for each target feature associated channel's pixel, multiply it by its corresponding final integration weight, and then sum the weighted pixel values ​​of all channels to obtain the pixel value of the desired image. The pixel value range is normalized to 0-255. If the summed pixel value exceeds 255, it is taken as 255; if it is below 0, it is taken as 0, ensuring that the image format conforms to the standard. For example, the near-infrared channel (final integration weight 0.5364) and green band channel (0.283) of the new image from the remote sensing equipment, the grayscale channel (0.48) of the new image from the high-definition equipment, and the thermal imaging channel (0.51) of the new image from the infrared equipment are weighted and fused to obtain the initial image. This image integrates vegetation feature information from multiple channels and multiple devices, and initially corrects the abnormal features of abnormal sub-regions.

[0109] In this embodiment, the Sobel operator is used to calculate the pixel gradient difference, and the steps are as follows: Construct 3×3 Sobel operator templates for the x and y directions respectively: x direction: [[1,0,-1],[2,0,-2],[1,0,-1]]; y direction: [[1,2,1],[0,0,0],[-1,-2,-1]]; The initial image and the surrounding normal sub-region image are convolved using an operator template to obtain gradient maps in the x and y directions. For the corresponding pixels in the two gradient maps, the square root of the sum of the squares of the gradient values ​​is calculated to obtain the pixel gradient difference.

[0110] In this embodiment, the Gaussian fusion algorithm is implemented using the OpenCV library in Python. The steps are as follows: Determine the edge region range: Regions with a pixel gradient difference > 15 are identified as edge regions; Choosing the Gaussian kernel size and standard deviation: When the pixel gradient difference in the edge region is large, a 5×5 Gaussian kernel is used. =2.0; When the gradient difference is small, use a 3×3 Gaussian kernel. =1.5; Gaussian convolution is applied to the edge regions to achieve gradient transition; The processed image is stitched together with the surrounding normal sub-region images to obtain an image with a natural transition.

[0111] In this embodiment, if the abnormal feature of the abnormal sub-region is an abnormal temperature feature (the temperature value in the infrared image is significantly higher), the target feature association channel of the new images from remote sensing, high-definition, and infrared equipment is weighted and fused to obtain the required image. Then, the edge region is processed by Gaussian fusion to generate a replacement image. The temperature feature of this image conforms to the normal vegetation pattern (25-30℃), and there are no obvious splicing traces between the edge and the surrounding area, which can accurately reflect the normal vegetation status of the abnormal sub-region.

[0112] The beneficial effects of the above technical solution are as follows: by matching target features and associating channels to focus on core features, and combining channel priority, feature space focus, and brightness confidence probability to perform multi-dimensional weight correction, a scientific and reasonable final integrated weight is constructed to achieve accurate fusion of new images from multiple devices. Then, the edge transition effect is optimized by Gaussian fusion algorithm. The generated replacement image has the characteristics of comprehensive features, high reliability, and smooth edges, ensuring the accuracy and naturalness of abnormal replacement, and significantly improving the overall quality and credibility of vegetation detection results.

[0113] This invention provides a vegetation detection system based on unmanned aerial vehicles (UAVs), such as... Figure 2 As shown, it includes: The fusion processing module is used to acquire several initial images of the target area based on the multi-source devices carried by the UAV, and to fuse all the initial images from the same source device to obtain a complete image. The multi-source devices include: remote sensing devices, high-definition devices, and infrared devices. The clustering and adjustment module is used to perform clustering analysis on each complete image, stack all cluster maps to lock the cluster boundary lines, and adjust the cluster boundary lines according to the common points of the boundary lines to obtain the adjusted image. The anomaly replacement module is used to extract features and perform anomaly analysis on the adjusted image, retrieve reference images from all initial images for anomaly replacement, obtain the detection results of the target region, and output them.

[0114] The beneficial effects of the above technical solution are as follows: by using a drone equipped with remote sensing, high-definition, and infrared multi-source equipment to collect initial images of the target area, and then using image fusion of the same source equipment to obtain a comprehensive and complete image, the image region division is optimized through cluster analysis and boundary adjustment, and finally, accurate detection is achieved by combining feature extraction, anomaly analysis and reference image replacement, which significantly improves the comprehensiveness, accuracy and reliability of vegetation detection, and provides high-quality data support for subsequent vegetation management and decision-making.

[0115] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A vegetation detection method based on unmanned aerial vehicles (UAVs), characterized in that, include: Step 1: Acquire several initial images of the target area using the multi-source devices carried by the UAV, and fuse all the initial images from the same source devices to obtain a complete image. The multi-source devices include: remote sensing devices, high-definition devices, and infrared devices. Step 2: Perform cluster analysis on each complete image, stack all cluster maps to lock the cluster boundary lines, and adjust the cluster boundary lines according to the common points of the boundary lines to obtain the adjusted image; Step 3: Perform feature extraction and anomaly analysis on the adjusted image, retrieve reference images from all initial images for anomaly replacement, obtain the detection results of the target region, and output them; Step 2 includes: Determine the feature complexity of each pixel in each complete image, and perform cluster analysis on all pixels in the corresponding complete image based on the feature complexity and pixel value of each pixel to obtain the cluster map of each complete image. All clustering diagrams are stacked, with any one clustering diagram as the first diagram and the remaining clustering diagrams as the second diagram. The clustering ambiguity vectors of the boundary points of each clustering boundary line in the first diagram and the corresponding points in each second diagram are extracted. The boundary points are then arranged in the order of the clustering boundary lines to construct the confusion matrix of the corresponding clustering boundary lines. Extract the self-multidimensional matrix of the boundary shared points in the first image and the single row vector of the corresponding boundary shared points acting on each second image, and construct a multidimensional auxiliary matrix based on the single row vector. The number of dimensions of the self-multidimensional matrix is ​​the frequency of the corresponding boundary shared points being used, and the number of dimensions of the multidimensional auxiliary matrix is ​​the number of second images. Based on the confusion matrix of the cluster boundary line and the self-multidimensional matrix and multidimensional auxiliary matrix of the boundary common points in the confusion matrix, the corresponding boundary points are pre-analyzed and the adjustment line of the cluster boundary line is obtained, resulting in the third figure. Using the aforementioned adjustment line as a reference, each second figure is adjusted to obtain the corresponding fourth figure; The third and fourth images are the adjusted images; This involves clustering analysis of all pixels in the corresponding complete image based on the feature complexity and pixel value of each pixel, including: Select a preset number of neighboring pixels around the corresponding pixel, calculate the Pearson correlation coefficient between the feature complexity of the corresponding pixel and each neighboring pixel, take the mean of all correlation coefficients as the neighborhood feature complexity correlation degree, and use the neighborhood feature complexity correlation degree as a correction factor to perform weighted correction on the feature complexity of the corresponding pixel to obtain the corrected complexity. Based on the pixel value distribution of the complete image, it is divided into dense vegetation sub-regions, sparse vegetation sub-regions, and non-vegetation sub-regions. The corrected complexity of each pixel and the regional weight of its sub-region are used as joint features. Density clustering algorithm is used for clustering, and the neighborhood radius of the cluster is adjusted according to the correlation of the neighborhood feature complexity, so as to obtain the clustering map of each complete image.

2. The vegetation detection method based on unmanned aerial vehicles according to claim 1, characterized in that, Step 3 includes: The vegetation features and feature confidence scores of each sub-region enclosed by the adjusted boundary line are extracted from the third image and each fourth image to construct the feature-confidence matrix of the target region. Fitting analysis is performed on each column vector of the feature-confidence matrix, and trend synchronization analysis is performed on the first row vector and each of the remaining row vectors to screen out abnormal sub-regions and determine the abnormal factors of the abnormal sub-regions. The region number of the abnormal sub-region is locked, and based on the abnormal factor and the device type of the first and second images, the number of initial images with the same region code captured from each source device is determined and the retrieved images are regarded as reference images. A new image is obtained by aligning the pixels of the reference image of each source device, and a replacement image is obtained by comprehensive analysis based on the new images of all source devices. Then, the vegetation features of the corresponding sub-region are replaced based on the replacement features of the replacement image.

3. The vegetation detection method based on unmanned aerial vehicles according to claim 1, characterized in that, Pre-analysis of the corresponding boundary points and obtaining the adjustment lines of the cluster boundary lines include: Extract the deviation sequence based on feature complexity and the mapping difference sequence based on pixel value of the same boundary point in the complete images of different devices, and combine the device attributes of the multi-source devices carried by the UAV to determine the cross-device correlation coefficient; When the boundary point is the intersection of vegetation and non-vegetation, a nonlinear regression model is constructed to calculate the adjustment offset using the ambiguity vector of the confusion matrix, the cross-device correlation coefficient, and the frequency of its own multidimensional matrix as input. When the boundary point is the intersection of different vegetation types, the decision tree model is constructed to calculate the adjustment offset by taking the ambiguity vector of the confusion matrix, the cross-device correlation coefficient, and the row vector of the multidimensional auxiliary matrix as input. The adjustment offsets of all boundary points are arranged according to the topological order of the cluster boundary lines. The actual geographical contour of the target area is used as a constraint to perform geometric correction on the arranged offsets. The adjustment line of the cluster boundary lines is obtained by fitting a Bézier curve.

4. The vegetation detection method based on unmanned aerial vehicles according to claim 2, characterized in that, Filtering out anomalous sub-regions and identifying anomalous factors within these sub-regions includes: Determine the confidence interval range for each column vector, match the fitting strategy for the corresponding column vector from the range-strategy lookup table, and perform fitting processing on the corresponding column vector based on the fitting strategy to obtain the fitting residual of the corresponding column vector. If the fitting residual exceeds n1 times the fitting standard deviation of the corresponding interval, then the sub-region of the corresponding column vector is marked as the first candidate sub-region. Taking the first row vector of the feature-confidence matrix as a reference, calculate the trend synchronization coefficient with each of the remaining row vectors, and correct the corresponding trend synchronization coefficient with the average confidence of each row vector as the weight to obtain the weighted synchronization coefficient. If the weighted synchronization coefficient is lower than a preset threshold, the corresponding sub-region is marked as the second candidate sub-region; The intersection of the first candidate sub-region and the second candidate sub-region is used to obtain the abnormal sub-region; For each anomalous sub-region, the vegetation feature type of the column with the largest residual in the corresponding feature-confidence matrix is ​​extracted. Triples are constructed by combining the deviation direction of the fitting residual and the deviation degree of the weighted synchronization coefficient, which serve as the anomalous factor for the corresponding anomalous sub-region.

5. The vegetation detection method based on unmanned aerial vehicles according to claim 2, characterized in that, Replacement images are obtained by comprehensive analysis of new images from all source devices, including: Based on different vegetation feature types, match the corresponding target feature association channels in new images from the same source device; Predict the importance priority of each target feature associated channel, generate the channel priority coefficient of each channel, and predict the vegetation spatial focus distribution of each new image in the target feature associated channel to generate the feature space priority map of the corresponding new image; perform local brightness extraction on each target feature associated channel to obtain the local brightness distribution features in the channel, and perform light detection on the target feature associated channel to identify the vegetation sub-region labels covered by different light types in the channel and the proportion of the corresponding sub-regions in the channel, and calculate the brightness confidence probability of the corresponding channel. Calculate the mean and variance of the significant values ​​of the associated channel priorities of all target features, and correct the original first weight based on the channel priority coefficient in combination with the brightness confidence probability to obtain the brightness-adapted channel priority weight. Based on the feature space priority map and the brightness confidence probability, the original second weight based on the feature space priority map is corrected to obtain the feature space weight after brightness adaptation. The final integrated weight is obtained by weighting and fusing the channel priority weight and feature space weight with the overall confidence of each new image as the base weight and the degree of deviation in the anomaly factor. Based on the final integrated weight, the target feature association channels of the new images from all source devices are weighted and fused to obtain the desired image. The pixel gradient difference between the desired image and the surrounding normal sub-regions is calculated, and the Gaussian fusion algorithm is used to perform gradient transition processing on the edge regions of the initial image to obtain the replacement image.

6. A vegetation detection system based on unmanned aerial vehicles (UAVs), characterized in that, include: The fusion processing module is used to acquire several initial images of the target area based on the multi-source devices carried by the UAV, and to fuse all the initial images from the same source device to obtain a complete image. The multi-source devices include: remote sensing devices, high-definition devices, and infrared devices. The clustering and adjustment module is used to perform clustering analysis on each complete image, stack all cluster maps to lock the cluster boundary lines, and adjust the cluster boundary lines according to the common points of the boundary lines to obtain the adjusted image. An anomaly replacement module is used to extract features and perform anomaly analysis on the adjusted image, retrieve reference images from all initial images for anomaly replacement, obtain the detection results of the target region, and output them. The clustering and adjustment module includes: Determine the feature complexity of each pixel in each complete image, and perform cluster analysis on all pixels in the corresponding complete image based on the feature complexity and pixel value of each pixel to obtain the cluster map of each complete image. All clustering diagrams are stacked, with any one clustering diagram as the first diagram and the remaining clustering diagrams as the second diagram. The clustering ambiguity vectors of the boundary points of each clustering boundary line in the first diagram and the corresponding points in each second diagram are extracted. The boundary points are then arranged in the order of the clustering boundary lines to construct the confusion matrix of the corresponding clustering boundary lines. Extract the self-multidimensional matrix of the boundary shared points in the first image and the single row vector of the corresponding boundary shared points acting on each second image, and construct a multidimensional auxiliary matrix based on the single row vector. The number of dimensions of the self-multidimensional matrix is ​​the frequency of the corresponding boundary shared points being used, and the number of dimensions of the multidimensional auxiliary matrix is ​​the number of second images. Based on the confusion matrix of the cluster boundary line and the self-multidimensional matrix and multidimensional auxiliary matrix of the boundary common points in the confusion matrix, the corresponding boundary points are pre-analyzed and the adjustment line of the cluster boundary line is obtained, resulting in the third figure. Using the aforementioned adjustment line as a reference, each second figure is adjusted to obtain the corresponding fourth figure; The third and fourth images are the adjusted images; This involves clustering analysis of all pixels in the corresponding complete image based on the feature complexity and pixel value of each pixel, including: Select a preset number of neighboring pixels around the corresponding pixel, calculate the Pearson correlation coefficient between the feature complexity of the corresponding pixel and each neighboring pixel, take the mean of all correlation coefficients as the neighborhood feature complexity correlation degree, and use the neighborhood feature complexity correlation degree as a correction factor to perform weighted correction on the feature complexity of the corresponding pixel to obtain the corrected complexity. Based on the pixel value distribution of the complete image, it is divided into dense vegetation sub-regions, sparse vegetation sub-regions, and non-vegetation sub-regions. The corrected complexity of each pixel and the regional weight of its sub-region are used as joint features. Density clustering algorithm is used for clustering, and the neighborhood radius of the cluster is adjusted according to the correlation of the neighborhood feature complexity, so as to obtain the clustering map of each complete image.

Images