Pasture reseeding method and system based on multi-spectral image of unmanned aerial vehicle

Abstract

The invention discloses a pasture reseeding method and system based on an unmanned aerial vehicle multispectral image, and the method comprises the steps: carrying a multispectral camera through an unmanned aerial vehicle to shoot a target pasture according to a planned route, so as to obtain detailed multispectral image data; splitting and coding the multispectral image data by adopting an image coding technology based on deep learning to mine space structure information of a plaque region in each wave band in the multispectral image data, so as to enhance the expression capability of each wave band feature; and further performing feature selection on the features in each wave band to extract fine-grained features most representative for detection of the plaque region in the wave band, and realizing detection and segmentation of the grassland plaque region based on the features. Thus, through effective coding and feature fusion of the wave band spectral image data, the accuracy of detection of the grassland plaque area is improved, and thus the precision and effectiveness of pasture reseeding are improved.

Description

Technical Field

[0001] The present application relates to the field of image analysis, and more specifically, to a forage reseeding method and system based on multispectral imaging of unmanned aerial vehicles. Background Art

[0002] In the field of grassland management and ecological restoration, forage reseeding is a vital task, which is not only related to the productivity and economic benefits of the pasture, but also directly related to the health and stability of the grassland ecosystem. With the advancement of science and technology, the application of drone technology and multispectral image analysis has brought new opportunities for forage reseeding. However, although the current technical solutions have achieved certain results, there are still many limitations in processing multispectral image data, especially when facing grassland spot detection and segmentation tasks in complex environments.

[0003] Patent CN118865125A provides a grassland spot detection and forage reseeding method based on multispectral imaging of unmanned aerial vehicles. It collects near- and low-altitude image data through a multispectral camera mounted on a drone, and further uses a deep learning algorithm that integrates U-Net and DeepLabV3+ to process the image data collected by the multispectral camera to achieve forage reseeding. Although this solution overcomes the difficulty of grassland spot detection and segmentation in complex environments, however, since the various features within the band are often highly correlated, that is, some features may be individually important, but the information provided by adjacent features may be repeated or similar. A large amount of repeated information will lead to the introduction of a large amount of redundant information. This redundancy not only increases the complexity of data processing, but may also lead to overfitting during model training, thereby reducing algorithm efficiency and prediction accuracy.

[0004] Therefore, a forage reseeding method and system based on UAV multispectral imaging is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method and system for grass reseeding based on multispectral imaging of unmanned aerial vehicles, which uses a multispectral camera equipped with an unmanned aerial vehicle to shoot the target pasture according to the planned route to obtain detailed multispectral image data, and uses image coding technology based on deep learning to split and encode the multispectral image data to mine the spatial structure information of the empty spot area in each band in the multispectral image data, so as to enhance the expression ability of each band feature, and further perform feature selection on the features in each band to extract the most representative fine-grained features for the detection of empty spot areas in the band, and realize the detection and segmentation of grassland empty spot areas based on these features. In this way, through the effective encoding and feature fusion of band spectral image data, the accuracy of grassland empty spot area detection is improved, thereby improving the accuracy and effectiveness of grass reseeding.

[0006] According to one aspect of the present application, there is provided a method for reseeding pasture based on multi-spectral images of unmanned aerial vehicles, which includes: planning the flight path of a multi-spectral unmanned aerial vehicle according to the geographical environment information of the target pasture on the same day; obtaining multi-spectral image data of the target pasture through a multi-spectral camera deployed on the multi-spectral unmanned aerial vehicle; processing the multi-spectral image data to detect and segment the grassland bare patch area, and generating reseeding prescription data based on the exposure degree, coordinate position and size information of the grassland bare patch area; and inputting the reseeding prescription data into a plant protection unmanned aerial vehicle to complete the reseeding of the pasture in the target pasture by the plant protection unmanned aerial vehicle.

[0007] According to another aspect of the present application, there is provided a system for reseeding pasture based on multi-spectral images of unmanned aerial vehicles, which includes:

[0008] A flight path planning module, configured to plan the flight path of a multi-spectral unmanned aerial vehicle according to the geographical environment information of the target pasture on the same day;

[0009] A multi-spectral image data acquisition module, configured to obtain multi-spectral image data of the target pasture through a multi-spectral camera deployed on the multi-spectral unmanned aerial vehicle;

[0010] A bare patch area detection and segmentation module, configured to process the multi-spectral image data to detect and segment the grassland bare patch area, and generate reseeding prescription data based on the exposure degree, coordinate position and size information of the grassland bare patch area;

[0011] A pasture reseeding module, configured to input the reseeding prescription data into a plant protection unmanned aerial vehicle to complete the reseeding of the pasture in the target pasture by the plant protection unmanned aerial vehicle.

[0012] Compared with the prior art, a method and system for reseeding pasture based on multi-spectral images of unmanned aerial vehicles provided by the present application use an unmanned aerial vehicle equipped with a multi-spectral camera to photograph a target pasture according to a planned flight path to obtain detailed multi-spectral image data, and use an image coding technology based on deep learning to split and code the multi-spectral image data to mine the spatial structure information of the bare patch area in each band of the multi-spectral image data, so as to enhance the expression ability of the features of each band, further perform feature selection on the features in each band to extract the most representative fine-grained features for detecting the bare patch area within the band, and detect and segment the grassland bare patch area based on these features. In this way, through the effective coding and feature fusion of the band spectral image data, the accuracy of detecting the grassland bare patch area is improved, thereby improving the accuracy and effectiveness of pasture reseeding. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other objects, features, and advantages of the present application will become more apparent by describing the embodiments of the present application in more detail with reference to the accompanying drawings. The drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0014] Figure 1 It is a flowchart of a forage reseeding method based on UAV multispectral images according to an embodiment of the present application;

[0015] Figure 2 It is a schematic diagram of data flow of a forage reseeding method based on UAV multispectral images according to an embodiment of the present application;

[0016] Figure 3 It is a flowchart of sub-step S3 of a forage reseeding method based on UAV multispectral images according to an embodiment of the present application;

[0017] Figure 4 It is a flowchart of sub-step S33 of a forage reseeding method based on UAV multispectral images according to an embodiment of the present application;

[0018] Figure 5 It is a block diagram of a forage reseeding system based on UAV multispectral images according to an embodiment of the present application. Detailed Embodiments

[0019] Next, exemplary embodiments according to the present application will be described in detail with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0020] As shown in the present application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0021] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0022] In this application, flowcharts are used to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the operations before or below do not necessarily need to be executed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0023] Next, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all embodiments of this application. It should be understood that this application is not limited by the exemplary embodiments described herein.

[0024] Patent CN118865125A provides a method for detecting grassland patches and reseeding forage based on multi-spectral images of unmanned aerial vehicles. First, the optimal flight path of the multi-spectral unmanned aerial vehicle is planned according to the geographical environment information of the target pasture on the same day. Then, near-low-altitude images of the grassland pasture are collected by the carried multi-spectral camera. Further, a deep learning algorithm that combines U-Net and DeepLabV3+ is used to process the image data to achieve precise detection and segmentation of the grassland patch area. However, during the process of processing the said image data, since there is often a high degree of correlation between the features within the band, that is, although some features are individually important, the information provided by them may be repetitive or similar to that of adjacent features. A large amount of repetitive information will lead to the introduction of a large amount of redundant information. This redundancy not only increases the complexity of data processing, but may also cause overfitting during the model training process, thereby reducing the algorithm efficiency and prediction accuracy.

[0025] Based on this, in the technical solution of this application, a method for reseeding forage based on multi-spectral images of unmanned aerial vehicles is proposed. Figure 1 It is a flowchart of the method for reseeding forage based on multi-spectral images of unmanned aerial vehicles according to the embodiments of this application. Figure 2 It is a schematic diagram of data flow of the method for reseeding forage based on multi-spectral images of unmanned aerial vehicles according to the embodiments of this application. As Figure 1 and Figure 2 shown, the method for reseeding forage based on multi-spectral images of unmanned aerial vehicles according to the embodiments of this application includes the steps: S1, planning the flight path of the multi-spectral unmanned aerial vehicle according to the geographical environment information of the target pasture on the same day; S2, obtaining the multi-spectral image data of the target pasture through the multi-spectral camera deployed on the multi-spectral unmanned aerial vehicle; S3, processing the multi-spectral image data to achieve the detection and segmentation of the grassland patch area, and generating reseeding prescription data based on the exposure degree, coordinate position and size information of the grassland patch area; S4, inputting the reseeding prescription data into the plant protection unmanned aerial vehicle to complete the reseeding of forage in the target pasture by the plant protection unmanned aerial vehicle.

[0026] Specifically, for the step S1, plan the flight route of the multispectral drone according to the geographical environment information of the target ranch on the current day. That is, based on the geographical environment information of the target ranch on the current day, determine the optimal terrain-following flight altitude, flight speed, and reasonably set the heading overlap rate and side overlap rate, so as to collect qualified multispectral image data, laying a solid data foundation for subsequent grassland bare patch detection and forage seeding work. In one example, a handheld RTK (Real-Time Kinematic) device can be used to accurately locate the boundary corner points of the ranch fence, and transmit these latitude and longitude coordinate data to the drone to provide accurate basic geographical information for subsequent flight route planning. Next, in order to adapt to the actual flight conditions, the flight parameters of the drone will be adjusted according to factors such as the wind speed, wind direction, and atmospheric visibility on the current day, such as modifying the flight forward speed and aerial photography altitude, and reasonably setting the heading overlap rate and side overlap rate. Such adjustments can ensure clear and stable image data can be obtained under different weather conditions. At the same time, when designing the flight path of the drone considering the terrain of the ranch, try to reduce the number of turns of the drone to increase the proportion of straight flight time, which not only improves the flight efficiency but also makes the collected data more continuous and complete. In addition, for the selection of the specific flight altitude, it is necessary to comprehensively consider the requirements of terrain-following flight, that is, as close to the ground as possible while avoiding obstacle interference, to ensure that the coverage range of the multispectral image data is wide enough and of good quality. The flight speed should be considered such that it does not affect the imaging quality and can improve the operation efficiency. And setting appropriate heading overlap rate and side overlap rate is to ensure that there is enough intersection between adjacent photos for easy subsequent stitching processing, and at the same time can also improve the overall accuracy of the finally generated image.

[0027] Specifically, for the step S2, obtain the multispectral image data of the target ranch through the multispectral camera deployed on the multispectral drone. In one example, the multispectral drone flies according to the pre-planned route and continuously collects multispectral images during the flight. These images contain information in different bands, such as red, green, near-infrared (NIR), red edge, etc. Each band reflects the reflection characteristics of surface objects within a specific electromagnetic wavelength range. To ensure the quality and consistency of the data, use Pix4Dmapper software to check the alignment of the acquired multispectral remote sensing images, and at the same time input the spectral correction parameters of each band (such as green, red, near-infrared, red edge), and complete radiometric correction during the stitching process, which can reduce data deviation caused by environmental changes or sensor differences. In addition, the corner point latitude and longitude coordinate information obtained through the handheld RTK device is imported into Pix4Dmapper as the ground control points for the stitched images to correct the distortion of the stitched images and improve the image processing accuracy. This process is crucial for ensuring the accuracy of the finally generated large-area stitched image, providing a reliable basis for subsequent analysis.

[0028] Specifically, in step S3, the multispectral image data is processed to detect and segment the grassland bare patch area, and seeding prescription data is generated based on the exposure degree, coordinate position, and size information of the grassland bare patch area. Specifically, first, in a specific example of the present application, as Figure 3 shown, step S3 includes: S31, splitting the multispectral image data based on bands to obtain a set of band spectral image data; S32, extracting band spectral image features from the set of band spectral image data to obtain a set of band spectral image encoded features; S33, performing spectral feature selection on each band spectral image encoded feature in the set of band spectral image encoded features to obtain a set of sparsified band spectral image encoded features; S34, fusing the set of sparsified band spectral image encoded features to obtain a sparsified spectral image full-band feature cascade encoded feature; S35, performing semantic segmentation on the sparsified spectral image full-band feature cascade encoded feature to detect and segment the grassland bare patch area.

[0029] Specifically, in step S31, the multispectral image data is split based on bands to obtain a set of band spectral image data. It should be understood that a multispectral image is composed of a combination of images in multiple different wavelength ranges (i.e., bands), and each band can capture different electromagnetic radiation characteristics reflected or emitted by surface objects. However, since the information in different bands has different emphases and may be redundant with each other, directly using the unprocessed multispectral image may increase the complexity of subsequent algorithm processing, waste computing resources, and even affect the effect of model training. Therefore, the original multispectral image needs to be separated into individual image data sets according to each independent band, so that the information provided by each band can be analyzed in fine detail, thereby improving the accuracy of detecting the grassland bare patch area.

[0030] Specifically, in S32, band spectral image feature extraction is performed on the set of the band spectral image data to obtain a set of band spectral image coding features. It should be understood that in the set of the band spectral image data after band splitting, each band spectral image data represents a specific aspect of the surface reflection characteristics, and the information in these images is often complex, containing a large amount of spatial structure information and features at different scales. To effectively extract these features and convert them into information helpful for bare spot detection to improve the efficiency of forage reseeding, in the technical solution of this application, a neural network model based on deep learning is used to encode each band spectral image data in the set of the band spectral image data to extract the spatial distribution features of the bare spot regions in each band spectral image data. However, considering that the bare spot regions may present irregular shapes and different sizes, and a larger receptive field helps to better understand complex background information. Compared with traditional convolutional neural networks, dilated convolution can effectively expand the receptive field of each filter by inserting holes in the convolution kernel, that is, adding intervals between filter elements, so as to capture spatial information in a larger range. In addition, since dilated convolution can maintain the resolution of the input image and avoid the loss of spatial details caused by traditional downsampling operations, it is very suitable for the bare spot detection task that requires high-precision positioning. Therefore, in the embodiments of this application, a dilated convolution neural network model is used to perform dilated convolution encoding on each band spectral image data in the set of the band spectral image data to obtain a set of band spectral image coding feature maps as the set of the band spectral image coding features.

[0031] Specifically, in step S33, spectral feature selection is performed on each of the spectral image coding features in the set of spectral image coding features of the band spectrum to obtain a set of sparsified spectral image coding features of the band spectrum. It should be understood that the set of spectral image coding feature maps of the band spectrum contains the spectral image features of each band spectral image, and these features may contain a large amount of redundant information or noise, which is not conducive to the accuracy of grassland patch detection and segmentation. At the same time, in multi-spectral images, there may be a high degree of correlation between data in different bands. That is, although some features are individually important, the information they provide may be repetitive or similar. This redundant information will increase the complexity of data processing and may lead to overfitting during model training, thereby reducing the algorithm efficiency and the accuracy of patch detection. Therefore, in order to more effectively identify the distribution of grassland patches in pastures and improve the efficiency and accuracy of forage reseeding, in the technical solution of this application, further spectral feature selection is performed on each of the spectral image coding features in the set of spectral image coding features of the band spectrum to obtain a set of sparsified spectral image coding features of the band spectrum. By selecting spectral features, it is possible to optimize the feature representation of the patch region by effectively evaluating the interaction degree between the local distribution features of patches and neighboring features in each band spectral image, and retain the fine-grained features that are most representative of patch detection. In particular, in addition, since feature selection emphasizes the relationship between features rather than relying solely on the importance of individual features, the selected features are often more capable of reflecting the true characteristics of the data, thereby enhancing the interpretability and credibility of the model to achieve more accurate and efficient detection and segmentation of grassland patch regions, guiding subsequent forage reseeding work, and promoting the healthy and stable development of the grassland ecosystem. Specifically, in a specific example of this application, as Figure 4 shown, step S33 includes: S331, performing feature decoupling and feature flattening on each of the spectral image coding feature maps in the set of spectral image coding feature maps of the band spectrum to obtain a set of local feature vectors of the spectral image coding of the band spectrum; S332, calculating the local feature importance scores of each of the local feature vectors of the spectral image coding of the band spectrum in the set of local feature vectors of the spectral image coding of the band spectrum to obtain a set of local feature importance score values of the spectral image of the band spectrum; S333, based on the set of local feature importance score values of the spectral image of the band spectrum, performing feature selection and feature shape reshaping on the set of local feature vectors of the spectral image coding of the band spectrum to obtain a set of sparsified spectral image coding feature maps as the set of sparsified spectral image coding features.

[0032] More specifically, for the S331, feature decoupling and feature flattening are performed on each of the band spectral image encoding feature maps in the set of band spectral image encoding feature maps to obtain a set of band spectral image encoding local feature vectors. Considering that in the band spectral image data encoded by the dilated convolutional neural network model, although rich spatial structure information has been extracted, this information still exists in the form of two-dimensional feature maps, which contain a large amount of redundant and interrelated information. To more effectively extract fine-grained features within each band and reduce the impact of redundant information on subsequent processing steps, feature decoupling and feature flattening are performed on each of the band spectral image encoding feature maps in the set of band spectral image encoding feature maps to obtain a set of band spectral image encoding local feature vectors. In this way, it can be ensured that the elements in each local feature vector represent different but related feature dimensions, thus providing a clearer and more independent component representation for subsequent feature selection. In particular, through feature decoupling and feature flattening, not only can the speckle distribution in each local region be analyzed, but also the information of all feature points in this region can be captured, providing a solid data basis for subsequent feature importance evaluation and neighborhood activity calculation. In a specific example, the following feature decoupling and feature flattening formula is used to perform feature decoupling and feature flattening on each of the band spectral image encoding feature maps in the set of band spectral image encoding feature maps to obtain a set of band spectral image encoding local feature vectors; where the feature decoupling and feature flattening formula is:

[0033] Decouple(F) = {M 1 , M 2 ,..., M i ,..., M n}

[0034] Flatten{M 1 , M 2 ,..., M i ,..., M n} = {x 1 , x 2 ,..., x i ,..., x n}

[0035] where F represents the band spectral image encoding feature map, Decouple(·) is the feature decoupling operation, M 1 , M 2 , M i , M n respectively represent the sets of the 1st, 2nd, ith, and nth band spectral image encoding feature matrices in the set of band spectral image encoding feature matrices, Flatten(·) is the feature flattening process, and x1 , x 2 , x i , x n respectively represent the 1st, 2nd, ith, and nth band spectral image coding feature vectors in the set of the band spectral image coding feature vectors.

[0036] More specifically, in S332, calculate the local feature importance scores of each band spectral image coding local feature vector in the set of the band spectral image coding local feature vectors to obtain a set of band spectral image local feature importance score values. Specifically, input each band spectral image coding local feature vector in the set of the band spectral image coding local feature vectors into the importance metric module to obtain the set of the band spectral image local feature importance score values. It should be understood that although each local feature vector in the set of the band spectral image coding local feature vectors captures the information in a specific area, not all features are equally important for the bare spot detection task. To improve the model efficiency, reduce redundant information, and avoid overfitting, it is necessary to evaluate the importance of each local feature vector for identifying the bare spot area on the grassland and perform screening accordingly. In the technical solution of this application, by introducing the importance metric module to calculate the feature importance scores, a scoring system regarding the relative importance of each band spectral image coding local feature vector can be obtained to ensure that only the features with the highest representativeness and contribution degree for bare spot area detection are retained in the subsequent steps, thereby optimizing the model structure and improving the bare spot detection accuracy. In a specific example, use the following importance score calculation formula to calculate the local feature importance scores of each band spectral image coding local feature vector in the set of the band spectral image coding local feature vectors to obtain the set of the band spectral image local feature importance score values; wherein, the importance score calculation formula is:

[0037]

[0038] where, W i is the weight matrix corresponding to x i , b i is the bias vector corresponding to x i , is matrix multiplication, is the transposed vector of the modulation vector, s i is the band spectral image local feature importance score value corresponding to x i .

[0039] More specifically, for the S333, based on the set of local feature importance score values of the band spectral image, feature selection and feature shape reshaping are performed on the set of encoded local feature vectors of the band spectral image to obtain a set of sparsified band spectral image encoded feature maps as the set of sparsified band spectral image encoded features. That is, in the technical solution of this application, first, based on the set of local feature importance score values of the band spectral image, the set of encoded local feature vectors of the band spectral image is sorted in descending order to obtain the descending distribution of the local feature vectors of the band spectral image. That is, according to the obtained local feature importance score values of the band spectral image, the set of encoded local feature vectors of the band spectral image is sorted from high to low in importance to form a descending sequence, so as to identify those features that are most critical for the detection of air holes, and a certain proportion or number of high-score features can also be retained according to actual needs, so as to achieve the purpose of sparsification. This step not only simplifies the feature selection process of the band spectral image, but also enables subsequent computing resources to be concentrated on the most important features. In a specific example, the following permutation formula is used to sort the set of encoded local feature vectors of the band spectral image in descending order to obtain the descending distribution of the local feature vectors of the band spectral image; where the permutation formula is:

[0040]

[0041] Where represents the descending sorting operation, v 1 , v 2 , v k , v n respectively represent the 1st, 2nd, kth, and nth band spectral image local feature vectors in the descending distribution of the band spectral image local feature vectors.

[0042] Next, based on the neighborhood features of each band spectral image local feature vector in the descending order distribution of the band spectral image local feature vectors, feature selection is performed on the descending order distribution of the band spectral image local feature vectors to obtain the descending order distribution of the selected band spectral image local feature vectors. Specifically, first, based on the neighborhood features of each band spectral image local feature vector in the descending order distribution of the band spectral image local feature vectors, the feature neighborhood activity of each band spectral image local feature vector is calculated to obtain a set of band spectral image local feature neighborhood activities. Here, the calculation of the feature neighborhood activity aims to measure the association strength between each band spectral image local feature vector and its neighboring features, emphasizing the synergistic effect between features, which is crucial for understanding complex data structures. In the embodiments of the present application, taking the first band spectral image local feature vector and the second band spectral image local feature vector as examples, the specific steps for calculating the feature neighborhood activities of the first band spectral image local feature vector and the second band spectral image local feature vector include: extracting the first band spectral image local feature vector and the second band spectral image local feature vector from the descending order distribution of the band spectral image local feature vectors; calculating the band spectral image neighborhood feature factors of the first band spectral image local feature vector and the second band spectral image local feature vector to obtain the first band spectral image neighborhood feature factor and the second band spectral image neighborhood feature factor; dividing the second band spectral image neighborhood feature factor by the first band spectral image neighborhood feature factor and taking the obtained quotient as the band spectral image local feature neighborhood activity of the first band spectral image local feature vector. Furthermore, to further optimize the feature selection process, avoid overfitting and improve the model efficiency, it is necessary to evaluate the interaction degree between each band spectral image local feature and its neighborhood features, so as to highlight the most representative and expressive features when detecting the void area. Therefore, in the technical solution of the present application, based on the set of band spectral image local feature neighborhood activities, feature selection is performed on the descending order distribution of the band spectral image local feature vectors to obtain the descending order distribution of the selected band spectral image local feature vectors. That is, based on the set of band spectral image local feature neighborhood activities, the band spectral image local features that are most valuable for void detection are further screened out.In an embodiment of the present application, taking the local feature vector of the first-band spectral image as an example, the specific steps for feature selection of the descending order distribution of the local feature vector of the band spectral image are as follows: Extract the local feature neighborhood activity of the band spectral image of the local feature vector of the first-band spectral image from the set of local feature neighborhood activities of the band spectral image; Compare the local feature neighborhood activity of the band spectral image of the local feature vector of the first-band spectral image with a predetermined threshold; In response to the local feature neighborhood activity of the band spectral image of the local feature vector of the first-band spectral image being greater than or equal to the predetermined threshold, retain the local feature vector of the first-band spectral image; Otherwise, delete the local feature vector of the first-band spectral image. The features selected in this way can not only well represent the band spectral image data, but also better capture the feature distribution law in the band spectral image data. The descending order distribution of the selected local feature vector of the band spectral image will be more compact and more expressive, which is beneficial to improving the model performance and interpretability. In a specific example, the following feature selection formula is used to perform feature selection on the descending order distribution of the local feature vector of the band spectral image to obtain the descending order distribution of the selected local feature vector of the band spectral image; where, the feature selection formula is:

[0043]

[0044] Select({ 1 ,v 2 ,...,v k ,...,v n})={v 1 ’,v 2 ’,...,v k ’,...,v m ’}

[0045]

[0046] Wherein, represents the j-th position eigenvalue of the local feature vector of the band spectral image at the i-th position in the descending order distribution of the local feature vector of the band spectral image, L represents the number of eigenvalues of the local feature vector of the band spectral image at the i-th position, u i represents the band spectral image neighborhood feature factor corresponding to the local feature vector of the band spectral image at the i-th position, u i+1 represents the band spectral image neighborhood feature factor corresponding to the local feature vector of the band spectral image at the i + 1-th position, t i represents the local feature neighborhood activity of the band spectral image corresponding to the local feature vector of the band spectral image at the i-th position, Select(·) is the feature selection process, v 1 ’,v2 ’, v k ’, v m ’ respectively represent the 1st, 2nd, k-th, and m-th selected local feature vectors of the band spectral image after selection in the descending order distribution of the local feature vectors of the band spectral image after selection, and τ is a predetermined threshold.

[0047] Furthermore, feature reshaping is performed on each selected local feature vector of the band spectral image after selection in the descending order distribution to obtain a set of the encoded feature maps of the sparsified band spectral image. Considering that in the technical solution of this application, the descending order distribution of the selected local feature vectors of the band spectral image cannot be directly used for subsequent tasks such as semantic segmentation. For the convenience of further processing and analysis, it is necessary to perform feature reshaping on each selected local feature vector of the band spectral image in the descending order distribution. During this process, each selected local feature vector of the band spectral image is mapped to the corresponding position according to the coordinate information in the original multi-spectral image, so as to reconstruct a feature map that matches the original input format. The encoded feature map of the sparsified band spectral image obtained in this way not only retains the advantage of removing redundant information in the feature selection stage, but also can accurately reflect the spatial structure information of the original image, and further can more accurately locate the bare areas on the grassland, realizing an efficient grass seeding operation. In a specific example, the following feature reshaping formula is used to perform feature reshaping on each selected local feature vector of the band spectral image in the descending order distribution to obtain a set of the encoded feature maps of the sparsified band spectral image; where, the feature reshaping formula is:

[0048] F s = reshape{v 1 ’, v 2 ’,..., v k ’,..., v m ’}

[0049] where reshape(·) is the feature shape reshaping process, and F s represents the encoded feature map of the sparsified band spectral image.

[0050] Specifically, in S34, the set of sparsified band spectral image coding features is fused to obtain the sparsified spectral image full-band feature concatenated coding features. In the technical solution of this application, the set of sparsified band spectral image coding feature maps is concatenated to obtain the sparsified spectral image full-band feature concatenated coding feature map. Considering that each sparsified band spectral image coding feature map in the set of sparsified band spectral image coding feature maps represents the fine-grained features most representative of the detection of void regions within a specific band, however, the information provided by each band has unique value and complementarity, and processing each band separately cannot fully utilize this complementarity. To maximize the utilization of information in the multispectral data, the set of sparsified band spectral image coding feature maps is concatenated to obtain a comprehensive feature representation of all bands in the multispectral data, which can then more comprehensively reflect the grassland coverage, providing a solid foundation for subsequent semantic segmentation and promoting deeper learning and better generalization ability.

[0051] Specifically, in S35, semantic segmentation is performed on the sparsified spectral image full-band feature concatenated coding features to achieve the detection and segmentation of grassland void regions. That is, semantic segmentation is used to clearly distinguish healthy grasslands from bare or degraded areas, providing support for subsequent generation of effective reseeding prescription data.

[0052] Preferably, in one example, when performing semantic segmentation on the sparsified spectral image full-band feature concatenated coding feature map to achieve the detection and segmentation of grassland void regions, feature distribution optimization is performed on the sparsified spectral image full-band feature concatenated coding feature map, including:

[0053] Arrange the respective feature values in the sparsified spectral image full-band feature concatenated coding feature map in ascending order to obtain the sparsified spectral image full-band feature concatenated coding feature set;

[0054] In response to the absolute value of the difference between the i-th feature value and the (i + 1)-th feature value in the sparsified spectral image full-band feature concatenated coding feature set being less than or equal to the distance difference hyperparameter ε, that is, |f i+1 -f i | ≤ ε, calculate the weighted sum between the i-th feature value and the (i + 1)-th feature value as the optimized (i + 1)-th feature value;

[0055] Calculate the square root of the sum of the squares of all feature values in the sparsified spectral image full-band feature concatenated coding feature set, and multiply the square root by 2 and then divide by the square of the scale of the sparsified spectral image full-band feature concatenated coding feature map to obtain the medical image shallow-deep aggregation perception coding space basis value:

[0056]

[0057] β = 2α / S 2

[0058] Wherein, the scale S of the sparsified spectral image full-band feature cascaded coding feature map is equal to the width of the feature matrix of the sparsified spectral image full-band feature cascaded coding feature map multiplied by the height and then multiplied by the number of channels of the sparsified spectral image full-band feature cascaded coding feature map;

[0059] In response to the absolute value of the difference between the i-th eigenvalue and the (i + 1)-th eigenvalue in the set of sparsified spectral image full-band feature cascaded coding features being greater than the distance difference hyperparameter ε, that is, |f i+1 - f i | > ε, multiply the medical image shallow-deep aggregation perception coding space primitive value by the i-th eigenvalue, and calculate the weighted subtraction between the product and the (i + 1)-th eigenvalue to obtain the optimized (i + 1)-th eigenvalue;

[0060] Combine the optimized (i + 1)-th eigenvalues of the set of sparsified spectral image full-band feature cascaded coding features to obtain an optimized sparsified spectral image full-band feature cascaded coding feature map, wherein the minimum eigenvalue of the set of sparsified spectral image full-band feature cascaded coding features remains unchanged; and

[0061] Here, when each sparsified band spectral image coding feature map in the set of sparsified band spectral image coding feature maps respectively represents the spectral sparsification feature of the spectral image data of each band, when cascading the set of sparsified band spectral image coding feature maps to achieve feature aggregation, due to feature sparsification, the redundancy between features will be reduced, and simple feature cascading cannot utilize the complementary information between features, resulting in insufficient long-distance aggregation perception representation of the sparsified spectral image full-band feature cascaded coding feature map, thereby reducing the expression effect of the sparsified spectral image full-band feature cascaded coding feature map and affecting the accuracy of its semantic segmentation.

[0062] Therefore, for the problem of insufficient global aggregation perception representation ability caused by long distances exceeding the predetermined local distribution interval threshold in the feature set of the sparsified spectral image full-band feature cascade coding feature map under the predetermined eigenvalue order distribution, the high-dimensional feature space primitive representation based on self-inner product fusion of the sparsified spectral image full-band feature cascade coding feature map is used to capture the complex structure of the eigenvalue global network interaction, so as to reconstruct the aggregation perception relationship between the eigenvalues of the sparsified spectral image full-band feature cascade coding feature map by simulating the scale-based high-dimensional feature space potential primitive, so as to realize the coding reconstruction of the true sequence distribution behavior of the sparsified spectral image full-band feature cascade coding feature map at long distances, improve the aggregation perception expression effect of the sparsified spectral image full-band feature cascade coding feature map, and improve the accuracy of its semantic segmentation.

[0063] Furthermore, after the detection and segmentation of the grassland bare patch area are completed, seeding prescription data is generated based on the exposure degree, coordinate position, and size information of the grassland bare patch area. The seeding prescription data is a set of detailed instructions or parameter sets customized for the grassland health status of a specific area and used to guide precise forage seeding. The seeding prescription data is based on in-depth analysis of the multispectral image data of the target pasture, especially the results obtained through the precise detection and segmentation of grassland bare patches (i.e., bare ground or degraded areas). These results include key indicators such as the specific coordinate position, size information, and exposure degree of the bare patch area. By generating the seeding prescription data, it is possible to provide a scientific basis and technical support for subsequent forage seeding work, ensure that the seeding activities can be carried out efficiently and accurately, and thus promote the restoration and stability of the grassland ecosystem.

[0064] Specifically, in S4, the seeding prescription data is input into a plant protection UAV to complete the forage seeding of the target pasture by the plant protection UAV. In one example, first, the seeding prescription data is imported into the control system of the plant protection UAV to upload all relevant parameters including geographical coordinates, bare patch size, exposure degree assessment, soil conditions, and climate factors to the flight control platform of the UAV. Then, the plant protection UAV will start flying according to the preset path and parameters to carry out precise seeding and fertilization work.

[0065] In summary, the forage reseeding method based on multispectral imaging of unmanned aerial vehicles according to the embodiment of the present application is explained, which uses a multispectral camera equipped with an unmanned aerial vehicle to shoot the target pasture according to the planned route to obtain detailed multispectral image data, and uses image coding technology based on deep learning to split and encode the multispectral image data to mine the spatial structure information of the empty spot area in each band in the multispectral image data, so as to enhance the expression ability of each band feature, and further perform feature selection on the features in each band to extract the most representative fine-grained features for the detection of empty spot areas in the band, and realize the detection and segmentation of grassland empty spot areas based on these features. In this way, through the effective encoding and feature fusion of band spectral image data, the accuracy of grassland empty spot area detection is improved, thereby improving the accuracy and effectiveness of forage reseeding.

[0066] Furthermore, a forage reseeding method based on unmanned aerial vehicle multispectral imaging is also provided.

[0067] Figure 5 FIG. 1 is a block diagram of a forage reseeding system based on multispectral imaging of unmanned aerial vehicles according to an embodiment of the present application. Figure 5 As shown, according to the embodiment of the present application, the forage reseeding system 300 based on multispectral imaging of unmanned aerial vehicles includes: a route planning module 310, which is used to plan the route of the multispectral unmanned aerial vehicle according to the geographical environment information of the target pasture on that day; a multispectral image data acquisition module 320, which is used to acquire the multispectral image data of the target pasture through a multispectral camera deployed on the multispectral unmanned aerial vehicle; a spot area detection and segmentation module 330, which is used to process the multispectral image data to realize the detection and segmentation of grassland spot areas, and generate reseeding prescription data based on the exposure degree, coordinate position and size information of the grassland spot areas; a forage reseeding module 340, which is used to input the reseeding prescription data into the plant protection unmanned aerial vehicle so that the plant protection unmanned aerial vehicle can complete the forage reseeding of the target pasture.

[0068] As described above, the forage reseeding system 300 based on multispectral imagery of unmanned aerial vehicles according to the embodiment of the present application can be implemented in various wireless terminals, such as a server having a forage reseeding algorithm based on multispectral imagery of unmanned aerial vehicles. In a possible implementation, the forage reseeding system 300 based on multispectral imagery of unmanned aerial vehicles according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the forage reseeding system 300 based on multispectral imagery of unmanned aerial vehicles can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the forage reseeding system 300 based on multispectral imagery of unmanned aerial vehicles can also be one of the many hardware modules of the wireless terminal.

[0069] Alternatively, in another example, the forage reseeding system 300 based on UAV multispectral images and the wireless terminal can also be separate devices, and the forage reseeding system 300 based on UAV multispectral images can be connected to the wireless terminal through a wired and / or wireless network and transmit interaction information in accordance with a predefined data format.

[0070] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to technologies in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for reseeding pasture based on multispectral images of unmanned aerial vehicles, comprising the following steps: planning the route of a multispectral unmanned aerial vehicle according to the geographical environment information of a target pasture on that day; acquiring multispectral image data of the target pasture by a multispectral camera deployed on the multispectral unmanned aerial vehicle; processing the multispectral image data to detect and segment grassland empty spots, and generating reseeding prescription data based on the exposure degree, coordinate position and size information of the grassland empty spots; and inputting the reseeding prescription data into a plant protection unmanned aerial vehicle so that the plant protection unmanned aerial vehicle completes the reseeding of pasture on the target pasture, characterized in that: Processing the multispectral image data to detect and segment grassland spot areas includes: Performing data splitting on the multispectral image data based on the bands to obtain a set of band spectral image data; Extracting band spectral image features from the set of band spectral image data to obtain a set of band spectral image coding features; Performing spectral feature selection on each band spectral image coding feature in the set of band spectral image coding features to obtain a set of sparse band spectral image coding features; Fusion of the set of sparse band spectral image coding features to obtain sparse spectral image full band feature cascade coding features; Semantic segmentation is performed on the full-band feature cascade coding features of the sparse spectral image to achieve detection and segmentation of grassland spot areas.

2. The forage reseeding method based on multispectral imaging of unmanned aerial vehicles according to claim 1 is characterized in that: Extracting band spectral image features from the set of band spectral image data to obtain a set of band spectral image coding features includes: A dilated convolutional neural network model is used to perform dilated convolution encoding on each band spectral image data in the set of band spectral image data to obtain a set of band spectral image encoding feature maps as the set of band spectral image encoding features.

3. The forage reseeding method based on multispectral imaging of unmanned aerial vehicles according to claim 2 is characterized in that: Performing spectral feature selection on each band spectral image coding feature in the set of band spectral image coding features to obtain a set of sparse band spectral image coding features, including: Performing feature decoupling and feature flattening on each band spectral image coding feature map in the set of band spectral image coding feature maps to obtain a set of band spectral image coding local feature vectors; Calculating the local feature importance score of each band spectral image encoding local feature vector in the set of band spectral image encoding local feature vectors to obtain a set of band spectral image local feature importance score values; Based on the set of importance score values ​​of the local features of the band spectral image, feature selection and feature shape reshaping are performed on the set of local feature vectors of the band spectral image encoding to obtain a set of sparse band spectral image encoding feature maps as a set of sparse band spectral image encoding features.

4. The method for reseeding pasture based on multispectral imaging of unmanned aerial vehicles according to claim 3, characterized in that: Calculating the local feature importance score of each band spectral image encoding local feature vector in the set of band spectral image encoding local feature vectors to obtain a set of band spectral image local feature importance score values, including: Each band spectral image encoding local feature vector in the set of band spectral image encoding local feature vectors is input into an importance measurement module to obtain a set of importance score values ​​of the band spectral image local features.

5. The forage reseeding method based on multispectral imaging of unmanned aerial vehicles according to claim 4 is characterized in that: Based on the set of importance score values ​​of the local features of the band spectral image, feature selection and feature shape reshaping are performed on the set of local feature vectors of the band spectral image coding to obtain a set of sparse band spectral image coding feature maps as a set of sparse band spectral image coding features, including: Based on the set of importance score values ​​of the local features of the band spectral image, the set of the encoded local feature vectors of the band spectral image is arranged in descending order to obtain a descending distribution of the local feature vectors of the band spectral image; Based on the neighborhood features of each band spectral image local feature vector in the descending distribution of the band spectral image local feature vector, feature selection is performed on the descending distribution of the band spectral image local feature vector to obtain a selected descending distribution of the band spectral image local feature vector; The features of each selected band spectral image local feature vector in the descending distribution of the selected band spectral image local feature vector are reshaped to obtain a set of the sparse band spectral image encoding feature maps.

6. The method for reseeding pasture based on multispectral imaging of unmanned aerial vehicles according to claim 5, characterized in that: Based on the neighborhood features of each band spectral image local feature vector in the descending distribution of the band spectral image local feature vector, feature selection is performed on the descending distribution of the band spectral image local feature vector to obtain the descending distribution of the selected band spectral image local feature vector, including: Based on the neighborhood features of the local feature vectors of the band spectral images in the descending distribution of the local feature vectors of the band spectral images, the feature neighborhood activity of the local feature vectors of the band spectral images is calculated to obtain a set of the local feature neighborhood activity of the band spectral images; Based on the set of neighborhood activity of the local features of the band spectral image, feature selection is performed on the descending distribution of the local feature vectors of the band spectral image to obtain the descending distribution of the selected local feature vectors of the band spectral image.

7. The method for reseeding pasture based on multispectral imaging of unmanned aerial vehicles according to claim 6, characterized in that: Based on the neighborhood features of the local feature vectors of the band spectral images in the descending distribution of the local feature vectors of the band spectral images, the feature neighborhood activity of the local feature vectors of the band spectral images is calculated to obtain a set of the local feature neighborhood activity of the band spectral images, including: Extracting a first band spectral image local feature vector and a second band spectral image local feature vector from the descending distribution of the band spectral image local feature vectors; Calculating the band spectral image neighborhood characteristic factors of the local characteristic vector of the first band spectral image and the local characteristic vector of the second band spectral image to obtain the first band spectral image neighborhood characteristic factor and the second band spectral image neighborhood characteristic factor; The neighborhood characteristic factor of the second-band spectral image is divided by the neighborhood characteristic factor of the first-band spectral image, and the obtained quotient is used as the band spectral image local characteristic neighborhood activity of the local characteristic vector of the first-band spectral image.

8. The method for reseeding pasture based on multispectral imaging of unmanned aerial vehicles according to claim 7, characterized in that: Based on the set of neighborhood activity of the local features of the band spectral image, feature selection is performed on the descending distribution of the local feature vectors of the band spectral image to obtain the descending distribution of the selected local feature vectors of the band spectral image, including: Extracting the band spectral image local feature neighborhood activity of the first band spectral image local feature vector from the set of the band spectral image local feature neighborhood activity; Comparing the neighborhood activity of the local feature vector of the first band spectral image with a predetermined threshold; In response to the band spectral image local feature neighborhood activity of the first band spectral image local feature vector being greater than or equal to the predetermined threshold, the first band spectral image local feature vector is retained; otherwise, the first band spectral image local feature vector is deleted.

9. The method for reseeding pasture based on multispectral imaging of unmanned aerial vehicles according to claim 8, characterized in that: The set of sparse band spectral image coding features is fused to obtain sparse spectral image full band feature cascade coding features, including: The set of the sparse band spectral image coding feature maps is cascaded to obtain a sparse spectral image full-band feature cascade coding feature map as the sparse spectral image full-band feature cascade coding feature.

10. A forage reseeding system based on multispectral imaging of unmanned aerial vehicles, characterized in that: include: The route planning module is used to plan the route of the multispectral UAV based on the geographical environment information of the target ranch on that day; A multispectral image data acquisition module, used for acquiring multispectral image data of the target pasture through a multispectral camera deployed on the multispectral UAV; A spot area detection and segmentation module is used to process the multispectral image data to detect and segment grassland spot areas, and generate reseeding prescription data based on the exposure degree, coordinate position and size information of the grassland spot areas; The forage reseeding module is used to input the reseeding prescription data into the plant protection drone so that the plant protection drone can complete the forage reseeding of the target pasture.

Images