Characteristic crop identification method and system cooperating with satellite-borne and airborne hyperspectral remote sensing

Abstract

The invention relates to a characteristic crop identification method and system cooperating with satellite-borne and airborne hyperspectral remote sensing, and relates to the technical field of characteristic crop identification remote sensing. The method comprises the following steps: determining a target research area, and obtaining a hyperspectral image data set of the target research area, the hyperspectral image data set comprises airborne hyperspectral image data and satellite-borne hyperspectral image data, carrying out spectral curve extraction processing according to the hyperspectral image data set to obtain a ground feature spectral curve set, carrying out adaptive matching according to the ground feature spectral curve set to obtain an identification sensitive wave band with high sensitivity to the identification of characteristic crops, and carrying out identification on the characteristic crops according to the identification sensitive wave band. According to the method, the identification sensitive wave band is identified and classified through the preset characteristic crop identification model to obtain the characteristic crop classification and identification result, so that the characteristic crops can be effectively identified, high-precision classification and identification of the ground features including the characteristic crops are realized, and the ground feature identification precision is improved.

Description

Technical Field

[0001] The present application relates to the field of remote sensing technology for identifying characteristic crops, and in particular to a method and system for identifying characteristic crops by coordinating satellite-borne and airborne hyperspectral remote sensing. Background Art

[0002] At present, specialty crops fully reflect the natural resources and geographical advantages of a region. Accurately obtaining information on the planting area and yield of specialty crops is of great significance to optimizing the planting structure of specialty crops, scientifically formulating agricultural policies, and promoting economic development.

[0003] The existing crop recognition method based on multispectral images mainly uses spectral images acquired by satellites for recognition, and has been widely used in crop recognition. However, due to the limitation of spectral resolution, the classification accuracy of crop recognition based on multispectral images is not high in areas with complex crop planting structures and fragmented plots. It is easy to have the phenomenon of different objects with the same spectrum, and it is difficult to achieve effective recognition of characteristic crops. It can be seen that the existing crop recognition technology based on multispectral images cannot effectively identify characteristic crops due to factors such as the environment and crop structure, and the classification accuracy of characteristic crops is low. Summary of the invention

[0004] The present application provides a method and system for identifying characteristic crops by coordinating satellite-borne and airborne hyperspectral remote sensing, which combines airborne hyperspectral images with satellite-borne hyperspectral images to jointly perform high-precision classification and identification of ground objects including characteristic crops in the region, and can effectively identify characteristic crops and improve the accuracy of ground object identification. At the same time, it has the ability of large-scale monitoring, and effectively solves the problems of the inability to effectively identify characteristic crops and the low accuracy of identification and classification in the prior art.

[0005] In a first aspect, the present application provides a method for identifying characteristic crops by coordinating satellite-borne and airborne hyperspectral remote sensing, comprising:

[0006] Identify the target study area;

[0007] Acquire a hyperspectral image dataset and a field sample dataset of the target study area, wherein the hyperspectral image dataset includes airborne hyperspectral image data and satellite-borne hyperspectral image data, and the field sample dataset includes ground object samples;

[0008] Performing spectral curve extraction processing based on the hyperspectral image data set and the field sample data set to obtain a ground object spectral curve set;

[0009] Adaptively matching the ground feature spectral curve set to obtain a sensitive band for identifying characteristic crops, wherein the sensitive band is a band that the satellite-borne hyperspectral image data is highly sensitive to for identifying the characteristic crops;

[0010] The identification sensitive bands are identified and classified by a preset characteristic crop identification model to obtain a characteristic crop classification identification result;

[0011] The airborne hyperspectral image data is spectral image data acquired by a drone, and the satellite-borne hyperspectral image data is spectral image data acquired by a satellite.

[0012] Optionally, the step of obtaining a hyperspectral image dataset and a field sample dataset of the target research area includes:

[0013] The target research area is imaged by a preset drone to obtain a drone hyperspectral image containing the characteristic crops, wherein the drone carries a hyperspectral sensor;

[0014] Perform correction and splicing processing based on the UAV hyperspectral image to obtain the airborne hyperspectral image data;

[0015] The target research area is imaged by a preset satellite to obtain a satellite-borne hyperspectral image;

[0016] Image preprocessing is performed based on the satellite-borne hyperspectral image to obtain high-quality satellite-borne hyperspectral image data, and a field sample data set is extracted from a preset database.

[0017] Optionally, performing spectral curve extraction processing based on the hyperspectral image dataset and the field sample dataset to obtain a ground object spectral curve set includes:

[0018] Determine an area of ​​interest based on the field sample data set, wherein the area of ​​interest includes at least one type of ground feature;

[0019] For the region of interest, combined with the field sample data set, the ground object spectrum is extracted from the airborne hyperspectral image data and the satellite-borne hyperspectral image data respectively to obtain an average spectrum curve set, and the average spectrum curve set is used as the ground object spectrum curve set;

[0020] The ground object spectrum curve set includes an airborne ground object spectrum curve corresponding to the airborne hyperspectral image data and a satellite-borne ground object spectrum curve corresponding to the satellite-borne hyperspectral image data.

[0021] Optionally, the adaptive matching is performed according to the ground feature spectrum curve set to obtain the identification sensitive band of the characteristic crop, including:

[0022] The airborne ground feature spectral curve is taken as a true value, and resampled based on the airborne hyperspectral image data according to a preset resolution set to obtain a spectral curve of the characteristic crop corresponding to each resolution in the resolution set;

[0023] In the resolution set, a spectral angle calculation is performed based on the airborne ground object spectral curve and the satellite-borne ground object spectral curve to obtain a spectral angle match;

[0024] The identification sensitive band is obtained by performing similarity analysis based on the spectral angle matching in combination with the spectral curve.

[0025] Optionally, performing spectral angle calculation based on the airborne ground object spectral curve and the satellite-borne ground object spectral curve to obtain spectral angle matching includes:

[0026] Taking the spectral response curves of each object in the airborne ground object spectral curve and the satellite-borne ground object spectral curve as a space vector;

[0027] According to the formula and Calculating the generalized angle θ between the space vectors;

[0028] The generalized angle θ is proportional to the similarity of the spectral curve. and They represent the vectors of two ground object spectral curves respectively, and Similarity represents the similarity of the two ground object spectral curves.

[0029] Optionally, the identifying sensitive bands are identified and classified by using a preset characteristic crop identification model to obtain characteristic crop classification identification results, including:

[0030] Extracting features of the identification sensitive bands through the characteristic crop identification model to obtain spectral features, spatial features and spatial-spectral joint features;

[0031] Perform feature fusion based on the spectral feature, the spatial feature and the spatial-spectral joint feature to obtain fused feature information;

[0032] The characteristic crop identification model performs classification mapping based on the fused feature information to obtain a characteristic crop classification identification result, and the characteristic crop classification identification result includes a characteristic crop classification map of the target research area.

[0033] Optionally, before identifying and classifying the identification sensitive bands by using a preset characteristic crop identification model, the method further includes:

[0034] Acquire sample data of a sample study area, wherein the sample data includes airborne hyperspectral image samples, satellite-borne hyperspectral image samples, and ground feature samples;

[0035] For the airborne hyperspectral image samples and the satellite-borne hyperspectral image samples, respectively extracting ground object spectral response curve samples corresponding to the ground object samples;

[0036] Determine, by a preset spectral curve adaptive matching algorithm, based on the spectral response curve samples of the ground objects, the identification sensitive band samples of the ground object samples in the satellite-borne hyperspectral image samples;

[0037] Constructing a three-branch hybrid convolutional neural network based on a convolutional module, and constructing a deep learning model based on the three-branch hybrid convolutional neural network;

[0038] Through the three-branch hybrid convolutional neural network, model feature extraction training is performed based on the sample data and the identified sensitive band samples to obtain feature samples, wherein the feature samples include spectral feature samples, spatial feature samples, and space-spectrum joint feature samples;

[0039] Constructing a training set and a test set based on the feature samples, the sample data and the identified sensitive band samples;

[0040] The deep learning model is trained using the training set to obtain the characteristic crop recognition model.

[0041] Optionally, the three-branch hybrid convolutional neural network is used to perform model feature extraction training based on the sample data and the identified sensitive band samples to obtain feature samples, including:

[0042] Constructing a spectral feature extraction branch, a spatial feature extraction branch and a space-spectrum joint feature extraction branch in the three-branch hybrid convolutional neural network;

[0043] Extracting local information in the spectral dimension from the sample data and the identified sensitive band samples through the spectral feature extraction branch to obtain a spectral feature sample;

[0044] Through the spatial feature extraction branch, image spatial feature extraction is performed on the sample data and the identified sensitive band samples to obtain spatial feature samples;

[0045] Through the space-spectrum joint feature extraction branch, the sample data and the identification sensitive band samples are extracted end-to-end to obtain a space-spectrum joint feature sample;

[0046] Among them, the spectral feature extraction branch includes a one-dimensional convolution layer, a pooling layer and a fully connected layer, the spatial feature extraction branch includes a two-dimensional convolution layer, a pooling layer and a fully connected layer, and the spatial-spectral joint feature extraction branch includes a three-dimensional convolution layer, a two-dimensional convolution layer, a pooling layer and a fully connected layer. The three-dimensional convolution layer is used to slide simultaneously between the spatial and spectral dimensions to meet the requirements of multi-scale and multi-resolution.

[0047] Optionally, after obtaining the characteristic crop identification model, the method further includes:

[0048] Obtaining the sample ground feature recognition result output by the characteristic crop recognition model for the sample data, and the sub-meter level image corresponding to the training set;

[0049] The sample ground feature recognition results are verified by using preset accuracy evaluation indicators, performing accuracy tests using the test set corresponding to the sample data, and visually interpreting and judging the texture information contained in the sub-meter image.

[0050] In a second aspect, the present application provides a characteristic crop identification system, comprising:

[0051] A target study area determination module is used to determine the target study area;

[0052] A data set acquisition module, used to acquire a hyperspectral image data set and a field sample data set of the target study area, wherein the hyperspectral image data set includes airborne hyperspectral image data and satellite-borne hyperspectral image data, and the field sample data set includes ground object samples;

[0053] A spectral curve extraction module is used to perform spectral curve extraction processing based on the hyperspectral image data set and the field sample data set to obtain a ground object spectral curve set;

[0054] An adaptive matching module, used for adaptively matching according to the ground object spectral curve set to obtain a sensitive band for identifying characteristic crops, wherein the sensitive band is a band that the satellite-borne hyperspectral image data has a high sensitivity to identifying the characteristic crops;

[0055] An identification and classification module is used to identify and classify the identification sensitive bands through a preset characteristic crop identification model to obtain a characteristic crop classification identification result;

[0056] The airborne hyperspectral image data is spectral image data acquired by a drone, and the satellite-borne hyperspectral image data is spectral image data acquired by a satellite.

[0057] In summary, the embodiment of the present application determines the target study area, obtains the airborne hyperspectral image data of the target study area through a drone, and obtains the satellite-borne hyperspectral image data of the target study area through a satellite, obtains a hyperspectral image data set, and then performs spectral curve extraction and processing according to the hyperspectral image data set to obtain a ground object spectral curve set, and adaptively matches according to the ground object spectral curve set to obtain the identification sensitive bands of the satellite-borne hyperspectral image data with high sensitivity to the identification of characteristic crops, and then identifies and classifies the identification sensitive bands through a preset characteristic crop identification model to obtain the classification and identification results of characteristic crops. It can be seen that the present application combines airborne hyperspectral images and satellite-borne hyperspectral images, which can effectively combine the advantages of the two hyperspectral images, and use the two hyperspectral images to jointly classify and identify the ground objects including characteristic crops in the region with high precision, which can effectively identify characteristic crops, and at the same time improve the ground object identification accuracy, it has the ability of large-scale monitoring, and effectively solves the problems of the inability to effectively identify characteristic crops and the low identification and classification accuracy existing in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0060] Figure 1 A flowchart of a method for identifying characteristic crops by coordinating satellite-borne and airborne hyperspectral remote sensing provided in an embodiment of the present application;

[0061] Figure 2 This is a schematic diagram of the steps of a method for identifying characteristic crops by coordinating satellite-borne and airborne hyperspectral remote sensing, provided by an optional embodiment of the present application;

[0062] Figure 3 It is a hyperspectral remote sensing image provided by an optional example of this application;

[0063] Figure 4 It is a technical roadmap for a characteristic crop identification method provided by an optional example of this application;

[0064] Figure 5 It is a ground feature spectral response curve diagram provided by an optional example of the present application;

[0065] Figure 6It is a spectrum curve diagram of litchi at different spatial resolutions provided by an optional example of the present application;

[0066] Figure 7 This is an optional example of the present application providing a classification diagram of litchi in City X;

[0067] Figure 8 It is a structural diagram of a three-branch hybrid convolutional neural network provided by an optional example of the present application;

[0068] Fig. 9 It is a spectral feature extraction branch diagram of a three-branch hybrid convolutional neural network provided by an optional example of the present application;

[0069] Fig.10 It is a spatial feature extraction branch diagram of a three-branch hybrid convolutional neural network provided by an optional example of the present application;

[0070] Fig.11 It is a branch diagram of space-spectrum joint feature extraction of a three-branch hybrid convolutional neural network provided by an optional example of the present application;

[0071] Fig.12 A structural block diagram of a characteristic crop identification system provided in an embodiment of the present application;

[0072] Fig.13 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0074] To facilitate the understanding of the embodiments of the present application, further explanation will be given below in conjunction with the drawings and specific embodiments. The embodiments do not constitute a limitation on the embodiments of the present application.

[0075] Figure 1 The present invention provides a flowchart of a method for identifying special crops by coordinating satellite-borne and airborne hyperspectral remote sensing. Figure 1 As shown, the method for identifying characteristic crops by coordinating satellite-borne and airborne hyperspectral remote sensing provided in the embodiment of the present application may specifically include the following steps:

[0076] Step 110, determining the target study area.

[0077] In this embodiment, the target study area refers to an area containing characteristic crops. The target study area may also include various types of areas, such as construction land, cultivated land, forest land, water bodies, gardens, etc.

[0078] It should be noted that the embodiments of the present application do not restrict the size, topography, etc. of the target study area. A certain area can be selected as the target study area according to actual needs to identify characteristic crops in the target study area.

[0079] Step 120: Acquire a hyperspectral image dataset and a field sample dataset of the target study area.

[0080] The hyperspectral image data set includes airborne hyperspectral image data and satellite-borne hyperspectral image data, the field sample data set includes ground object samples, the airborne hyperspectral image data is spectral image data acquired by unmanned aerial vehicles, and the satellite-borne hyperspectral image data is spectral image data acquired by satellites.

[0081] Specifically, the spectral image data set includes hyperspectral remote sensing image data (referred to as hyperspectral remote sensing), wherein the airborne hyperspectral image data may include airborne hyperspectral remote sensing images (referred to as airborne hyperspectral remote sensing). In this embodiment, hyperspectral images of the target study area may be collected by unmanned aerial vehicles, etc., to obtain the airborne hyperspectral image data of the target study area; the satellite-borne hyperspectral image data includes satellite-borne hyperspectral remote sensing images (referred to as satellite-borne hyperspectral remote sensing). In this embodiment, hyperspectral images of the target study area may be collected by satellites, etc., to obtain the satellite-borne hyperspectral image data of the target study area.

[0082] In this embodiment, the field sample data set may include a variety of ground feature samples, and the ground feature samples may be understood as spectral response curve samples corresponding to the ground features. Among them, the ground features may include but are not limited to: specialty crops (including but not limited to lychee and other specialty crops) and various types of regions (including but not limited to: woodlands, gardens, cultivated land, construction land and water bodies, etc.), so the ground feature samples may also include but are not limited to: specialty crop samples and samples of various types of regions, which are not limited in this embodiment.

[0083] In this embodiment, hyperspectral remote sensing can obtain dozens or even hundreds of continuous narrow band information for each pixel in a spectral interval, and has the characteristics of high spectral resolution, unified graph and spectrum, and continuous imaging within a certain spectral range. This embodiment uses hyperspectral remote sensing to capture diagnostic spectral features of different crops and detect ground objects that were originally undetectable in multispectral remote sensing.

[0084] In a specific implementation, airborne hyperspectral remote sensing has the characteristics of high spatial resolution and high spectral resolution, and has many advantages such as strong real-time performance, little influence by clouds, and high accuracy in classification and identification of objects; satellite-borne hyperspectral remote sensing can conduct long-term, large-scale ground monitoring, and the data acquisition cost is low. The embodiment of the present application combines satellite-borne hyperspectral remote sensing and airborne hyperspectral remote sensing for the identification of characteristic crops, thereby making full use of the various advantages of airborne hyperspectral remote sensing and satellite-borne hyperspectral remote sensing. When performing the identification and classification of characteristic crops, it can effectively improve the classification and recognition accuracy of objects, and solve various problems existing in the existing crop identification methods based on multispectral images.

[0085] In the related technologies, both airborne hyperspectral remote sensing and satellite-borne hyperspectral remote sensing have many advantages, but also have certain disadvantages, such as small coverage area and high cost of airborne hyperspectral remote sensing, which is not suitable for large-scale monitoring; and satellite-borne hyperspectral remote sensing has low spatial resolution, and data quality is easily affected by clouds, which has limitations in fine classification and identification of objects. Therefore, the single use of a certain type of hyperspectral remote sensing to classify and identify objects has certain limitations. The embodiment of the present application combines airborne hyperspectral remote sensing and satellite-borne hyperspectral remote sensing, so that the two types of hyperspectral remote sensing can complement each other and make up for their respective shortcomings. Therefore, when using collaborative satellite-borne and airborne hyperspectral remote sensing for object identification, it is possible to obtain fine spatial and spectral information to improve the accuracy of object identification, and have the ability to monitor over a large range, solving the problems of low object identification accuracy and small monitoring range in the prior art.

[0086] Step 130 , performing spectral curve extraction processing based on the hyperspectral image dataset and the field sample dataset to obtain a ground object spectral curve set.

[0087] In this embodiment, the ground object spectral curve set includes ground object spectral curves corresponding to airborne hyperspectral remote sensing and satellite-borne hyperspectral remote sensing. The ground object spectral curve is also called ground object spectral response curve, including but not limited to spectral response curves corresponding to multiple ground objects in the target study area. For example, when the target study area includes characteristic crops and multiple types of regions, the ground object spectral curve may include spectral response curves corresponding to characteristic crops and spectral response curves corresponding to each type of region. This embodiment does not limit this. Among them, the ground object spectral curve corresponding to airborne hyperspectral remote sensing can be called an airborne ground object spectral response curve, and the ground object spectral curve corresponding to satellite-borne hyperspectral remote sensing can be called a satellite-borne ground object spectral response curve.

[0088] In actual implementation, this embodiment can perform spectral curve extraction processing on airborne hyperspectral remote sensing and satellite-borne hyperspectral remote sensing respectively, identify ground objects within the range according to airborne hyperspectral remote sensing and satellite-borne hyperspectral remote sensing respectively, use hyperspectral remote sensing to extract spectral curves of various characteristic crops and various types of regions in the current study area, and then use the field sample data set, combined with the spectral curves extracted from hyperspectral remote sensing for comparison, to determine the ground objects corresponding to the spectral curves, thereby obtaining the spectral response curve corresponding to each type of hyperspectral remote sensing including the spectral curves of each ground object, that is, obtaining the airborne ground object spectral response curve and the satellite-borne ground object spectral response curve, and combining the ground object spectral curves corresponding to the two types of hyperspectral remote sensing to obtain a ground object spectral curve set.

[0089] Step 140 , performing adaptive matching according to the ground feature spectrum curve set to obtain the identification sensitive bands of characteristic crops.

[0090] The identification sensitive band is a band in which the satellite-borne hyperspectral image data is highly sensitive to the identification of the characteristic crops.

[0091] In this embodiment, after the airborne ground object spectral response curve and the satellite-borne ground object spectral response curve are obtained, they can be compared according to the airborne ground object spectral response curve and the satellite-borne ground object spectral response curve. Based on the characteristics of airborne hyperspectral remote sensing with high spatial resolution, high spectral resolution, strong real-time performance, and little influence of clouds, the airborne ground object spectral response curve is first taken as the true value, and the airborne ground object spectral response curve and the satellite-borne ground object spectral response curve are subjected to spectral similarity analysis through adaptive matching of the spectral curves, so as to determine the band with high sensitivity to the identification of characteristic crops in satellite-borne hyperspectral remote sensing as the identification sensitive band of characteristic crops, thereby effectively filtering out other irrelevant ground objects.

[0092] Step 150 , classifying the identification sensitive bands using a preset characteristic crop identification model to obtain a characteristic crop classification identification result.

[0093] In this embodiment, the classification and identification results of characteristic crops may include but are not limited to: characteristic crops in the target study area and the categories corresponding to the characteristic crops. Of course, the identification results of areas such as construction land, cultivated land, forest land, water bodies, and gardens can also be retained in the classification and identification results according to needs. This embodiment does not impose any restrictions on this.

[0094] In a specific implementation, the embodiment of the present application can input the identification sensitive band into a pre-trained characteristic crop identification model, and perform multiple feature extraction on the identification sensitive band through the characteristic crop identification model, including but not limited to: spectral feature extraction, spatial feature extraction, and spatial-spectral joint feature extraction, etc., and then the characteristic crop identification model can perform identification and classification according to the multiple extracted features, so as to determine which characteristic crop the identification sensitive band specifically corresponds to. For example, when the characteristic crop is litchi, for the identification sensitive band corresponding to litchi, the characteristic crop identification model can identify the corresponding characteristic crop as litchi based on the identification sensitive band. Therefore, the present embodiment realizes the combination of airborne and satellite-borne hyperspectral remote sensing images to jointly identify characteristic crops in the region, obtain corresponding characteristic crop classification and identification results, and can effectively and accurately identify characteristic crops.

[0095] It can be seen that the embodiment of the present application determines the target study area, obtains the airborne hyperspectral image data of the target study area through the drone, and obtains the satellite-borne hyperspectral image data of the target study area through the satellite, obtains the hyperspectral image data set, and then performs spectral curve extraction and processing according to the hyperspectral image data set to obtain the ground object spectral curve set, and adaptively matches according to the ground object spectral curve set to obtain the identification sensitive bands of the satellite-borne hyperspectral image data with high sensitivity to the identification of characteristic crops, and then identifies and classifies the identification sensitive bands through the preset characteristic crop identification model to obtain the classification and identification results of characteristic crops. Therefore, the present application combines airborne hyperspectral images and satellite-borne hyperspectral images, which can effectively combine the advantages of the two hyperspectral images, and use the two hyperspectral images to jointly classify and identify the ground objects including characteristic crops in the region with high precision, which can effectively identify characteristic crops, and at the same time improve the ground object identification accuracy, it has the ability of large-scale monitoring, and effectively solves the problems of the inability to effectively identify characteristic crops and the low identification and classification accuracy existing in the prior art.

[0096] Reference Figure 2 , shows a schematic flow chart of the steps of a method for identifying characteristic crops by coordinating satellite-borne and airborne hyperspectral remote sensing, provided in an optional embodiment of the present application. The method may specifically include the following steps:

[0097] Step 210, determining the target study area.

[0098] For the description of step 210 , please refer to the above step 110 , which will not be described in detail in this embodiment.

[0099] Step 220: Acquire a hyperspectral image dataset and a field sample dataset of the target study area.

[0100] The hyperspectral image data set includes airborne hyperspectral image data and satellite-borne hyperspectral image data, the field sample data set includes ground object samples, the airborne hyperspectral image data is spectral image data acquired by unmanned aerial vehicles, and the satellite-borne hyperspectral image data is spectral image data acquired by satellites.

[0101] The description of step 220 can refer to the above step 120, which will not be described in detail in this embodiment.

[0102] In an optional embodiment, the embodiment of the present application obtains a hyperspectral image dataset and a field sample dataset of the target study area, which may specifically include: acquiring images of the target study area through a preset drone to obtain a drone hyperspectral image containing the characteristic crops, the drone carrying a hyperspectral sensor; performing correction and splicing processing based on the drone hyperspectral image to obtain the airborne hyperspectral image data; acquiring images of the target study area through a preset satellite to obtain a satellite-borne hyperspectral image; performing image preprocessing based on the satellite-borne hyperspectral image to obtain high-quality satellite-borne hyperspectral image data, and extracting a field sample dataset from a preset database.

[0103] In actual implementation, a hyperspectral sensor can be carried on a UAV to collect images of the target research area through the hyperspectral sensor to obtain a UAV hyperspectral image, which can then be corrected and stitched. The image correction process may include but is not limited to: radiation brightness correction, reflectivity correction and orthorectification. After completing the image correction, the corrected image can be stitched to obtain airborne hyperspectral image data.

[0104] In this embodiment, the satellite-borne hyperspectral image of the target study area obtained by the satellite may be distorted during the atmospheric radiation transmission of the remote sensing image due to the position of the sun, atmospheric conditions and sensor performance limitations. To solve this problem, after the satellite-borne hyperspectral image is obtained in the embodiment of the present application, the satellite-borne hyperspectral image can be further preprocessed. The preprocessing of the satellite-borne hyperspectral image mainly includes: radiation correction, atmospheric correction, orthorectification, etc. Before atmospheric correction, it is necessary to repair the stripe noise and remove the bands that are severely affected by noise and water vapor. Finally, the images are spliced ​​and masked according to the scope of the study area to obtain high-quality satellite-borne hyperspectral image data.

[0105] In actual processing, for the field sample data set, this embodiment can select samples of construction land, cultivated land, forest land, water bodies, gardens, etc. from multiple data sources (such as annual land change survey data). Preferably, the garden is selected as a garden sample of non-specialty crops, and through field surveys, special crop samples are obtained. In this way, multiple sample data are collected to obtain a field sample data set, which can be stored in a database for easy access.

[0106] Exemplarily, taking City X as the study area, for the acquisition of airborne hyperspectral image data, the drone of this example can use the DJI M600Pro drone, and carry the Nano-Hyperspec hyperspectral sensor of Headwall on the DJI M600Pro drone. Among them, the spectral range of the Nano-Hyperspec hyperspectral sensor is 400-1000nm, the spectral resolution is 6nm, and the number of spectral channels is 270. The drone hyperspectral image is obtained by taking drone aerial photos of the characteristic crops in the study area. Preferably, this example can use a study area with clear and cloudless weather to acquire data, and perform dark current correction and whiteboard correction on the sensor. The drone flight altitude can be 100 meters (m), the flight setting speed is 5m / s (seconds), and the image stitching processing software is the hyperspectral sensor built-in software SpectralViewHyperspecIII. The acquired UAV hyperspectral image data is corrected for radiation brightness, reflectivity and orthophoto, and multiple UAV strips are stitched to obtain the UAV hyperspectral orthophoto as airborne hyperspectral image data. Figure 3 -As shown by drone hyperspectral remote sensing imagery.

[0107] For the acquisition of airborne hyperspectral image data, the satellite used in this example can be Gaofen-5 (GF-5), where the number of bands of GF-5 hyperspectral image is 330, of which the resolution of the visible spectrum is 5nm, the resolution of the short-wave infrared spectrum is 10nm, the spectral range is 400-2500nm, and the spatial resolution is 30 meters. In order to eliminate and correct the impact of the image in the atmospheric radiation transmission, the image data obtained by GF-5 is pre-processed by radiation correction, stripe repair, removal of invalid bands, atmospheric correction, orthorectification, etc. Finally, the images are spliced ​​and masked according to the scope of the study area to obtain the satellite-borne hyperspectral image data, such as Figure 3 -Shown in satellite hyperspectral remote sensing imagery.

[0108] Step 230: determining a region of interest based on the field sample data set.

[0109] Wherein, the area of ​​interest includes at least one type of land feature.

[0110] Step 240 , for the region of interest combined with the field sample data set, respectively extract ground object spectra from the airborne hyperspectral image data and the satellite-borne hyperspectral image data to obtain an average spectrum curve set, and use the average spectrum curve set as the ground object spectrum curve set.

[0111] The ground object spectrum curve set includes an airborne ground object spectrum curve corresponding to the airborne hyperspectral image data and a satellite-borne ground object spectrum curve corresponding to the satellite-borne hyperspectral image data.

[0112] A unified description of steps 230 to 240 is given as follows:

[0113] In this embodiment, the field sample data set includes samples of various categories of ground objects. According to these categories of ground objects, this embodiment can use them as regions of interest (ROI), that is, the regions of interest include but are not limited to: water bodies, buildings, characteristic crops, cultivated land, gardens and other categories. Subsequently, for the regions of interest and the field sample data set, the average spectral curves corresponding to the ground objects of each category can be extracted from the airborne hyperspectral image data and the satellite-borne hyperspectral image data respectively as the ground object spectral curves, thereby obtaining the airborne ground object spectral curves corresponding to the airborne hyperspectral image data and the satellite-borne ground object spectral curves corresponding to the satellite-borne hyperspectral image data.

[0114] For example, take litchi as a specialty crop and combine it with various types of landforms. Figure 4 The technical roadmap of the method for identifying characteristic crops by cooperating satellite-borne and airborne hyperspectral remote sensing is provided. This example extracts the corresponding region of interest from the characteristic crop samples, and then uses the image processing library in Python to automatically extract the average spectral curve of each category from the preprocessed satellite-borne hyperspectral data and drone hyperspectral data for the region of interest in combination with the characteristic crop samples. The average spectral curve is used as the spectral response curve of each ground object, where the ground object spectral response curve can be as follows Figure 5 shown.

[0115] Step 250 , taking the airborne ground feature spectral curve as a true value, and resampling based on the airborne hyperspectral image data according to a preset resolution set, to obtain a spectral curve of the characteristic crop corresponding to each resolution in the resolution set.

[0116] In this embodiment, after obtaining the spectral curves of the airborne and satellite-borne hyperspectral images, the spectral data of the airborne and satellite-borne objects can be compared, and the spectral curve of the airborne hyperspectral image object can be used as the true value to determine the sensitive band of the satellite-borne hyperspectral image for identifying characteristic crops through the spectral curve adaptive matching algorithm, so as to realize the line Figure 4 shown.

[0117] In actual implementation, this embodiment can preset multiple resolutions to obtain a resolution set. According to the different resolutions in the resolution set, the airborne hyperspectral image is resampled at different resolutions, and then combined with the satellite-borne hyperspectral image for calculation to obtain the spectral response curves of the airborne hyperspectral image and the satellite-borne hyperspectral image for characteristic crops at different resolutions.

[0118] For example, litchi is used as a specialty crop, referring to Figure 6 By sampling and analyzing the airborne hyperspectral images at different resolutions, the spectral response curves of litchi at different spatial resolutions were obtained. Figure 6 Middle: 0.01m, 0.1m, 1m, 2m, 5m and 10m correspond to different resolutions, and the spectral response curves of litchi at different resolutions may be different.

[0119] Step 260 , within the resolution set, spectral angle calculation is performed based on the airborne ground object spectral curve and the satellite-borne ground object spectral curve to obtain spectral angle matching.

[0120] In actual implementation, this embodiment calculates the spectral angles of the onboard ground object spectral curve and the satellite-borne ground object spectral curve for each resolution in the resolution set to obtain the spectral angle match (SAM), so that the spectral similarity analysis can be performed through the spectral angle matching to determine the sensitive bands of the satellite-borne hyperspectral image for the recognition of characteristic crops.

[0121] In an optional embodiment, the embodiment of the present application performs spectral angle calculation based on the airborne ground object spectral curve and the satellite-borne ground object spectral curve to obtain spectral angle matching, which may specifically include: using the spectral response curves of each object in the airborne ground object spectral curve and the satellite-borne ground object spectral curve as a spatial vector; according to the formula and Calculate the generalized angle θ between the space vectors; wherein the generalized angle θ is proportional to the similarity of the spectral curves, and They represent the vectors of two ground object spectral curves respectively, and Similarity represents the similarity of the two ground object spectral curves.

[0122] In actual implementation, this embodiment uses the spectral response curves of various objects in the image as vectors in n-dimensional space, and uses a formula to calculate the generalized angle between the vectors, wherein the smaller the angle, the more similar the spectra. Preferably, in this embodiment, when the similarity value of two spectral curves is 1, it indicates that the two spectra are exactly the same; when the Similarity value is 0, it indicates that the two spectra are completely dissimilar.

[0123] Step 270: Perform similarity analysis based on the spectral angle matching and the spectral curve to obtain the identification sensitive band.

[0124] In a specific implementation, for spectral curves, this embodiment performs spectral similarity analysis through spectral angle matching. The angle corresponding to the spectral angle matching is inversely proportional to the similarity between the spectral curves. Preferably, in this embodiment, the smaller the angle corresponding to the spectral angle matching, the more similar the two spectral curves are, thereby determining the sensitive bands for the identification of characteristic crops by satellite-borne hyperspectral images, providing a reference for the identification of characteristic crops in the next step.

[0125] Step 280 , classifying the identification sensitive bands using a preset characteristic crop identification model to obtain a characteristic crop classification identification result.

[0126] Optionally, the above-mentioned identifying and classifying the identification sensitive bands by using a preset characteristic crop identification model to obtain the characteristic crop classification identification result may specifically include the following sub-steps:

[0127] Sub-step 2801, extracting features of the identification sensitive bands through the characteristic crop identification model to obtain spectral features, spatial features and spatial-spectral joint features.

[0128] Sub-step 2802, performing feature fusion based on the spectral feature, the spatial feature and the spatial-spectral joint feature to obtain fused feature information.

[0129] Sub-step 2803, the characteristic crop recognition model performs classification mapping based on the fused feature information to obtain a characteristic crop classification recognition result, wherein the characteristic crop classification recognition result includes a characteristic crop classification map of the target research area.

[0130] Sub-steps 2801 and 2802 are described uniformly:

[0131] In actual implementation, the characteristic crop recognition model can be provided with a three-branch hybrid convolutional neural network based on a convolutional module. In this embodiment, the identification sensitive band is input into the pre-trained characteristic crop recognition model, and multiple features are extracted from the identification sensitive band through the three-branch hybrid convolutional neural network, including but not limited to: spectral features, spatial features, and spatial-spectral joint features. Subsequently, the extracted multiple features can be fused to obtain a feature fusion map that fuses multiple features as fusion feature information, and then identification and classification are performed based on the fusion feature information to obtain a classification map of characteristic crops in the target research area. For example, taking litchi, a characteristic crop of City X, as an example, its corresponding classification map of characteristic crops can be referred to Figure 7 shown.

[0132] In the related art, traditional classification methods or machine learning algorithms mainly rely on manual feature extraction and feature selection. Due to the large spatial heterogeneity of spectral features of hyperspectral images, it is impossible to fully explore the spectral information in hyperspectral images and the intrinsic correlation between space and spectrum. This embodiment uses a three-branch hybrid convolutional neural network based on a convolution module to construct a deep learning model, so that the deep learning model can automatically extract complex features from hyperspectral images, learn more abstract and discriminative features, and use it as a characteristic crop recognition model to identify characteristic crops and land objects after training. Compared with machine learning, it has better generalization ability and computational efficiency, and is an effective method for hyperspectral classification.

[0133] In an optional embodiment, before the embodiment of the present application uses a preset characteristic crop recognition model to identify and classify the identification sensitive bands, the embodiment may further specifically include: obtaining sample data of a sample study area, wherein the sample data includes airborne hyperspectral image samples, satellite-borne hyperspectral image samples, and ground object samples; for the airborne hyperspectral image samples and the satellite-borne hyperspectral image samples, respectively extracting ground object spectral response curve samples corresponding to the ground object samples; determining the identification sensitive bands of the ground object samples in the satellite-borne hyperspectral image samples based on the ground object spectral response curve samples through a preset spectral curve adaptive matching algorithm. Band samples; constructing a three-branch hybrid convolutional neural network based on a convolution module, and constructing a deep learning model based on the three-branch hybrid convolutional neural network; using the three-branch hybrid convolutional neural network, performing model feature extraction training based on the sample data and the identification sensitive band samples to obtain feature samples, wherein the feature samples include spectral feature samples, spatial feature samples, and space-spectrum joint feature samples; constructing a training set and a test set based on the feature samples, the sample data, and the identification sensitive band samples; using the training set to train the deep learning model to obtain the characteristic crop recognition model.

[0134] In a specific implementation, this embodiment can select a study area and obtain sample data corresponding to the study area. Specifically, for the study area, airborne hyperspectral image samples are obtained by drones, satellite-borne hyperspectral image samples are obtained by satellites, and ground object samples are obtained by data acquisition. For the area of ​​interest represented by the ground object samples, the corresponding ground object spectral response curve samples are extracted from the airborne hyperspectral image samples and the satellite-borne hyperspectral image samples, respectively, and then based on the ground object spectral response curve samples, the identification sensitive band samples are obtained. Subsequently, the identification sensitive band samples are input into a pre-constructed three-branch hybrid convolutional neural network for feature extraction to obtain feature samples, and a training set and a test set are constructed by combining multiple samples. The training set is used to train the deep learning model based on the three-branch hybrid convolutional neural network, thereby obtaining a trained characteristic crop recognition model, and the classification recognition results generated by the characteristic crop recognition model for the input sample data can also be obtained as the output result of the model training stage for subsequent model accuracy verification. A three-branch hybrid convolutional neural network based on a convolutional module is adopted to automatically extract spectral and spatial information. By superimposing convolutional layers to capture deep and shallow features, more global context information is collected to effectively alleviate the vanishing gradient.

[0135] Optionally, the embodiment of the present application uses the three-branch hybrid convolutional neural network to perform model feature extraction training based on the sample data and the identified sensitive band samples to obtain feature samples, which may specifically include: constructing a spectral feature extraction branch, a spatial feature extraction branch, and a spatial-spectral joint feature extraction branch in the three-branch hybrid convolutional neural network; extracting local information in the spectral dimension from the sample data and the identified sensitive band samples through the spectral feature extraction branch to obtain spectral feature samples; and performing mapping on the sample data and the identified sensitive band samples through the spatial feature extraction branch. The image space feature is extracted to obtain spatial feature samples; the sample data and the identification sensitive band samples are extracted end-to-end through the spatial-spectral joint feature extraction branch to obtain spatial-spectral joint feature samples; wherein the spectral feature extraction branch includes a one-dimensional convolution layer, a pooling layer and a fully connected layer, the spatial feature extraction branch includes a two-dimensional convolution layer, a pooling layer and a fully connected layer, the spatial-spectral joint feature extraction branch includes a three-dimensional convolution layer, a two-dimensional convolution layer, a pooling layer and a fully connected layer, and the three-dimensional convolution layer is used to slide between the spatial and spectral dimensions simultaneously to meet the requirements of multi-scale and multi-resolution.

[0136] In actual processing, refer to Fig. 9 The three-branch hybrid convolutional neural network in this embodiment is a neural network built based on a convolutional module. The three-branch hybrid convolutional neural network is mainly composed of a three-branch structure, which includes a spectral feature extraction branch, a spatial feature extraction branch, and a space-spectrum joint feature extraction branch. Specifically, refer to Figure 8 As shown in Figure 1, the spectral feature extraction branch is mainly composed of one-dimensional (1D) convolutional layers, pooling layers, and fully connected layers, which are mainly used to extract spectral features of images and effectively capture local information in the spectral dimension. Fig.10 The spatial feature extraction branch mainly consists of two-dimensional (2D) convolutional layers, pooling layers, and fully connected layers, which mainly realize the spatial feature extraction of images. Fig.11 The spatial-spectral joint feature extraction branch is mainly composed of a three-dimensional (3D) convolution layer, a 2D convolution layer, a pooling layer, and a fully connected layer, which can realize end-to-end extraction of spatial-spectral joint features. The 3D convolution kernel can slide between the spatial and spectral dimensions simultaneously to meet the requirements of multi-scale and multi-resolution. In this embodiment, the spatial-spectral joint feature extraction branch can extract more complex spatial-spectral joint information in an efficient way.

[0137] In actual processing, this embodiment uses a three-branch hybrid convolutional neural network to extract corresponding features from the input data using each feature extraction branch, and then can fuse the multiple features extracted by each feature extraction branch through a fully connected layer, and map the fused features to output categories and output category probabilities as the output results of the model training phase for subsequent model accuracy verification. Preferably, in this embodiment, when using a three-branch hybrid convolutional neural network to identify litchi in Gaozhou City, the computing environment of the model is i9-12900KCPU and NVIDIAGeForceRTX3060GPU; the hyperparameters are set as follows: the proportion of training sets to total samples is 0.1, the number of model training times is 500 times, the batch size is set to 256, the optimizer is Adam, the initial learning rate is set to 5×10-4, and the patch size of reading hyperspectral pixels is 9*9.

[0138] In an optional embodiment, after the embodiment of the present application obtains the characteristic crop recognition model, it may further specifically include: obtaining the sample land object recognition result output by the characteristic crop recognition model for the sample data, and the sub-meter level image corresponding to the training set; performing an accuracy test using a test set corresponding to the sample data through a preset accuracy evaluation index, and visually interpreting and judging the texture information contained in the sub-meter level image to verify the sample land object recognition result.

[0139] In this embodiment, the constructed three-branch hybrid convolutional neural network deep learning model uses satellite-borne hyperspectral image data as the data source, and uses the three-branch hybrid convolutional neural network model to identify characteristic crops based on the sample data of the study area (characteristic crops, construction land, cultivated land, forest land, water bodies, and gardens), and obtains a classification map of characteristic crops in the study area, that is, the identification result of characteristic crops in the study area. For this identification result, combined with the model accuracy evaluation index, the accuracy of the characteristic crop identification model is evaluated. Preferably, the model accuracy evaluation index includes: direct verification and remote sensing image visual verification, which is not limited in this embodiment.

[0140] For direct verification of the precision evaluation index, this embodiment uses evaluation indicators such as Overall Accuracy (OA), Average Accuracy (AA), and Kappa coefficient to evaluate the quality of the classification results, and combines accuracy, recall rate, and F1 value (i.e., the harmonic mean of accuracy and recall rate) to directly evaluate the model recognition accuracy. The evaluation accuracy of the characteristic crop recognition model in this embodiment on the test set is as follows: OA is 93.74, AA is 91.13, and Kappa is 91.92. The accuracy, recall rate, and F1 value of each category are shown in Table 1. Among them, accuracy refers to the proportion of samples predicted correctly by the model to the total number of samples; recall rate is the proportion of samples predicted by the model as positive examples that are truly positive examples; F1 value is the harmonic mean of accuracy and recall rate, which is used to comprehensively evaluate these two indicators.

[0141] Accuracy (%) Recall rate (%) F1 value Number of test set samples litchi 88.79 88.61 88.70 72266 Construction land 98.24 98.70 98.47 134827 woodland 93.20 95.46 94.32 127082 arable land 88.09 86.06 87.06 67016 Garden 91.57 78.52 84.54 13720 Water 99.67 99.48 99.58 36139

[0142] Table 1. Classification accuracy of litchi in City X

[0143] The following is a detailed description of the evaluation indicators such as OA, AA and Kappa coefficient:

[0144] OA represents the proportion of correctly classified samples in the test samples, and the calculation method is shown in formula (1). represents the number of correctly classified samples in the test samples, which is the total number of test samples.

[0145]

[0146] AA represents the average classification accuracy of each category, and the calculation method is shown in formula (2):

[0147]

[0148] Among them, correct c , total c They respectively represent the number of correctly classified and total number of test samples of the cth type of ground objects, and C is the number of ground object categories.

[0149] The Kappa coefficient is an indicator of the consistency between the test sample and the prediction result. Its value range is [0, 1]. The closer it is to 1, the higher the consistency of the model. The calculation method is shown in formula (3), where M c Refers to the number of test samples classified as class c objects.

[0150]

[0151] For visual verification of remote sensing images: With the help of Jilin-1, Gaofen-1 and other sub-meter-level images, visual interpretation and judgment are performed through texture information to verify the classification results of characteristic crops in the study area. In the visual interpretation and judgment, 10% of the classification results can be randomly sampled for verification. Preferably, the sampling spots can be evenly distributed throughout the study area.

[0152] In summary, the embodiment of the present application determines the target study area to obtain the hyperspectral image data set and the field sample data set of the target study area, determines the region of interest based on the field sample data set, and then extracts the ground object spectrum of the airborne hyperspectral image data and the satellite-borne hyperspectral image data respectively for the region of interest in combination with the field sample data set to obtain a ground object spectrum curve set, takes the airborne ground object spectrum curve in the ground object spectrum curve set as the true value, and resamples based on the airborne hyperspectral image data according to the preset resolution set to obtain the spectrum curve of the characteristic crops corresponding to each resolution in the resolution set, and within the resolution set, calculates the spectrum angle based on the airborne ground object spectrum curve and the satellite-borne ground object spectrum curve to obtain the spectrum angle matching, and then performs similarity analysis based on the spectrum angle matching combined with the spectrum curve to obtain the identification sensitive band, and classifies the identification sensitive band through the preset characteristic crop identification model to obtain the classification and identification result of the characteristic crops. Therefore, the embodiment of the present application solves the problem of low classification accuracy due to low resolution and mixed pixels in satellite-borne hyperspectral remote sensing. This application realizes the identification of characteristic crops based on a three-branch hybrid convolutional neural network by establishing an adaptive matching conversion algorithm for airborne and satellite-borne ground object spectral curves. While obtaining fine spatial and spectral information and improving the accuracy of characteristic crop identification, it has the ability of large-scale monitoring and realizes regional-scale characteristic crop identification based on hyperspectral satellite images, providing data support for the development of regional characteristic crops and other economic crops, and effectively solving the problems of the existing technology such as the inability to effectively identify characteristic crops, low recognition and classification accuracy, and insufficient recognition range.

[0153] It should be noted that, for the purpose of simple description, the method embodiments are expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited to the described order of actions, because according to the embodiments of the present application, certain steps may be performed in other orders or simultaneously.

[0154] like Fig.12 As shown, the embodiment of the present application further provides a characteristic crop identification system 1200, including:

[0155] A target research area determination module 1210 is used to determine a target research area;

[0156] The data set acquisition module 1220 is used to acquire a hyperspectral image data set and a field sample data set of the target study area, wherein the hyperspectral image data set includes airborne hyperspectral image data and satellite-borne hyperspectral image data, and the field sample data set includes ground object samples;

[0157] A spectral curve extraction module 1230 is used to perform spectral curve extraction processing based on the hyperspectral image data set and the field sample data set to obtain a ground object spectral curve set;

[0158] The adaptive matching module 1240 is used to perform adaptive matching according to the ground object spectral curve set to obtain the identification sensitive band of the characteristic crops, wherein the identification sensitive band is a band that the satellite-borne hyperspectral image data has high sensitivity to the identification of the characteristic crops;

[0159] The identification and classification module 1250 is used to identify and classify the identification sensitive bands through a preset characteristic crop identification model to obtain a characteristic crop classification identification result;

[0160] The airborne hyperspectral image data is spectral image data acquired by a drone, and the satellite-borne hyperspectral image data is spectral image data acquired by a satellite.

[0161] Optionally, the data set acquisition module 1220 includes:

[0162] The UAV hyperspectral image acquisition submodule is used to collect images of the target research area through a preset UAV to obtain a UAV hyperspectral image containing the characteristic crops, wherein the UAV carries a hyperspectral sensor;

[0163] A correction and splicing submodule is used to perform correction and splicing processing based on the UAV hyperspectral image to obtain the airborne hyperspectral image data;

[0164] A satellite-borne hyperspectral image acquisition submodule is used to acquire images of the target research area through a preset satellite to obtain a satellite-borne hyperspectral image;

[0165] The image preprocessing submodule is used to perform image preprocessing based on the satellite-borne hyperspectral image to obtain high-quality satellite-borne hyperspectral image data and to extract a field sample data set from a preset database.

[0166] Optionally, the spectrum curve extraction module 1230 includes:

[0167] An area of ​​interest determination submodule, used to determine an area of ​​interest based on the field sample data set, wherein the area of ​​interest includes at least one type of ground feature;

[0168] A ground object spectrum extraction submodule is used to extract ground object spectra from the airborne hyperspectral image data and the satellite-borne hyperspectral image data respectively for the region of interest in combination with the field sample data set, obtain an average spectrum curve set, and use the average spectrum curve set as the ground object spectrum curve set;

[0169] The ground object spectrum curve set includes the airborne ground object spectrum curve corresponding to the airborne hyperspectral image data and the satellite-borne ground object spectrum curve corresponding to the satellite-borne hyperspectral image data.

[0170] Optionally, the adaptive matching module 1240 includes:

[0171] A resampling submodule, for taking the airborne ground feature spectral curve as a true value and resampling based on the airborne hyperspectral image data according to a preset resolution set to obtain a spectral curve of the characteristic crop corresponding to each resolution in the resolution set;

[0172] A spectral angle calculation submodule, used to perform spectral angle calculation within the resolution set based on the airborne ground object spectral curve and the satellite-borne ground object spectral curve to obtain spectral angle matching;

[0173] The similarity analysis submodule is used to perform similarity analysis based on the spectral angle matching in combination with the spectral curve to obtain the identification sensitive band.

[0174] Optionally, the spectral angle calculation submodule is specifically used to: use the spectral response curves of each object in the airborne ground object spectral curve and the satellite-borne ground object spectral curve as a space vector; according to the formula and Calculate the generalized angle θ between the space vectors; wherein the generalized angle θ is proportional to the similarity of the spectral curves, and They represent the vectors of two ground object spectral curves respectively, and Similarity represents the similarity of the two ground object spectral curves.

[0175] Optionally, the identification and classification module 1250 includes:

[0176] A feature extraction submodule is used to extract features of the identification sensitive band through the characteristic crop identification model to obtain spectral features, spatial features and space-spectrum joint features;

[0177] A feature fusion submodule, used for performing feature fusion based on the spectral feature, the spatial feature and the spatial-spectral joint feature to obtain fused feature information;

[0178] The classification mapping submodule is used to perform classification mapping based on the fused feature information to obtain a classification and identification result of characteristic crops, wherein the classification and identification result of characteristic crops includes a classification map of characteristic crops in the target research area.

[0179] Optionally, the characteristic crop identification system 1200 further includes:

[0180] A sample acquisition module is used to acquire sample data of a sample study area, wherein the sample data includes airborne hyperspectral image samples, satellite-borne hyperspectral image samples and ground feature samples;

[0181] A curve sample extraction module is used to extract ground object spectral response curve samples corresponding to the ground object samples respectively for the airborne hyperspectral image samples and the satellite-borne hyperspectral image samples;

[0182] A sensitive band sample identification module is used to determine the identification sensitive band samples of the ground object samples in the satellite-borne hyperspectral image samples based on the ground object spectral response curve samples by using a preset spectral curve adaptive matching algorithm;

[0183] A construction module is used to construct a three-branch hybrid convolutional neural network based on a convolutional module, and to construct a deep learning model based on the three-branch hybrid convolutional neural network;

[0184] A feature extraction training module is used to perform model feature extraction training based on the sample data and the identified sensitive band samples through the three-branch hybrid convolutional neural network to obtain feature samples, wherein the feature samples include spectral feature samples, spatial feature samples, and space-spectrum joint feature samples;

[0185] A set construction module, used to construct a training set and a test set based on the feature samples, the sample data and the identified sensitive band samples;

[0186] The model training module is used to use the training set to perform model training on the deep learning model to obtain the characteristic crop recognition model.

[0187] The model accuracy verification module is used to obtain the sample land object recognition results output by the characteristic crop recognition model for the sample data, and the sub-meter level image corresponding to the training set; through the preset accuracy evaluation index, the accuracy test is performed using the test set corresponding to the sample data, and the texture information contained in the sub-meter level image is visually interpreted and judged to verify the sample land object recognition results.

[0188] Optionally, the feature extraction training module includes:

[0189] A feature extraction branch construction submodule, used to construct a spectral feature extraction branch, a spatial feature extraction branch and a space-spectrum joint feature extraction branch in the three-branch hybrid convolutional neural network;

[0190] The sample feature extraction submodule is used to extract local information in the spectral dimension from the sample data and the identified sensitive band samples through the spectral feature extraction branch to obtain spectral feature samples; to perform image space feature extraction on the sample data and the identified sensitive band samples through the spatial feature extraction branch to obtain spatial feature samples; and to perform end-to-end extraction on the sample data and the identified sensitive band samples through the spatial-spectral joint feature extraction branch to obtain spatial-spectral joint feature samples;

[0191] Among them, the spectral feature extraction branch includes a one-dimensional convolution layer, a pooling layer and a fully connected layer, the spatial feature extraction branch includes a two-dimensional convolution layer, a pooling layer and a fully connected layer, and the spatial-spectral joint feature extraction branch includes a three-dimensional convolution layer, a two-dimensional convolution layer, a pooling layer and a fully connected layer. The three-dimensional convolution layer is used to slide simultaneously between the spatial and spectral dimensions to meet the requirements of multi-scale and multi-resolution.

[0192] It should be noted that the characteristic crop identification system provided in the embodiment of the present application can execute the characteristic crop identification method of coordinated satellite-borne and airborne hyperspectral remote sensing provided in any embodiment of the present application, and has the corresponding functions and beneficial effects of executing the characteristic crop identification method.

[0193] In a specific implementation, the above-mentioned characteristic crop identification system can be integrated into a device, so that the device can use the airborne hyperspectral image data and satellite-borne hyperspectral image data of the target study area to identify and classify the characteristic crops and other ground objects in the target study area. As an electronic device, it can effectively identify the ground objects including characteristic crops and improve the accuracy of ground object identification. The electronic device can be composed of two or more physical entities, or it can be composed of one physical entity. For example, the electronic device can be a personal computer (PC), a computer, a server, etc., and the embodiments of the present application do not impose specific restrictions on this.

[0194] like Fig.13 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114; the memory 113 is used to store computer programs; the processor 111 is used to implement the steps of the method for identifying characteristic crops by coordinating satellite-borne hyperspectral remote sensing and airborne hyperspectral remote sensing provided by any of the aforementioned method embodiments when executing the program stored in the memory 113. Exemplarily, the steps of the method for identifying characteristic crops by coordinating satellite-borne hyperspectral remote sensing and airborne hyperspectral remote sensing may include the following steps: determining a target study area; obtaining a hyperspectral image dataset and a field sample dataset of the target study area, wherein the hyperspectral image dataset includes airborne hyperspectral image data and satellite-borne hyperspectral image data, and the field sample dataset includes ground object samples; performing spectral curve extraction processing based on the hyperspectral image dataset and the field sample dataset to obtain a ground object spectral curve set; performing adaptive matching based on the ground object spectral curve set to obtain identification sensitive bands of characteristic crops, wherein the identification sensitive bands are bands in which the satellite-borne hyperspectral image data has high sensitivity to the identification of the characteristic crops; identifying and classifying the identification sensitive bands through a preset characteristic crop identification model to obtain a classification and identification result of characteristic crops; wherein the airborne hyperspectral image data are spectral image data acquired by a drone, and the satellite-borne hyperspectral image data are spectral image data acquired by a satellite.

[0195] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of a method for identifying characteristic crops by coordinating satellite-borne hyperspectral remote sensing and airborne hyperspectral remote sensing as provided in any of the aforementioned method embodiments are implemented.

[0196] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0197] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for identifying characteristic crops by coordinating satellite-borne hyperspectral remote sensing and airborne hyperspectral remote sensing, characterized in that: include: Identify the target study area; Acquire a hyperspectral image dataset and a field sample dataset of the target study area, wherein the hyperspectral image dataset includes airborne hyperspectral image data and satellite-borne hyperspectral image data, and the field sample dataset includes ground object samples; Performing spectral curve extraction processing based on the hyperspectral image data set and the field sample data set to obtain a ground object spectral curve set; Adaptively matching the ground feature spectral curve set to obtain a sensitive band for identifying characteristic crops, wherein the sensitive band is a band that the satellite-borne hyperspectral image data is highly sensitive to for identifying the characteristic crops; The identification sensitive bands are identified and classified by using a preset characteristic crop identification model to obtain a characteristic crop classification identification result; The airborne hyperspectral image data is spectral image data acquired by a drone, and the satellite-borne hyperspectral image data is spectral image data acquired by a satellite.

2. The method according to claim 1, characterized in that The step of obtaining a hyperspectral image dataset and a field sample dataset of the target research area includes: The target research area is imaged by a preset drone to obtain a drone hyperspectral image containing the characteristic crops, wherein the drone carries a hyperspectral sensor; Perform correction and splicing processing based on the UAV hyperspectral image to obtain the airborne hyperspectral image data; The target research area is imaged by a preset satellite to obtain a satellite-borne hyperspectral image; Image preprocessing is performed based on the satellite-borne hyperspectral image to obtain high-quality satellite-borne hyperspectral image data, and a field sample data set is extracted from a preset database.

3. The method according to claim 1, characterized in that The extracting and processing of spectral curves based on the hyperspectral image dataset and the field sample dataset to obtain a ground object spectral curve set includes: Determine an area of ​​interest based on the field sample data set, wherein the area of ​​interest includes at least one type of ground feature; For the region of interest, combined with the field sample data set, the ground object spectrum is extracted from the airborne hyperspectral image data and the satellite-borne hyperspectral image data respectively to obtain an average spectrum curve set, and the average spectrum curve set is used as the ground object spectrum curve set; The ground object spectrum curve set includes an airborne ground object spectrum curve corresponding to the airborne hyperspectral image data and a satellite-borne ground object spectrum curve corresponding to the satellite-borne hyperspectral image data.

4. The method according to claim 3, characterized in that The adaptive matching is performed according to the ground object spectrum curve set to obtain the identification sensitive band of the characteristic crops, including: The airborne ground feature spectral curve is taken as a true value, and resampled based on the airborne hyperspectral image data according to a preset resolution set to obtain a spectral curve of the characteristic crop corresponding to each resolution in the resolution set; In the resolution set, a spectral angle calculation is performed based on the airborne ground object spectral curve and the satellite-borne ground object spectral curve to obtain a spectral angle match; The identification sensitive band is obtained by performing similarity analysis based on the spectral angle matching in combination with the spectral curve.

5. The method according to claim 4, characterized in that The performing spectral angle calculation based on the airborne ground object spectral curve and the satellite-borne ground object spectral curve to obtain spectral angle matching includes: Taking the spectral response curves of each object in the airborne ground object spectral curve and the satellite-borne ground object spectral curve as a space vector; According to the formula and Calculating the generalized angle θ between the space vectors; The generalized angle θ is proportional to the similarity of the spectral curve. and They represent the vectors of two ground object spectral curves respectively, and Similarity represents the similarity of the two ground object spectral curves.

6. The method according to any one of claims 1 to 5, characterized in that: The identification sensitive band is identified and classified by using a preset characteristic crop identification model to obtain a characteristic crop classification identification result, including: Extracting features of the identification sensitive bands through the characteristic crop identification model to obtain spectral features, spatial features and spatial-spectral joint features; Perform feature fusion based on the spectral feature, the spatial feature and the spatial-spectral joint feature to obtain fused feature information; The characteristic crop identification model performs classification mapping based on the fused feature information to obtain a characteristic crop classification identification result, and the characteristic crop classification identification result includes a characteristic crop classification map of the target research area.

7. The method according to claim 1, characterized in that Before the identification and classification of the identification sensitive bands by using the preset characteristic crop identification model, the method further includes: Acquire sample data of a sample study area, wherein the sample data includes airborne hyperspectral image samples, satellite-borne hyperspectral image samples, and ground feature samples; For the airborne hyperspectral image samples and the satellite-borne hyperspectral image samples, respectively extracting ground object spectral response curve samples corresponding to the ground object samples; Determine, by a preset spectral curve adaptive matching algorithm, based on the spectral response curve samples of the ground objects, the identification sensitive band samples of the ground object samples in the satellite-borne hyperspectral image samples; Constructing a three-branch hybrid convolutional neural network based on a convolutional module, and constructing a deep learning model based on the three-branch hybrid convolutional neural network; Through the three-branch hybrid convolutional neural network, model feature extraction training is performed based on the sample data and the identified sensitive band samples to obtain feature samples, wherein the feature samples include spectral feature samples, spatial feature samples, and space-spectrum joint feature samples; Constructing a training set and a test set based on the feature samples, the sample data and the identified sensitive band samples; The deep learning model is trained using the training set to obtain the characteristic crop recognition model.

8. The method according to claim 7, characterized in that The three-branch hybrid convolutional neural network is used to perform model feature extraction training based on the sample data and the identified sensitive band samples to obtain feature samples, including: Constructing a spectral feature extraction branch, a spatial feature extraction branch and a space-spectrum joint feature extraction branch in the three-branch hybrid convolutional neural network; Extracting local information in the spectral dimension from the sample data and the identified sensitive band samples through the spectral feature extraction branch to obtain a spectral feature sample; Through the spatial feature extraction branch, image spatial feature extraction is performed on the sample data and the identified sensitive band samples to obtain spatial feature samples; Through the space-spectrum joint feature extraction branch, the sample data and the identification sensitive band samples are extracted end-to-end to obtain space-spectrum joint feature samples; Among them, the spectral feature extraction branch includes a one-dimensional convolution layer, a pooling layer and a fully connected layer, the spatial feature extraction branch includes a two-dimensional convolution layer, a pooling layer and a fully connected layer, and the spatial-spectral joint feature extraction branch includes a three-dimensional convolution layer, a two-dimensional convolution layer, a pooling layer and a fully connected layer. The three-dimensional convolution layer is used to slide simultaneously between the spatial and spectral dimensions to meet the requirements of multi-scale and multi-resolution.

9. The method according to claim 7, characterized in that: After obtaining the characteristic crop identification model, the method further includes: Obtaining the sample ground feature recognition result output by the characteristic crop recognition model for the sample data, and the sub-meter level image corresponding to the training set; The sample ground feature recognition results are verified by using preset accuracy evaluation indicators, performing accuracy testing using the test set corresponding to the sample data, and visually interpreting and judging the texture information contained in the sub-meter image.

10. A characteristic crop identification system, characterized in that: include: A target study area determination module is used to determine the target study area; A data set acquisition module is used to acquire a hyperspectral image data set and a field sample data set of the target study area, wherein the hyperspectral image data set includes airborne hyperspectral image data and satellite-borne hyperspectral image data, and the field sample data set includes ground object samples; A spectral curve extraction module is used to perform spectral curve extraction processing based on the hyperspectral image data set and the field sample data set to obtain a ground object spectral curve set; An adaptive matching module, used for adaptively matching according to the ground object spectral curve set to obtain a sensitive band for identifying characteristic crops, wherein the sensitive band is a band that the satellite-borne hyperspectral image data has a high sensitivity to identifying the characteristic crops; An identification and classification module is used to identify and classify the identification sensitive bands through a preset characteristic crop identification model to obtain a characteristic crop classification identification result; The airborne hyperspectral image data is spectral image data acquired by a drone, and the satellite-borne hyperspectral image data is spectral image data acquired by a satellite.

Images