Early automatic identification method of winter crops based on multimodal knowledge graph

Abstract

The present invention proposes an early automatic identification method for winter crops based on a multimodal knowledge graph. Based on optical and radar time series data sets, a multimodal vegetation index and a time spectrum feature space of multipolarization radar coefficient enhancement transformation are designed respectively, and a multimodal knowledge graph for early identification of winter crop types is established to achieve the goal of early identification of winter crop types. In view of the differences in crop plant type and canopy structure of winter crops such as winter wheat and winter rapeseed, the time series curves of their multipolarization radar coefficients show a characteristic of one increasing while the other decreasing. After selecting a suitable power base to implement power index transformation, winter rapeseed and winter wheat respectively show the characteristics of stable low values ​​and sharp increase in high values. A high-precision winter crop type distribution map can be obtained in advance during the budding period of winter rapeseed and the greening period of winter wheat.

Description

Technical Field

[0001] The present invention belongs to the field of high-efficiency intelligent automatic monitoring of agricultural conditions, and relates to an early mapping method of crops, specifically an early automatic identification method of winter crops based on a multimodal knowledge graph. Background Art

[0002] By sowing winter crops and implementing multi-season planting, it is helpful to effectively increase the planting intensity of cultivated land and thus significantly increase crop yields. Obtaining winter crop type information with high accuracy and timeliness is very important for agricultural precision management needs such as food yield estimation and agricultural disaster insurance claims, and achieving the zero hunger goal of the United Nations Sustainable Development Goals.

[0003] As important grain and oil crops, wheat and rapeseed have always attracted much attention for crop mapping research using remote sensing technology. Winter wheat, due to its characteristic of overwintering and turning green, usually has two growth peaks throughout the growing season. Its vegetation index temporal spectrum characteristics are significantly different from those of other crops. Therefore, a series of winter wheat identification methods have been established based on this feature. Winter rapeseed also has characteristics that are different from other crops, such as a long flowering period and bright colors. During the flowering period, rapeseed presents temporal and spatial spectral characteristics that are different from other crops. These characteristics of winter wheat and winter rapeseed, which are different from other crops, can be well applied to post-season remote sensing mapping of winter wheat and winter rapeseed planting areas.

[0004] Compared with post-season crop mapping, early crop mapping faces the following two challenges: 1. In the early growth stages of different crops, the time series image features are very similar, and it is difficult to find iconic spatiotemporal spectral features that highlight different types of crops; 2. Due to the short time window in the early growth stages of crops, available remote sensing time series images are very scarce. For example, for rapeseed, few people pay attention to the phenological characteristics before the flowering period. The problem of scarcity of time series remote sensing in the early growth stages of crops is more serious in cloudy and rainy areas.

[0005] Radar time-series remote sensing images, with their ability to penetrate clouds and fog, will help improve the availability of time-series remote sensing images for early crop mapping. In particular, with the open sharing of Sentinel-1SAR, its VV and VH polarization radar backscatter coefficient time-series data will bring opportunities for crop identification. For example, a method for automatic rice identification based on synthetic aperture radar time-series data (application number: CN202010117995.5) uses the characteristic that the radar backscatter coefficient first decreases and then gradually increases due to the need for transplanting and flooding rice. This feature can be used to achieve automatic rice identification. Recent studies have shown that the fusion of optical and time-series images can effectively improve the accuracy of crop identification. For example, a tobacco-rice planting pattern recognition method based on optical and radar time-series data (application number: CN202111637590.5) establishes a tobacco-rice planting pattern recognition process by integrating the optical vegetation index and radar backscatter coefficient before and after the double-season crops. Summary of the invention

[0006] Considering that the current crop type recognition technology that integrates optical and radar time-series images is still in the exploratory research stage, it is basically based on post-season crop recognition models and has not been applied to the field of early automatic mapping of crops, especially early identification of crops such as winter wheat and winter rapeseed.

[0007] In view of the challenges faced by early crop mapping, current early crop mapping is mostly based on supervised classification methods such as random forests and deep neural networks. These methods can achieve very high accuracy when there is sufficient training sample data. However, the disadvantage is that when it is migrated and applied to other research areas or research years where training data is lacking, the accuracy of the model is difficult to guarantee. Since early crop mapping requires high timeliness, collecting training sample data is usually time-consuming and laborious, and it is usually difficult to meet the requirements of supervised classification training models in the short term. Therefore, it is urgent to establish a knowledge graph for early crop identification, reduce dependence on training sample data, and improve recognition accuracy and spatiotemporal generalization capabilities. The present invention intends to fuse optical and radar multimodal time series image data to establish a method for early identification of winter crop types, which can at least obtain the spatial distribution map of winter wheat and winter rapeseed in the study area before the heading period of winter wheat and the flowering period of winter rapeseed.

[0008] The scheme of the present invention is based on optical and radar time series data sets, and designs the time-spectral feature space of multimodal vegetation index and multi-polarization radar coefficient enhanced transformation respectively, and establishes a multimodal knowledge graph for early identification of winter crop types to achieve the goal of early identification of winter crop types.

[0009] In view of the differences in crop type and canopy structure of winter crops such as winter wheat and winter rapeseed, the time series curves of their multi-polarization radar coefficients show a characteristic of one increasing while the other decreases. After selecting a suitable power base to implement power index transformation, winter rapeseed and winter wheat show the characteristics of stable low values ​​and sharp increase in high values, respectively. A high-precision winter crop type distribution map can be obtained in advance in the budding period of winter rapeseed and the greening period of winter wheat.

[0010] The present invention solves the technical bottleneck that crop identification algorithms are heavily dependent on sample data and difficult to achieve generalization migration by establishing a knowledge graph of crop growth characteristics and remote sensing time spectrum characteristics. The present invention is simple and easy to implement, has strong interpretation ability, high degree of automation, is suitable for cloudy and rainy areas, and has the ability to be applied and promoted on a large scale across domains and years.

[0011] Terminology explanation:

[0012] The Sentinel-1 satellite carries a C-band synthetic aperture radar that can provide continuous radar data products in various weather conditions including rain clouds, such as VV and VH polarimetric radar data. VV and VH represent the backscatter coefficients of the VV and VH polarizations of the radar images, respectively.

[0013] Sentinel-2MSI is a new type of optical remote sensing satellite with multiple advantages such as wide bandwidth, high temporal and spatial resolution, and free sharing. Its unique red light edge band is very effective for monitoring vegetation growth and health status.

[0014] Optical vegetation index: Vegetation index is a factor that characterizes the growth status and spatial distribution density of vegetation. The vegetation index commonly referred to is the optical vegetation index. Common optical vegetation indices include NDVI and EVI2. NDVI is the normalized vegetation index, and its full name is NormalizedDifference Vegetation Index. EVI2 is the enhanced vegetation index, and its full name is EnhancedVegetation Index. The calculation formula of EVI2 index is: where ρ Red ,ρ NIR They are the reflectivity of the red light and near-infrared bands of the Sentinel images, respectively.

[0015] Radar vegetation index: In order to make up for the lack of data of optical vegetation index which is often affected by cloudy and rainy weather, radar vegetation index (Radarbased Vegetation Indices, referred to as RVI) is further established. The calculation formula of radar vegetation index RVI used in the present invention is: RVI = VH-VV.

[0016] Additive multi-polarization radar coefficient: Additive multi-polarization radar coefficient (Sum of VH and VV, referred to as SVV), its calculation formula is: SVV = VH + VV.

[0017] The technical solution adopted by the present invention to solve the technical problem is:

[0018] A method for automatic early identification of winter crops based on a multimodal knowledge graph, characterized by comprising the following steps:

[0019] Step S01: Establish optical and radar multimodal vegetation index time series data sets respectively;

[0020] Step S02: establishing an additive multi-polarization radar coefficient time series data set;

[0021] Step S03: construct optical and radar multimodal vegetation index difference sequence data sets respectively;

[0022] Step S04: constructing the temporal spectrum characteristics of the multimodal vegetation index in the early stage of winter crop growth;

[0023] Step S05: performing power-index transformation on the additive multi-polarization radar coefficient time series data;

[0024] Step S06: constructing the time-spectrum characteristics of the multi-polarization radar coefficient enhancement transformation;

[0025] Step S07: Evaluate the early identification window of winter crop types;

[0026] Step S08: Establish a multimodal knowledge graph for early identification of winter crop types.

[0027] Furthermore, in step S01, in the study area, the optical vegetation index EVI2 is calculated pixel by pixel and period by period; according to the cloud coverage, the observation records with clouds are eliminated, and the effective observation time series data set of the EVI2 vegetation index on cloudless days is based on the pixel by pixel, and then the Whittaker Smoother data smoothing method is used to establish a smoothed optical vegetation index time series data set for the study area; the radar vegetation index RVI is calculated pixel by pixel and period by period, and the calculation formula is: RVI=VH-VV, where VV and VH represent the backscattering coefficients of the polarization of the radar images VV and VH, respectively; the Whittaker Smoother data smoothing method is used to establish a smoothed radar vegetation index time series data set for the study area, thereby forming an optical and radar multimodal vegetation index time series data set for the study area.

[0028] Furthermore, in step S02, based on the VH and VV multi-polarization radar coefficients of Sentinel-1, the additive multi-polarization radar coefficients are calculated pixel by pixel and period by period, so as to obtain the additive multi-polarization radar coefficient time series data set of the study area in that year; the calculation formula of the additive multi-polarization radar coefficient SVV is: SVV=VH+VV, wherein VV and VH represent the backscattering coefficients of the VV and VH polarizations of the radar images, respectively; the Whittaker Smoother data smoothing method is used to construct a smoothed additive multi-polarization radar coefficient time series data set.

[0029] Furthermore, in step S03, based on the optical and radar vegetation index time series data sets, time series difference is performed pixel by pixel and period by period to obtain optical and radar vegetation index difference sequence data sets, respectively.

[0030] Furthermore, in step S04, based on the optical and radar vegetation index time series data sets, multimodal vegetation index time spectrum features are extracted, including: an accumulation index of winter crop growth early vegetation index ADVI based on the optical vegetation index and an accumulation index of winter crop growth early vegetation index ADRVI based on the radar vegetation index, and the calculation formulas are:

[0031]

[0032]

[0033] in, Represents the pixel t i EVI2 and RVI values ​​corresponding to the moment; Represents the pixel t i+1 The EVI2 and RVI values ​​corresponding to the moment.

[0034] Furthermore, in step S05, in order to further enhance the feature space of different winter crop types, the additive multi-polarization radar coefficient time series data is subjected to a power index transformation, and the calculation formula is as follows:

[0035]

[0036] Among them, SVV ti is the value of the additive multi-polarization radar coefficient time series data SVV at time ti, M is the power base, and its value is 20.

[0037] Further, in step S06, after performing power-exponential transformation on the additive multi-polarization radar coefficient time series data pixel by pixel, a multi-polarization radar coefficient enhanced transformation time series curve is generated; based on the multi-polarization radar coefficient enhanced transformation sequence data set, a time spectrum feature of the multi-polarization radar coefficient enhanced transformation is constructed, which is the multi-polarization radar coefficient enhanced transformation cumulative index APSVV, and the expression is:

[0038]

[0039] in, is the value of the multi-polarization radar coefficient enhancement transformation sequence data at time ti and ti+1.

[0040] Furthermore, in step S07, early identification of different winter crop types is performed based on the time-spectral feature space based on the multimodal vegetation index and the multi-polarization radar coefficient enhanced transformation.

[0041] Furthermore, in step S08, a multimodal knowledge graph for early identification of winter crop types is obtained based on the multimodal vegetation index of early winter crop growth and the time-spectral feature space constructed by the multi-polarization radar coefficient enhancement transformation; based on the multimodal knowledge graph for early identification of winter crop types, the decision rules corresponding to the division of feature spaces of different winter crop types are created as follows:

[0042] If ADVI>ω 1 Or ADRVI>ω 2 , it is judged as winter crop; further judge if APSVV<ω 3 When it is determined to be winter rapeseed, if APSVV≥ω 4 When the number of crops is 1, it is determined to be winter wheat, otherwise it is other winter crops;

[0043] Among them, ADVI, ADRVI, and APSVV are the cumulative index of the early winter crop growth vegetation index based on the optical vegetation index, the cumulative index of the early winter crop growth vegetation index based on the radar vegetation index, and the cumulative index of the multi-polarization radar coefficient enhanced transform.

[0044] Furthermore, step S09 is also included: based on establishing a multimodal knowledge graph for early identification of winter crop types, a spatial distribution map of winter crops in the study area is obtained.

[0045] (1) The method has good robustness. By establishing a knowledge graph for early crop identification, it can be promoted and applied in different regions and years without the need for training based on additional reference sample data.

[0046] (2) The method is simple and easy to implement, with relatively small amount of calculation. Based on the three designed time series indicators and simple decision rules, the automatic identification of winter crop types can be achieved.

[0047] (3) The method does not require high time series remote sensing data, and makes full use of the publicly available and free Sentinel-1 radar time series data to avoid interference caused by the lack of optical data in cloudy and rainy areas, thus ensuring the robustness and application promotion capabilities of the method.

[0048] (4) Based on the overall time series variability characteristics of radar time series signals, time series indicators are designed through power value transformation to improve the separation of the spectral characteristics of different crops in the early growth stages. This can achieve early mapping during the bolting period of winter rapeseed and the greening period of winter wheat. This is different from the post-season crop identification model and improves the timeliness and algorithm accuracy of crop identification.

[0049] In summary, the present invention has the advantages of being simple, easy to operate, and having good robustness. It has the ability to automatically map early winter crop types on a large scale for multiple years without relying on training sample data. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0051] Figure 1 The figure is a flowchart of an implementation of an embodiment of the present invention.

[0052] Figure 2 It is a time series signal diagram of optical and radar vegetation indexes of winter crops and winter fallow fields in an embodiment of the present invention.

[0053] Figure 3 It is a time series signal diagram of the additive radar coefficients of winter rapeseed and winter wheat in an embodiment of the present invention.

[0054] Figure 4 Schematic diagram of the time-spectrum characteristics of the multi-polarization radar coefficient enhancement transformation in an embodiment of the present invention.

[0055] Figure 5 Schematic diagram of decision space for early identification of winter crop types in an embodiment of the present invention.

[0056] Figure 6 This is a spatial distribution map of winter crop types in the study area in the embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail as follows:

[0058] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0059] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0060] The embodiment of the present invention provides a method for early mapping of winter rapeseed and winter wheat based on vegetation-additive multi-polarization radar coefficient-radar vegetation index time series characteristics, such as Figure 1 As shown, the following steps are included:

[0061] Step S01: Establish optical and radar multimodal vegetation index time series data sets respectively;

[0062] Step S02: establishing an additive multi-polarization radar coefficient time series data set;

[0063] Step S03: construct optical and radar multimodal vegetation index difference sequence data sets respectively;

[0064] Step S04: constructing the temporal spectrum characteristics of the multimodal vegetation index in the early stage of winter crop growth;

[0065] Step S05: performing power-index transformation on the additive multi-polarization radar coefficient time series data;

[0066] Step S06: constructing the time-spectrum characteristics of the multi-polarization radar coefficient enhancement transformation;

[0067] Step S07: Evaluate the early identification window of winter crop types;

[0068] Step S08: Establishing a multimodal knowledge graph for early identification of winter crop types;

[0069] Step S09: Obtain the spatial distribution map of winter crops in the study area.

[0070] The following is a further introduction to the preferred solutions of each step of this embodiment:

[0071] Step S01: Establish optical and radar multimodal vegetation index time series dataset.

[0072] Based on the red and near-infrared reflectance data of Sentinel-2, the vegetation index EVI2 is calculated pixel by pixel and period by period to obtain the EVI2 time series dataset of the study area in that year. According to the cloud cover, the observation records with clouds are removed. Based on the effective observation time series dataset of EVI2 index on cloudless days, the Whittaker Smoother data smoothing method is used to construct a smoothed optical vegetation index EVI2 time series dataset.

[0073] Based on the VH and VV multi-polarization radar coefficients of Sentinel-1, the radar vegetation index is calculated pixel by pixel and period by period to obtain the radar vegetation index RVI time series dataset of the study area in that year. The Whittaker Smoother data smoothing method is used to construct a smoothed radar vegetation index RVI time series dataset. Based on the optical and radar multimodal vegetation index time series dataset of the study area, the time series signal diagrams of the optical and radar vegetation index EVI2 and RVI of different winter crops are constructed, see Figure 2 As can be seen from the figure, from November to the end of March, the optical or radar multimodal vegetation index of winter crops showed a significant upward trend compared with winter fallow fields.

[0074] Step S02: establishing an additive multi-polarization radar coefficient time series data set.

[0075] Based on the VH and VV multi-polarization radar coefficients of Sentinel-1, the additive multi-polarization radar coefficients are calculated pixel by pixel and period by period, thus obtaining the additive multi-polarization radar coefficient time series dataset of the study area in that year. The WhittakerSmoother data smoothing method is used to construct a smoothed additive multi-polarization radar coefficient time series dataset.

[0076] Step S03: construct optical and radar vegetation index difference sequence data sets respectively.

[0077] Based on the optical vegetation index EVI2 time series data set, the time series difference is performed pixel by pixel and period by period to obtain the optical vegetation index difference series data set. Based on the radar vegetation index RVI time series data set, the time series difference is performed pixel by pixel and period by period to obtain the radar vegetation index difference series data set.

[0078] Step S04: constructing the temporal spectrum characteristics of the multimodal vegetation index in the early stage of winter crop growth.

[0079] Vegetation index can well reflect the growth and development process of crops. Although considering factors such as winter weed growth, the vegetation index of winter fallow fields may show a certain increasing trend, due to the lack of fertilization and irrigation management measures, the growth rate of its vegetation index is obviously not as good as that of winter crops. By designing the cumulative index of vegetation index in the early stage of winter crop growth, the difference between winter crops and winter fallow fields is characterized. Taking into account that the optical vegetation index may be disturbed by cloud and rainy weather factors, resulting in serious data loss, the time spectrum characteristics of the multimodal vegetation index in the early stage of winter crop growth are designed based on the optical and radar multimodal vegetation index time series data set. In this embodiment, the time spectrum characteristics of the designed multimodal vegetation index in the early stage of winter crop growth are: the cumulative index of the vegetation index in the early stage of winter crop growth based on the optical vegetation index (Accumulated DifferencedVI, referred to as ADVI), the cumulative index of the vegetation index in the early stage of winter crop growth based on the radar vegetation index (Accumulated DifferencedRVI, referred to as ADRVI), and the calculation formulas are:

[0080]

[0081]

[0082] in, Represents the pixel t i EVI2 and RVI values ​​corresponding to the moment; Represents the pixel t i+1 The EVI2 and RVI values ​​corresponding to the moment.

[0083] Step S05: performing a power-exponential transformation on the additive multi-polarization radar coefficient time series data.

[0084] Based on the time series data of additive multi-polarization radar coefficients, the time series variability characteristics of different winter crops in the early growth stage are analyzed. The results show that from the sowing period of winter crops in October to the end of March when the crops return to green and joint, winter wheat shows a sharp downward trend, from about -26dB to less than -36dB. For winter rapeseed, the additive multi-polarization radar coefficient remained basically stable from the end of October when it was sown to the end of February, but from the beginning of March, it gradually showed an upward trend, which was the opposite of the change trend of winter wheat. The time series curves of the additive multi-polarization radar coefficients of winter wheat and winter rapeseed together form a "trumpet"-shaped feature, as shown in Figure 2. Figure 3 In order to further enhance the feature space of different winter crop types such as winter wheat and winter rapeseed, the power index transformation is applied to the additive multi-polarization radar coefficient time series data, and the calculation formula is as follows:

[0085]

[0086] Among them, SVV ti is the value of the additive multi-polarization radar coefficient time series data SVV at time ti, and M is the power base. The purpose of using the power base is to make the value of the winter rapeseed power function close to 1. Figure 3 It can be seen that winter rapeseed gradually approaches the peak value of -20dB near the flowering period. In this embodiment, M is set to 20. After the additive multi-polarization radar coefficient time series data is subjected to power exponential transformation pixel by pixel, a multi-polarization radar coefficient enhanced transformation sequence data set (Pawn function of SVV, abbreviated as PSVV) is generated.

[0087] Step S06: constructing the time-spectrum characteristics of the multi-polarization radar coefficient enhancement transformation.

[0088] After applying power index transformation to the additive multi-polarization radar coefficient time series data pixel by pixel, the multi-polarization radar coefficient enhancement transformation time series curve is generated. For example, for major winter crops such as winter wheat and winter rapeseed, the multi-polarization radar coefficient enhancement transformation time series curve is shown in Figure 4 . As can be seen from the figure, from October to March, the value of winter rapeseed is very small and remains stable, while winter wheat has shown a rapid upward trend since January. Therefore, based on the multi-polarization radar coefficient enhancement transformation sequence data set, the time spectrum characteristics of the multi-polarization radar coefficient enhancement transformation are constructed. In this embodiment, the constructed time spectrum characteristics of the multi-polarization radar coefficient enhancement transformation are the multi-polarization radar coefficient enhancement transformation cumulative index (Accumulated PSVV, referred to as APSVV), and its expression is:

[0089]

[0090] in, is the value of the multi-polarization radar coefficient enhancement transformation sequence data at time ti and ti+1.

[0091] In step S07, the early identification window of winter crop types is evaluated.

[0092] The early identification window of different winter crop types is evaluated according to the F1 score. The crop identification accuracy usually increases with the expansion of the time window, and reaches the maximum value after the crops mature and are harvested, that is, the post-season crop identification accuracy. However, when the early crop identification accuracy is close to or reaches about 90% of the post-season crop identification accuracy, or when the early crop identification accuracy reaches a relatively high classification accuracy, the corresponding crop phenological period at this time is the early crop identification window. In this embodiment, the designed multi-polarization radar coefficient enhanced transformation cumulative index has a good degree of distinction for different winter crop types such as winter wheat and winter rapeseed, starting from the greening period of winter wheat and the budding period of winter rapeseed. At this time, the early identification of winter crop types can obtain a classification accuracy of about 0.86 with F1. Therefore, the time window for early identification of winter crop types can be advanced to the greening period of winter wheat and the budding period of winter rapeseed, and the spatial distribution map of winter crops can be obtained a few months before the winter crops are harvested. That is, starting from the greening period of winter wheat and the budding period of winter rapeseed, the constructed multimodal feature space has already had a high degree of differentiation for different winter crop types such as winter wheat and winter rapeseed; in the greening period of winter wheat and the budding period of winter rapeseed, the performance requirements for early identification of winter crop types using the constructed multimodal feature space have been met.

[0093] The FI calculation formula is:

[0094]

[0095] Among them: F1 is an indicator to measure the accuracy of feature classification, PA is the producer accuracy, and UA is the user accuracy.

[0096] In step S08, a multimodal knowledge graph for early identification of winter crop types is established.

[0097] The winter crop vegetation index shows an obvious upward trend, so the optical or radar early vegetation index accumulation index value is larger than that of the winter fallow field. Due to the differences in plant type and crop canopy structure between winter wheat and winter rapeseed, there is an obvious difference in the additive multi-polarization radar coefficient time series curve. The multi-polarization radar coefficient enhanced transformation accumulation index is further constructed through power index transformation, and the winter wheat value is significantly greater than that of winter rapeseed. Based on the multi-modal vegetation index of winter crop growth in the early stage and the time-spectral feature space constructed by the multi-polarization radar coefficient enhanced transformation, the multi-modal knowledge graph for early identification of winter crop types is obtained; based on the multi-modal knowledge graph for early identification of winter crop types, the decision rules corresponding to the division of feature space of different winter crop types are created as follows (see Figure 5 ):

[0098] If ADVI>ω 1 Or ADRVI>ω 2, it is judged as winter crop; further judge if APSVV<ω 3 When it is determined to be winter rapeseed, if APSVV≥ω 4 When the temperature is 0.040 °C, it is judged as winter wheat, otherwise it is other winter crops.

[0099] Among them, ADVI, ADRVI, and APSVV are the cumulative index of the early winter crop growth vegetation index based on the optical vegetation index, the cumulative index of the early winter crop growth vegetation index based on the radar vegetation index, and the cumulative index of the multi-polarization radar coefficient enhanced transformation. In this example, the threshold ω 1 ,ω 2 ,ω 3 ,ω 4 They are 0.1, 0.02, 10, and 450 respectively. The threshold can be adjusted appropriately according to different regions.

[0100] In step S09, a spatial distribution map of winter crops in the study area is obtained.

[0101] Taking Jianli County, Jingzhou City, Hubei Province as the study area, according to the above process, the cumulative index of winter crop growth early vegetation index based on optical vegetation index and radar vegetation index and the cumulative index of multi-polarization radar coefficient enhanced transformation were established pixel by pixel respectively. Based on the technical process of early identification of winter crop types, at the end of January 2021, corresponding to the budding period of winter rapeseed and the greening period of winter wheat, the distribution map of winter crop types in the study area in 2020-2021 can be obtained (see Figure 6 ).

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0106] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

[0107] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of early automatic identification methods of winter crops based on multimodal knowledge graphs under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.

Claims

1. An automatic early identification method for winter crops based on multimodal knowledge graph, It is characterized in that The following steps are involved: Step S01: Establish optical and radar multimodal vegetation index time series data sets respectively; Step S02: establishing an additive multi-polarization radar coefficient time series data set; Step S03: construct optical and radar multimodal vegetation index difference sequence data sets respectively; Step S04: constructing the temporal spectrum characteristics of the multimodal vegetation index in the early stage of winter crop growth; Step S05: performing power-index transformation on the additive multi-polarization radar coefficient time series data; Step S06: constructing the time-spectrum characteristics of the multi-polarization radar coefficient enhancement transformation; Step S07: Evaluate the early identification window of winter crop types; Step S08: Establishing a multimodal knowledge graph for early identification of winter crop types; In step S01, in the study area, the optical vegetation index EVI2 is calculated pixel by pixel and period by period; according to the cloud coverage, the observation records with clouds are eliminated, and the effective observation time series data set of EVI2 vegetation index on cloudless days is based on each pixel, and then the Whittaker Smoother data smoothing method is used to establish a smoothed optical vegetation index time series data set for the study area; the radar vegetation index RVI is calculated pixel by pixel and period by period, and the calculation formula is: RVI = VH-VV, where VV and VH represent the backscatter coefficients of the polarization of radar images VV and VH respectively; the Whittaker Smoother data smoothing method is used to establish a smoothed radar vegetation index time series data set for the study area, thereby forming an optical and radar multimodal vegetation index time series data set for the study area; In step S05, in order to further enhance the feature space of different winter crop types, the additive multi-polarization radar coefficient time series data is subjected to power index transformation, and the calculation formula is as follows: Among them, SVV ti is the value of the additive multi-polarization radar coefficient time series data SVV at time ti, M is the power base, and its value is 20.

2. The method for automatic early identification of winter crops based on multimodal knowledge graph according to claim 1, Features: In step S02, based on the VH and VV multi-polarization radar coefficients of Sentinel-1, the additive multi-polarization radar coefficients are calculated pixel by pixel and period by period, so as to obtain the additive multi-polarization radar coefficient time series data set of the corresponding year of the study area; the calculation formula of the additive multi-polarization radar coefficient SVV is: SVV=VH+VV, where VV and VH represent the backscattering coefficients of the VV and VH polarizations of the radar images respectively; the Whittaker Smoother data smoothing method is used to construct a smoothed additive multi-polarization radar coefficient time series data set.

3. The method for automatic early identification of winter crops based on multimodal knowledge graph according to claim 1, Features: In step S03, based on the optical and radar vegetation index time series data sets, time series difference is performed pixel by pixel and period by period to obtain optical and radar vegetation index difference sequence data sets respectively.

4. The method for automatic early identification of winter crops based on multimodal knowledge graph according to claim 1, Features: In step S04, based on the optical and radar vegetation index time series data sets, the multimodal vegetation index time spectrum features are extracted, including: the winter crop growth early vegetation index accumulation index ADVI based on the optical vegetation index and the winter crop growth early vegetation index accumulation index ADRVI based on the radar vegetation index, and the calculation formulas are: in, Represents the corresponding pixel t i EVI2 and RVI values ​​corresponding to the moment; Represents the pixel t i+1 The EVI2 and RVI values ​​corresponding to the moment.

5. The method for automatic early identification of winter crops based on multimodal knowledge graph according to claim 1, Features: In step S06, after performing power exponential transformation on the additive multi-polarization radar coefficient time series data pixel by pixel, a multi-polarization radar coefficient enhanced transformation time series curve is generated; based on the multi-polarization radar coefficient enhanced transformation sequence data set, a time spectrum feature of the multi-polarization radar coefficient enhanced transformation is constructed, which is the multi-polarization radar coefficient enhanced transformation cumulative index APSVV, and the expression is: in, is the value of the multi-polarization radar coefficient enhancement transformation sequence data at time ti and ti+1.

6. The method for automatic early identification of winter crops based on multimodal knowledge graph according to claim 1, Features: In step S07, different winter crop types are identified early according to the time-spectral feature space based on the multimodal vegetation index and the multi-polarization radar coefficient enhanced transformation.

7. The method for automatic early identification of winter crops based on multimodal knowledge graph according to claim 1, Features: In step S08, a multimodal knowledge graph for early identification of winter crop types is obtained based on the multimodal vegetation index of early winter crop growth and the time-spectral feature space constructed by the multi-polarization radar coefficient enhancement transformation; based on the multimodal knowledge graph for early identification of winter crop types, the decision rules corresponding to the division of feature spaces of different winter crop types are created as follows: If ADVI>ω 1 Or ADRVI>ω 2 , it is judged as winter crop; further judge if APSVV<ω 3 When it is determined to be winter rapeseed, if APSVV≥ω 4 When the number of crops is 1, it is determined to be winter wheat, otherwise it is other winter crops; Among them, ADVI, ADRVI, and APSVV are the cumulative index of the early winter crop growth vegetation index based on the optical vegetation index, the cumulative index of the early winter crop growth vegetation index based on the radar vegetation index, and the cumulative index of the multi-polarization radar coefficient enhanced transform.

8. The method for automatic early identification of winter crops based on multimodal knowledge graph according to claim 1, Features: The method also includes step S09: obtaining a spatial distribution map of winter crops in the study area based on establishing a multimodal knowledge graph for early identification of winter crop types.

Images