Winter wheat planting area extraction method and device based on multi-modal data fusion

Abstract

The invention provides a winter wheat planting area extraction method and equipment based on multi-modal data fusion. The winter wheat planting area extraction method and equipment can improve the winter wheat extraction precision and environmental adaptability and improve the winter wheat planting area extraction effect. The method comprises the following steps: acquiring an optical satellite remote sensing image and an SAR satellite image corresponding to a target area; processing the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image; performing classification processing on the target optical satellite remote sensing image to obtain land block boundary information; performing feature extraction on the target SAR satellite image based on the plot boundary information to obtain target features; and determining the planting area of the winter wheat in the target area according to the target features and a pre-trained classification model.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular, to a method and device for extracting winter wheat planting area based on multi-modal data fusion. Background Art

[0002] Winter wheat is one of the world's major food crops. Timely understanding of the winter wheat planting area is of great social significance for carrying out work such as monitoring the growth and estimating the yield of winter wheat, and evaluating regional food security. The accuracy of the winter wheat planting area obtained by traditional survey and statistical methods is not high, and it is impossible to obtain the yield estimation of crops and the distribution of planting areas. Moreover, the cost is huge, which cannot well meet the needs of modern agricultural development. Satellite remote sensing technology has the advantages of high spatial resolution, wide distribution area, strong timeliness, etc., so it is widely used in the monitoring of winter wheat planting area.

[0003] However, the extraction of the winter wheat planting area is affected by various factors, such as crop types, planting methods, climate conditions, etc., making it a challenging task to accurately extract its area. Currently, the commonly used remote sensing extraction methods are mainly based on single-modal data, such as optical remote sensing images or radar data. However, due to the limitations of single-modal data, for example, the recognition accuracy of optical remote sensing images is not high in complex terrain or poor climate conditions; while radar data lacks spectral features, and there are certain defects in the recognition of ground objects in areas with large buildings or terrain undulations. Therefore, it is necessary to study a method for extracting winter wheat planting area based on multi-modal data fusion to improve the extraction accuracy and environmental adaptability. Summary of the Invention

[0004] The embodiments of the present invention provide a method and device for extracting winter wheat planting area based on multi-modal data fusion, which improve the accuracy of winter wheat extraction and environmental adaptability, and improve the extraction effect of winter wheat planting area.

[0005] The first aspect of the present invention provides a method for extracting winter wheat planting area based on multi-modal data fusion, including:

[0006] Obtaining the optical satellite remote sensing image and the SAR satellite image corresponding to the target area;

[0007] Processing the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image;

[0008] Performing classification processing on the target optical satellite remote sensing image to obtain plot boundary information;

[0009] Extracting features from the target SAR satellite image based on the plot boundary information to obtain target features;

[0010] Determine the planting area of winter wheat in the target area according to the target feature and the pre-trained classification model.

[0011] The second aspect of the present invention provides a device for extracting the planting area of winter wheat based on multi-modal data fusion, including:

[0012] An acquisition module for acquiring the optical satellite remote sensing image and the SAR satellite image corresponding to the target area;

[0013] An image processing module for processing the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image;

[0014] A classification processing module for classifying the target optical satellite remote sensing image to obtain plot boundary information;

[0015] A feature extraction module for extracting features from the target SAR satellite image based on the plot boundary information to obtain target features;

[0016] A determination module for determining the planting area of winter wheat in the target area according to the target feature and the pre-trained classification model.

[0017] In a possible design, the image processing module is specifically used for:

[0018] Preprocess the optical satellite remote sensing image and the SAR satellite image;

[0019] Register the preprocessed optical satellite remote sensing image and the preprocessed SAR satellite image to obtain the target optical satellite remote sensing image and the target SAR satellite image.

[0020] In a possible design, the image processing module classifies the target optical satellite remote sensing image to obtain plot boundary information, including:

[0021] Coarsely classify the target optical satellite remote sensing image to obtain a preliminary classification result;

[0022] Eliminate the regions in the preliminary classification result that are more different from the winter wheat features than a preset threshold to obtain an elimination classification result;

[0023] Process each region in the elimination classification result to obtain the plot boundary information, where the plot boundary information is an image polygon of any scale and with similar attribute information.

[0024] In a possible design, the feature extraction module is specifically used for:

[0025] Convert the segmentation vector corresponding to the plot boundary information into the SAR image coordinate system;

[0026] Perform buffer processing on the plot boundary information;

[0027] Extract the multi-polarization feature combination and calculate the texture feature for each plot in the plot boundary information at each time point:

[0028] Stack the multi-polarization feature combination and the texture feature in time series to form a multi-dimensional feature cube, and enhance the time series features in the multi-dimensional feature cube to obtain the target feature.

[0029] In a possible design, the determining module is specifically configured to:

[0030] Convert the target feature into a two-dimensional array;

[0031] Input the two-dimensional array into the classification model to obtain a prediction result;

[0032] Reconstruct the prediction result into a two-dimensional matrix according to the row and column numbers of the optical satellite remote sensing image;

[0033] Generate the spatial distribution map corresponding to the winter wheat based on the two-dimensional matrix;

[0034] Remove the small noises in the spatial distribution map to obtain the target spatial distribution map;

[0035] Calculate the planting area of the winter wheat in the target spatial distribution map.

[0036] In a possible design, the determining module calculates the planting area of the winter wheat in the target spatial distribution map, including:

[0037] Count the number of pixels of the winter wheat and the pixel area of each pixel in the target spatial distribution map;

[0038] Determine the planting area according to the number of pixels and the pixel area;

[0039] Or,

[0040] Convert the grid corresponding to the target spatial distribution map into a polygon vector;

[0041] Process the polygon vector through GIS software to obtain the planting area.

[0042] In a possible design, the determining module is further configured to:

[0043] Determine the actual area of the winter wheat within the target area;

[0044] Adjust the classification model based on the actual area and the planted area.

[0045] The third aspect of the present invention provides an electronic device, including a memory and a processor. When the processor executes a computer management program stored in the memory, the steps of the method for extracting the planted area of winter wheat based on multi-modal data fusion as described in the first aspect above are implemented.

[0046] The fourth aspect of the present invention provides a computer-readable storage medium, on which a computer management program is stored. When the computer management program is executed by a processor, the steps of the method for extracting the planted area of winter wheat based on multi-modal data fusion as described in the first aspect above are implemented.

[0047] In summary, it can be seen that in the embodiments provided by the present invention, through multi-modal data fusion, the advantages of optical and SAR image data are fully utilized, the problem of difficult acquisition of long-time series data is solved, the distribution information of winter wheat plots can be extracted relying on less temporal remote sensing image data, and the accuracy and environmental adaptability of winter wheat extraction are improved; at the same time, the feature fusion efficiency is improved through rough classification of ground objects, and the error caused by speckle noise in the SAR image is removed through the multi-scale segmentation method, improving the effect of extracting the planted area of winter wheat. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic flowchart of a method for extracting the planted area of winter wheat based on multi-modal data fusion provided by an embodiment of the present invention;

[0049] Figure 2 It is a schematic virtual structure diagram of a device for extracting the planted area of winter wheat based on multi-modal data fusion provided by an embodiment of the present invention;

[0050] Figure 3 It is a schematic hardware structure diagram of a device for extracting the planted area of winter wheat based on multi-modal data fusion provided by an embodiment of the present invention;

[0051] Figure 4 It is a schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0054] In the following description, specific embodiments of the present invention will be described with reference to steps and symbols executed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to as being executed by a computer several times. The computer execution referred to herein includes the operations of a computer processing unit that represents electronic signals in a structured form of data. This operation transforms the data or maintains it at a position in the computer's memory system, which can be reconfigured or otherwise changed in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of the present invention are described in the above text, which does not represent a limitation. Those skilled in the art will understand that the various steps and operations described below can also be implemented in hardware.

[0055] The principles of the present invention are operated using many other general-purpose or specific-purpose computing, communication environments or configurations. Examples of well-known computing systems, environments, and configurations suitable for the present invention may include (but are not limited to) mobile phones, personal computers, servers, multi-processor systems, microcomputer-based systems, mainframe computers, and distributed computing environments, including any of the above systems or devices.

[0056] The terms "first", "second", "third", etc. in the present invention are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0057] In China, the types of crops are complex, and the spectral characteristics between different crops are relatively similar. It is difficult to extract the target crops only through the spectral recognition technology of a single temporal phase. The current research mainly analyzes and applies the growth laws of crops and uses multi-temporal remote sensing data for crop recognition. However, the current crop recognition methods based on multi-temporal remote sensing data often only use a single remote sensing data source or optical images of different data sources, and it is difficult to obtain effective results in many scenarios. The reasons are as follows: on the one hand, it is difficult to form an observation sequence of high-resolution remote sensing data within the required period due to the long revisit cycle and cloud interference; on the other hand, there are relatively serious mixed pixel problems in low-resolution remote sensing data. In addition, these methods often cannot fully combine the temporal characteristics and spectral characteristics of the target crops, so the scheme design is often too complex or the crop extraction accuracy is not high.

[0058] The biggest advantage of SAR images compared with optical images is that they can penetrate clouds and rain, are not affected by weather and time, can work all day and all weather, and can stably obtain a continuous image time series. Some current research results on winter wheat show that during the growth process of winter wheat, the backscattering characteristics change significantly, and there are large differences from the backscattering characteristics of other vegetation, which can provide a theoretical basis for the extraction of the planting area of winter wheat. In order to overcome the problems of the lack of optical satellite images in the key phenological periods of winter crops and the similarity of the spectra of major crops, the Smart Agriculture Team of the College of Agriculture, Nanjing Agricultural University explored a strategy for distinguishing winter wheat and winter rape based on SAR, and pointed out that the VH polarization based on the flowering period of rape can most effectively distinguish winter wheat and other crop pixels. However, the significant salt-and-pepper noise in SAR images and the lower signal-to-noise ratio compared with optical images also pose great challenges to crop recognition in small and fragmented areas of plots (especially the identification of plot boundaries).

[0059] Due to the working limitations and different principles of remote sensing data, a single sensor is difficult to fully reflect the characteristics of land cover. Therefore, the methods for extracting the planting area of crops using pure optical images and pure SAR images both have limitations. The current methods for extracting the planting area of crops based on the fusion of multi-modal remote sensing images mainly take rice as the research object, and there are few research cases on winter wheat.

[0060] The mature stage of winter wheat is a crucial period for remote sensing monitoring. Around May when winter wheat reaches maturity, many planting areas are cloudy and rainy, making it difficult to obtain complete optical image data during the key growth and development stages of crops. As a result, it is impossible to construct a high-quality large-area, long-time image dataset. Therefore, a detection method that is not affected by weather and time is needed. While conducting macroscopic large-area monitoring of winter wheat, local extraction is also required. Current research on extracting the planting area of winter wheat mostly focuses on large, regular farmlands in the northern region. In contrast to the northern region, farmlands in southern China have problems such as complex planting structures and scattered plots, and are easily affected by mixed pixels during the recognition of temporal features. Therefore, it is necessary to fully combine the advantages of temporal feature recognition and spectral feature recognition to improve the current crop recognition method based on multi-temporal remote sensing data.

[0061] The method for extracting the planting area of winter wheat based on multi-modal data fusion provided by the present invention will be described from the perspective of the device for extracting the planting area of winter wheat based on multi-modal data fusion. This device for extracting the planting area of winter wheat based on multi-modal data fusion can be a server or a service unit in the server. For the sake of convenience in description, the device for extracting the planting area of winter wheat based on multi-modal data fusion will be described as a server below.

[0062] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of the method for extracting the planting area of winter wheat based on multi-modal data fusion provided by an embodiment of the present invention, including:

[0063] 101. Obtain the optical satellite remote sensing image and the SAR satellite image corresponding to the target area.

[0064] In this embodiment, the server can obtain the optical satellite remote sensing image and the SAR satellite image corresponding to the target area. Among them, the target area is the area where the planting area of winter wheat is to be determined. The optical satellite remote sensing image is for the mature stage of winter wheat (the key phenological stage, and of course, it can also be adjusted according to the actual situation, and no specific limitation is made). The SAR satellite image is a SAR satellite image in the C band. Here, the specific method of obtaining the satellite remote sensing image and the SAR satellite image is not limited as long as they can be obtained.

[0065] 102. Process the optical satellite remote sensing image and the SAR satellite image to obtain the target optical satellite remote sensing image and the target SAR satellite image.

[0066] In this embodiment, after the server obtains the optical satellite remote sensing image and the SAR satellite image, it can process the optical satellite remote sensing image and the SAR satellite image to obtain the target optical satellite remote sensing image and the target SAR satellite image. Specifically, the server can first preprocess the optical satellite image and the SAR satellite image:

[0067] The preprocessing of the optical satellite remote sensing image includes radiometric correction, atmospheric correction, and geometric correction. Among them, radiometric correction refers to converting the original DN value into the apparent reflectance (TOA), using the calibration parameters (gain / offset) provided by the sensor to eliminate the influence of sensor response differences and illumination conditions. The specific formula is as follows:

[0068] θ sun is the solar zenith angle corresponding to the optical satellite remote sensing image;

[0069] The purpose of atmospheric correction is to remove the influence of atmospheric scattering and absorption and obtain the true surface reflectance. The purpose of geometric correction is to correct the geometric distortion caused by terrain undulation and sensor attitude.

[0070] The preprocessing of the SAR satellite image includes radiometric calibration (converting the original data into the backscattering coefficient), multi-look processing (reducing speckle noise and improving the signal-to-noise ratio), filtering and denoising, and geocoding (converting the slant range coordinate system into the geographic coordinate system).

[0071] The server registers the preprocessed optical satellite remote sensing image and the preprocessed SAR satellite remote sensing image. Specifically, it extracts feature points from the optical satellite remote sensing image, adapts the speckle noise of the SAR satellite remote sensing image, and uses an automatic registration tool for registration to obtain the target satellite optical remote sensing image and the target SAR satellite image, so that the ground objects between the two can be accurately corresponding, thus meeting the subsequent requirements for image data fusion.

[0072] 103. Classify the target optical satellite remote sensing image to obtain the plot boundary information.

[0073] In this embodiment, after the server determines the target satellite remote sensing image, it can classify the target satellite remote sensing image to obtain the plot boundary information. The following will detail how to classify the target optical satellite remote sensing image to obtain the plot boundary information:

[0074] Coarsely classify the target optical satellite remote sensing image to obtain a preliminary classification result;

[0075] Eliminate the regions in the preliminary classification result that are more different from the characteristics of winter wheat than the preset threshold to obtain an elimination classification result;

[0076] Process each region in the classification result after elimination to obtain plot boundary information, where the plot boundary information is an image polygon of any scale and with similar attribute information.

[0077] That is, the server can use a supervised classification algorithm to roughly classify the target optical image to obtain a preliminary classification result. Here, it is illustrated by taking the preliminary classification result including five types of ground objects such as winter wheat, water body, impervious water body, other vegetation (other vegetation in the target optical satellite remote sensing image except winter wheat), and bare land as an example. Of course, it can also include other ground objects, and specific ones are not limited.

[0078] It can be understood that in the rough classification result, regions such as water bodies and bare land with large differences from the characteristics of winter wheat can be directly eliminated, that is, eliminate the regions in the preliminary classification result with a difference from the characteristics of winter wheat greater than a preset threshold to obtain the classification result after elimination;

[0079] After that, process each region in the classification result after elimination to obtain plot boundary information, where the plot boundary information is an image polygon of any scale and with similar attribute information. That is, after the server obtains the classification result after elimination, it can use a multi-scale segmentation method to process the easily confused regions identified as winter wheat, impervious water body, and other vegetation after rough classification, generate image polygons (objects) of any scale and with similar attribute information as natural plot boundaries, that is, plot boundary information, and then perform object-oriented analysis on the target SAR image based on this as the basic unit, so as to eliminate the adverse effects of a large number of speckle noises in the target SAR remote sensing image on the analysis and interpretation of the image.

[0080] It should be noted that multi-scale segmentation is a necessary prerequisite for classification recognition and information extraction. The selection of segmentation parameters (including size, shape, and compactness) directly affects the quality of the segmentation result, and further affects the final classification accuracy. Among all segmentation parameters, the shape parameter determines the influence of spectral values and shapes when objects are formed; the compactness parameter determines the compactness of the boundaries of the segmented objects; the size factor is the most important parameter among them and has an important impact on the segmentation result. When the size factor is set small, the overall segmentation result has a significant over-segmentation problem and the segmented objects are fragmented; when the size factor is set large, the overall segmentation result has a significant under-segmentation problem, and the obtained boundaries may contain multiple plots. When the size factor is selected as 15 (segmentation is performed based on the eCognition software), the segmentation result coincides with the actual plot.

[0081] 104. Extract features from the target SAR satellite image based on the plot boundary information to obtain target features.

[0082] In this embodiment, the server can process the target SAR satellite image using the plot boundary information obtained from the multi-scale segmentation parameters, and extract the target features in the SAR time-series feature data in the area where the winter wheat characteristics are more obvious. It can be understood that, according to the C-band SAR image data in the middle and late May when the winter wheat is in the mature stage, the VV+VH multi-polarization combination features and texture features of the SAR image are jointly used as the preferred feature set (i.e., the target features) for extracting the winter wheat planting area in the present invention. The preferred feature set includes several polarization features and several texture features. Specifically, the server can first convert the segmentation vector corresponding to the plot boundary information into the SAR image coordinate system and perform buffer processing on the plot boundary information; extract the multi-polarization feature combination (the multi-polarization feature combination includes the VV / VH ratio and the VV-VH difference. Calculate the VV / VH ratio to enhance the sensitivity to the winter wheat canopy structure, and calculate the VV-VH difference to distinguish the scattering differences of different vegetation types) and calculate the texture features (the texture features can include, for example, contrast, entropy, correlation, and mean) for each plot in the plot boundary information at each time point: stack the multi-polarization feature combination and the texture features in time series to form a multi-dimensional feature cube, and enhance the time-series features in the multi-dimensional feature cube to obtain the target features.

[0083] It should be noted that before feature extraction, the target SAR satellite image can be cropped according to the vector boundary to reduce the computational amount of invalid data; in addition, when extracting texture features, the selection of the texture window can be adjusted according to the resolution of the target SAR satellite image. It can be understood that for winter wheat fields, due to uniform growth, the contrast can be appropriately selected to be lower (for example, less than 15); for forest land, due to tree trunks and shadows, the contrast can be appropriately selected to be higher (for example, greater than 25). The texture complexity of wheat fields is medium (entropy value 2.5 - 3.5), and for bare land, the entropy value is relatively high due to random noise (for example, greater than 4). Due to periodic planting, the correlation between pixels of winter wheat can be selected to be greater than 0.7, while for impervious surfaces, the correlation is low due to artificial structures (for example, less than 0.4).

[0084] 105. Determine the planting area of winter wheat in the target area according to the target features and the pre-trained classification model.

[0085] In this embodiment, after obtaining the target features, the server can determine the planting area of winter wheat in the target area according to the target features and the pre-trained classification model. Specifically, the server can first convert the target features into a two-dimensional array (the order of the features included in the two-dimensional array is the same as the order of the features included in the data during the training of the classification model, for example, VV first, then VH, and finally the texture features), and input the two-dimensional array into the classification model to obtain a prediction result (the prediction result can be, for example, 0 for non-winter wheat and 1 for winter wheat); then reconstruct the prediction result into a two-dimensional matrix according to the row and column numbers of the optical satellite remote sensing image, generate a spatial distribution map based on the two-dimensional matrix, use opening operation (the method of erosion first and then dilation) and closing operation to remove small noises, finally label the connected regions, merge the fragmented patches, and remove the non-farmland regions with an area smaller than a certain threshold (here, it can be based on the agricultural management unit standard) to obtain the target spatial distribution map; finally, calculate the planting area of winter wheat in the target spatial distribution map.

[0086] It should be noted that when calculating the planting area of winter wheat, the number of pixels of winter wheat and the pixel area of each pixel in the target spatial distribution map can be counted, and then the planting area can be determined according to the pixel area and the number of pixels; or, the grid corresponding to the target spatial distribution map can be converted into a polygon vector, and then the polygon vector can be processed by GIS software to obtain the planting area.

[0087] It should be noted that the classification model includes an SVM model and a CART model. The SVM outputs the class probability of the pixel, and the CART outputs the class label. After determining the planting area of winter wheat, the accuracy of the planting area of winter wheat can be evaluated. Here, the overall classification accuracy (OA), user accuracy (UA), producer accuracy (PA), and Kappa coefficient and other indicators are mainly used, and the above indicators can be calculated based on the confusion matrix of the classification results.

[0088] While conducting theoretical accuracy analysis through coefficient indicators, the winter wheat extraction results are compared and verified with the field survey results. At the same time, the method of fusing multi-temporal optical images and extracting the planting area of winter wheat only using SAR time-series images is used as a control group to compare the winter wheat extraction accuracy under the three classification methods, and further judge the correctness and reliability of extracting the planting area of winter wheat based on the present invention. The comparison results show that the method of the present invention has significantly improved in various accuracy indicators compared with the method of only using SAR images or only using optical images, indicating that the addition of the boundary information of the optical image plot plays an important role in improving the accurate identification of winter wheat in time-series SAR images. In addition, the accuracy of the method of the present invention is better than the winter wheat identification accuracy of multi-temporal optical images, which indicates that in the case of lack of multi-scene optical data and relatively fragmented plots, the method provided by the present invention can stably and effectively identify winter wheat.

[0089] In summary, it can be seen that in the embodiments provided by the present invention, through multi-modal data fusion, the advantages of optical and SAR image data are fully utilized, the problem of difficult acquisition of long-term sequence data is solved, the distribution information of winter wheat plots can be extracted relying on less-temporal remote sensing image data, and the accuracy and environmental adaptability of winter wheat extraction are improved; meanwhile, the feature fusion efficiency is improved through rough classification of ground objects, and the error caused by speckle noise in the SAR image is removed through a multi-scale segmentation method, improving the extraction effect of the winter wheat planting area.

[0090] The above has described the embodiments of the present invention from the method for extracting the winter wheat planting area based on multi-modal data fusion. Next, the embodiments of the present invention will be described from the device for extracting the winter wheat planting area based on multi-modal data fusion.

[0091] Please refer to Figure 2 , the virtual structure schematic diagram of the computing device for exciting the intermediate frequency point unit of the array antenna in the embodiments of the present invention. The device 200 for extracting the winter wheat planting area based on multi-modal data fusion includes:

[0092] An acquisition module 201, configured to acquire the optical satellite remote sensing image and the SAR satellite image corresponding to the target area;

[0093] An image processing module 202, configured to process the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image;

[0094] A classification processing module 203, configured to perform classification processing on the target optical satellite remote sensing image to obtain plot boundary information;

[0095] A feature extraction module 204, configured to extract features from the target SAR satellite image based on the plot boundary information to obtain target features;

[0096] A determination module 205, configured to determine the planting area of winter wheat in the target area according to the target features and a pre-trained classification model.

[0097] In a possible design, the image processing module 202 is specifically configured to:

[0098] Perform preprocessing on the optical satellite remote sensing image and the SAR satellite image;

[0099] Register the preprocessed optical satellite remote sensing image and the preprocessed SAR satellite image to obtain the target optical satellite remote sensing image and the target SAR satellite image.

[0100] In a possible design, the image processing module 202 classifies the target optical satellite remote sensing image to obtain the plot boundary information, including:

[0101] Coarsely classify the target optical satellite remote sensing image to obtain a preliminary classification result;

[0102] Eliminate the regions in the preliminary classification result that differ from the winter wheat characteristics by more than a preset threshold to obtain an elimination classification result;

[0103] Process each region in the elimination classification result to obtain the plot boundary information, where the plot boundary information is an image polygon of any scale and with similar attribute information.

[0104] In a possible design, the feature extraction module 204 is specifically configured to:

[0105] Convert the segmentation vector corresponding to the plot boundary information into the SAR image coordinate system;

[0106] Perform buffer processing on the plot boundary information;

[0107] Extract the multi-polarization feature combination and calculate the texture feature for each plot in the plot boundary information at each time point:

[0108] Stack the multi-polarization feature combination and the texture feature in time series to form a multi-dimensional feature cube, and enhance the time series features in the multi-dimensional feature cube to obtain the target features.

[0109] In a possible design, the determination module 205 is specifically configured to:

[0110] Convert the target features into a two-dimensional array;

[0111] Input the two-dimensional array into the classification model to obtain a prediction result;

[0112] Reconstruct the prediction result into a two-dimensional matrix according to the row and column numbers of the optical satellite remote sensing image;

[0113] Generate the spatial distribution map corresponding to the winter wheat based on the two-dimensional matrix;

[0114] Eliminate the small noises in the spatial distribution map to obtain the target spatial distribution map;

[0115] Calculate the planting area of the winter wheat in the target spatial distribution map.

[0116] In a possible design, the determination module 205 calculates the planting area of the winter wheat in the target spatial distribution map, including:

[0117] Count the number of pixels of the winter wheat in the target spatial distribution map and the pixel area of each pixel;

[0118] Determine the planting area according to the number of pixels and the pixel area;

[0119] Or,

[0120] Convert the grid corresponding to the target spatial distribution map into a polygon vector;

[0121] Process the polygon vector through GIS software to obtain the planting area.

[0122] In a possible design, the determining module 205 is further configured to:

[0123] Determine the actual area of the winter wheat in the target area;

[0124] Adjust the classification model based on the actual area and the planting area.

[0125] Above Figure 3 The winter wheat planting area extraction device based on multi-modal data fusion in the embodiments of the present invention has been described from the perspective of modular functional entities. Next, the winter wheat planting area extraction device based on multi-modal data fusion in the embodiments of the present invention will be described in detail from the perspective of hardware processing. Please refer to Figure 3 , the schematic diagram of the embodiment of the winter wheat planting area extraction device 300 based on multi-modal data fusion in the embodiments of the present invention. The winter wheat planting area extraction device 300 based on multi-modal data fusion includes:

[0126] An input device 301, an output device 302, a processor 303, and a memory 304 (where the number of processors 303 can be one or more, Figure 3 Taking one processor 303 as an example). In some embodiments of the present invention, the input device 301, the output device 302, the processor 303, and the memory 304 can be connected through a communication bus or other means. Among them, Figure 3 Taking the connection through the communication bus as an example.

[0127] Among them, by calling the operation instructions stored in the memory 304, the processor 303 is configured to execute the following steps:

[0128] Obtain the optical satellite remote sensing image and the SAR satellite image corresponding to the target area;

[0129] Process the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image;

[0130] Classify the target optical satellite remote sensing image to obtain plot boundary information;

[0131] Extract features from the target SAR satellite image based on the plot boundary information to obtain target features;

[0132] Determine the planting area of winter wheat in the target area according to the target features and a pre-trained classification model.

[0133] By invoking the operation instructions stored in the memory 304, the processor 303 is further configured to execute Figure 1 any one of the corresponding embodiments.

[0134] Please refer to Figure 4 , Figure 4 which is a schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention.

[0135] As Figure 4 shown, an embodiment of the present invention provides an electronic device, including a memory 410, a processor 420, and a computer program 411 stored on the memory 410 and executable on the processor 620. When the processor 420 executes the computer program 411, the following steps are implemented:

[0136] Obtain the optical satellite remote sensing image and the SAR satellite image corresponding to the target area;

[0137] Process the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image;

[0138] Classify the target optical satellite remote sensing image to obtain plot boundary information;

[0139] Extract features from the target SAR satellite image based on the plot boundary information to obtain target features;

[0140] Determine the planting area of winter wheat in the target area according to the target features and a pre-trained classification model.

[0141] In a specific implementation process, when the processor 420 executes the computer program 411, it can implement Figure 1 any one of the corresponding embodiments.

[0142] Since the electronic device introduced in this embodiment is the device used in the computing device for exciting the intermediate frequency point unit of an array antenna in the embodiments of the present invention, based on the method introduced in the embodiments of the present invention, those skilled in the art can understand the specific implementation manners of the electronic device in this embodiment and its various forms of changes. Therefore, the specific implementation of how this electronic device implements the method in the embodiments of the present invention will not be described in detail here. As long as the device used by those skilled in the art to implement the method in the embodiments of the present invention belongs to the scope protected by the present invention.

[0143] Please refer to Figure 5 , Figure 5 which is a schematic diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention.

[0144] As Figure 5 shown, an embodiment of the present invention also provides a computer-readable storage medium 500, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, the following steps are implemented:

[0145] Obtain the optical satellite remote sensing image and the SAR satellite image corresponding to the target area;

[0146] Process the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image;

[0147] Perform classification processing on the target optical satellite remote sensing image to obtain plot boundary information;

[0148] Extract features from the target SAR satellite image based on the plot boundary information to obtain target features;

[0149] Determine the planting area of winter wheat in the target area according to the target features and a pre-trained classification model.

[0150] In the specific implementation process, the computer program 511 can be executed by the processor to implement Figure 1 any one of the implementation manners in the corresponding embodiments.

[0151] It should be noted that in the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0153] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0154] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0156] Embodiments of the present invention also provide a computer program product, which includes computer software instructions. When the computer software instructions run on a processing device, the processing device is caused to execute as Figure 1 the processes in the corresponding embodiments.

[0157] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, they implement, wholly or partly, the processes or functions described in the embodiments of the present invention. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (such as coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless means (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0158] Those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein again.

[0159] In several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed among each other may be indirect couplings or communication connections through some interfaces, devices, or units, and may be in electrical, mechanical, or other forms.

[0160] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0161] In addition, in each embodiment of the present invention, each functional unit may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0162] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0163] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A method for extracting the planting area of winter wheat based on multi-modal data fusion, characterized in that, Including: Obtain the optical satellite remote sensing image and the SAR satellite image corresponding to the target area; Process the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image; Perform classification processing on the target optical satellite remote sensing image to obtain plot boundary information; Extract features from the target SAR satellite image based on the plot boundary information to obtain target features; Determine the planting area of winter wheat in the target area according to the target features and a pre-trained classification model.

2. The method according to claim 1, characterized in that, The processing of the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image includes: Preprocess the optical satellite remote sensing image and the SAR satellite image; Register the preprocessed optical satellite remote sensing image and the preprocessed SAR satellite image to obtain the target optical satellite remote sensing image and the target SAR satellite image.

3. The method according to claim 1, characterized in that The classification processing of the target optical satellite remote sensing image to obtain plot boundary information includes: Coarsely classify the target optical satellite remote sensing image to obtain a preliminary classification result; Eliminate the regions in the preliminary classification result that have a difference greater than a preset threshold from the characteristics of winter wheat to obtain an elimination classification result; Process each region in the elimination classification result to obtain the plot boundary information, where the plot boundary information is an image polygon of any scale and with similar attribute information.

4. The method according to claim 1, characterized in that The extracting features from the target SAR satellite image based on the plot boundary information to obtain target features includes: Convert the segmentation vector corresponding to the plot boundary information into the SAR image coordinate system; Perform buffer processing on the plot boundary information; Extract a multi-polarization feature combination and calculate texture features for each plot in the plot boundary information at each time point; Stack the multi-polarization feature combination and the texture features in a time series to form a multi-dimensional feature cube, and enhance the time series features in the multi-dimensional feature cube to obtain the target features.

5. The method according to any one of claims 1 to 4, characterized in that The determining the planting area of winter wheat in the target area according to the target features and a pre-trained classification model includes: Convert the target features into a two-dimensional array; Input the two-dimensional array into the classification model to obtain a prediction result; Reconstruct the prediction result into a two-dimensional matrix according to the row and column numbers of the optical satellite remote sensing image; Generate a spatial distribution map corresponding to the winter wheat based on the two-dimensional matrix; Eliminate small noises in the spatial distribution map to obtain a target spatial distribution map; Calculate the planting area of winter wheat in the target spatial distribution map.

6. The method according to claim 5, characterized in that, The calculating the planting area of winter wheat in the target spatial distribution map includes: Count the number of pixels of winter wheat in the target spatial distribution map and the pixel area of each pixel; Determine the planting area according to the number of pixels and the pixel area; Or, Convert the grid corresponding to the target spatial distribution map into a polygon vector; The polygon vector is processed by GIS software to obtain the planting area.

7. The method according to any one of claims 1 to 4 and 6, characterized in that, The method further includes: determining the actual area of the winter wheat in the target area; adjusting the classification model based on the actual area and the planting area.

8. An extraction device for winter wheat planting area based on multi-modal data fusion, characterized in that, including: an acquisition module for acquiring the optical satellite remote sensing image and the SAR satellite image corresponding to the target area; an image processing module for processing the optical satellite remote sensing image and the SAR satellite image to obtain a target optical satellite remote sensing image and a target SAR satellite image; a classification processing module for classifying the target optical satellite remote sensing image to obtain plot boundary information; a feature extraction module for extracting features from the target SAR satellite image based on the plot boundary information to obtain target features; a determination module for determining the planting area of winter wheat in the target area according to the target features and a pre-trained classification model.

9. An electronic device, characterized in that, including: a memory and a processor, the processor is used to implement the steps of the method for extracting the planting area of winter wheat based on multi-modal data fusion according to any one of claims 1 to 7 when executing the computer management program stored in the memory.

10. A computer-readable storage medium, on which a computer management program is stored, characterized in that, The computer management program implements the steps of the method for extracting the planting area of winter wheat based on multi-modal data fusion according to any one of claims 1 to 7 when executed by the processor.

Images