Cultivated land identification method, system and equipment based on modal fusion and medium

Abstract

The invention provides a cultivated land identification method, system and device based on modal fusion and a medium, and belongs to the technical field of cultivated land identification. The method comprises the following steps: acquiring image data of to-be-identified cultivated land through an unmanned aerial vehicle, and acquiring GIS vector data and historical time sequence images of the to-be-identified cultivated land to form a basic data set; performing correction, splicing and segmentation processing on image data in the basic data set, performing unified coordinate system processing on GIS vector data in the basic data set, and calculating field piece regularity; processing the basic data set by using a single-mode visual perception model, outputting a preliminary detection result containing a probability graph and boundary information, correcting the preliminary detection result by using a multi-mode logical reasoning module in combination with auxiliary data, and generating an identification result; and carrying out visualization and vectorization processing on the identification result to generate visual data, and displaying the visual data. According to the invention, through multi-modal fusion and multi-dimensional processing, the precision and reliability of cultivated land identification are significantly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of cultivated land identification, and more particularly to a cultivated land identification method, system, device and medium based on modal fusion. BACKGROUND

[0002] In the field of agricultural geographic information identification technology, with the development of remote sensing technology and geographic information system (GIS), cultivated land identification, as an important part of agricultural resource monitoring and management, has gradually attracted widespread attention. Traditional cultivated land identification methods mainly rely on single modal data, such as using only visible light or multispectral images collected by unmanned aerial vehicles or satellites, and classifying and detecting cultivated land by extracting visual features such as color and texture. However, these methods often have problems such as insufficient accuracy, high omission rate and lack of compliance when facing complex agricultural scenes such as fallow period bare land, cloudy weather or fragmented fields.

[0003] Currently, although some studies have attempted to improve the accuracy of cultivated land identification by introducing multi-modal data, these methods mostly remain at the level of simple data superposition and lack the mining and utilization of deep correlations between multi-source data. For example, GIS vector data (such as roads, irrigation facilities, field boundaries, etc.) are not systematically fused for spatial relationship reasoning, multi-year historical time series image data are not fully utilized for time series analysis, and agricultural domain knowledge (such as basic farmland policy, fallow rules, etc.) is not effectively embedded into the identification process.

[0004] Therefore, the related technology still has significant defects in cultivated land identification accuracy, adaptability to complex scenes and compliance. These problems not only limit the application effect of cultivated land identification technology in agricultural resource digital management, but also affect the scientificity of agricultural planning. SUMMARY

[0005] To solve the above problems, the purpose of the present application is to provide a cultivated land identification method, system, device and medium based on modal fusion, which significantly improves the accuracy and reliability of cultivated land identification through multi-modal fusion and multi-dimensional processing.

[0006] To achieve the above purpose, the present application realizes the following technical solutions: In a first aspect, the present application provides a cultivated land identification method based on modal fusion, comprising: An unmanned aerial vehicle is used to collect image data of the cultivated land to be identified containing RGB and multispectral bands, and GIS vector data and historical time series images of the cultivated land to be identified are obtained simultaneously to form a basic data set; The image data in the basic data set is corrected, spliced and segmented, the GIS vector data in the basic data set is processed in a unified coordinate system and the regularity of the field is calculated; The preset monomodal visual perception model is used to process the basic data set, and a preliminary detection result containing a probability graph and boundary information is output. The preset multimodal logical inference module is used to correct the preliminary detection result in combination with auxiliary data, and a recognition result is generated. The recognition result is visualized and vectorized, and visualized data is generated and displayed.

[0007] In an optional implementation, the image data of the to-be-identified cultivated land containing RGB and multispectral bands is collected by the sensor carried by the unmanned aerial vehicle, and the GIS vector data and historical time-series images of the to-be-identified cultivated land are acquired to constitute a basic data set, including: The visible light sensor and the multispectral sensor carried by the unmanned aerial vehicle are used to acquire the image data of the to-be-identified cultivated land containing RGB three-channel images and near-infrared and red edge bands according to the set flight parameters, so as to extract the color, texture features and index for calculating the cultivated land, respectively; The land use status information, irrigation facility account and surveyed road network information related to the to-be-identified cultivated land are acquired to extract the related GIS vector data, the historical time-series images are supplemented by Sentinel 2 L2A data, and data alignment and rule modeling processing are performed; The acquired image data, GIS vector data and historical time-series images constitute a basic data set.

[0008] In an optional implementation, the image data in the basic data set is corrected, spliced and segmented, including: The radiation calibration and atmospheric correction, orthorectification and splicing, and superpixel segmentation operation are sequentially performed on the image data in the basic data set, interference factors in the image data are eliminated, core features are extracted, and the image is converted into feature data suitable for cultivated land identification.

[0009] In an optional implementation, the GIS vector data in the basic data set is processed in a unified coordinate system and the field block regularity is calculated, including: The metadata of the GIS vector data is read by using ArcGIS or QGIS to determine the currently used coordinate system of the GIS vector data in the basic data set. A target coordinate system is set, the source vector data layer is loaded, and the data is converted into the coordinates of the target coordinate system through the built-in projection conversion algorithm. The GIS vector data in the target coordinate system is loaded by using the Shapely library based on Python, the field block vector data is identified, the geometric vertex coordinate information of each field block polygon is extracted, and a corresponding ShapelyPolygon object is constructed. Call the convex_hull method provided by Shapely to calculate the convex hull of each field Polygon object, which includes the minimum convex polygon of all vertices of the field; Get the area of the original field polygon through the area attribute of the Polygon object and calculate the corresponding convex hull area.

[0010] Calculate the regularity value corresponding to each field by the formula: regularity = actual area of the field / convex hull area; Add the calculated regularity value of each field as an attribute field to the corresponding vector data layer.

[0011] In an optional embodiment, the preset single-modal visual perception model is used to process the basic data set, and a preliminary detection result containing a probability map and boundary information is output, which includes: A trained U-Net++ or YOLOv8n-seg model is used to construct a single-modal visual perception model; A combination of cross-entropy and Dice loss is used as a loss function, and the class weights are reasonably set according to the sample class imbalance; An AdamW optimizer is selected, and the initial learning rate, batch size, and training period are set for model training; The image data in the basic data set is input into the trained single-modal visual perception model, and a preliminary detection result of cultivated land is output, which includes a probability map of cultivated land and boundary information of the field.

[0012] In an optional embodiment, the preset multi-modal logic reasoning module includes a spatial relationship reasoning unit, a time series reasoning unit, and a domain knowledge embedding unit; The spatial relationship reasoning unit is used to construct a graph network with field nodes and facility nodes as the core, calculate edge features between nodes, and the edge features include distance from road, angle with irrigation canal, and neighbor cultivated land proportion; the influence mechanism of spatial relationship on cultivated land probability is learned by means of graph convolution network, and the result is post-processed by a rule engine; The time series reasoning unit is used to construct a time series feature vector, which includes NDVI mean, standard deviation, trend slope, and fallow frequency; the time series feature vector is analyzed in depth by a Transformer model to obtain a time series consistency score of cultivated land, and the preliminary detection result is corrected according to the time series consistency score; The domain knowledge embedding unit is used to construct a knowledge graph containing field entities, policy document entities, rotation period entities, and their relationships, and a rule engine is used to forcibly correct and label the preliminary detection result according to the preset rules.

[0013] In an optional embodiment, the visualization data is in a GeoJSON format including but not limited to a farmland probability, a basic farmland number, and the like.

[0014] In a second aspect, the embodiments of the present application further provide a farmland recognition system based on modal fusion, comprising: A data acquisition module is configured to acquire image data of a to-be-recognized farmland containing RGB and multi-spectral bands by a sensor carried by a UAV, and simultaneously acquire GIS vector data and historical time-series images of the to-be-recognized farmland to form a basic data set. A data preprocessing module is configured to correct, stitch and segment the image data in the basic data set, and perform a unified coordinate system processing on the GIS vector data in the basic data set and calculate a field block regularity. A farmland recognition module is configured to process the basic data set by using a preset single-modal visual perception model, output a preliminary detection result containing a probability map and boundary information, correct the preliminary detection result by using a preset multi-modal logical reasoning module in combination with auxiliary data, and generate a recognition result. A result output module is configured to perform a visualization and vectorization processing on the recognition result, generate visualization data and display the visualization data.

[0015] In a third aspect, the embodiments of the present application further provide an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the farmland recognition method based on modal fusion according to any one of the above embodiments when executing the program.

[0016] In a fourth aspect, the embodiments of the present application further provide a storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the steps of the farmland recognition method based on modal fusion according to any one of the above embodiments.

[0017] As can be seen from the above technical solutions, the present application has the following advantages: The application provides a cultivated land recognition method based on modal fusion. Image data containing RGB and multispectral bands is collected by a UAV, combined with GIS vector data and historical time series images to form a basic data set, realizing the fusion and utilization of multi-source data and providing a rich information base for cultivated land recognition. The image data is corrected, spliced and segmented, and the GIS vector data is processed in a unified coordinate system and the field block regularity is calculated, effectively eliminating data interference factors and improving data quality and consistency. The preset single-modal visual perception model is used to process the basic data set, outputting preliminary detection results containing probability maps and boundary information, combined with a multi-modal logical reasoning module composed of a spatial relationship reasoning unit, a time series reasoning unit and a domain knowledge embedding unit, which can comprehensively consider spatial relationships, time series characteristics and domain knowledge to modify the preliminary detection results, effectively improving the accuracy and robustness of cultivated land recognition. The recognition results are finally visualized and vectorized and displayed, facilitating practical application and decision-making reference. The method significantly improves the precision and reliability of cultivated land recognition through multi-modal fusion and multi-dimensional processing, and has strong practicality and promotional value.

[0018] The application collects RGB and multispectral image data by a UAV, combines GIS vector data and historical time series images, and constructs a multi-dimensional basic data set containing spectral characteristics, spatial location, time series and domain knowledge, providing rich and complementary information support for cultivated land recognition.

[0019] The application eliminates noise interference and extracts core features by radiometric calibration, atmospheric correction, ortho-mosaic and superpixel segmentation of image data; unifies the coordinate system of GIS vector data and calculates the field block regularity (quantifies the geometric regularity of field blocks based on the convex hull algorithm), improving data consistency and spatial feature analyzability, laying a high-quality data foundation for subsequent model processing.

[0020] The application uses a single-modal visual perception model combined with a combination loss function and optimization strategy to output high-precision cultivated land probability maps and boundary information; uses a multi-modal logical reasoning module to correct the misjudgment of single-modal results through spatial relationship reasoning, time series reasoning and domain knowledge embedding, significantly improving the accuracy and anti-interference ability of cultivated land recognition in complex scenes.

[0021] The application calculates the field block regularity by the Shapely library, converts the geometric morphological characteristics of the field block into quantifiable indicators, assists in distinguishing the spatial structure differences between cultivated land and non-cultivated land, provides key spatial rule constraints for multi-modal reasoning, and avoids misjudgment caused by relying only on spectral characteristics.

[0022] The application converts the final recognition result into GeoJSON format data containing cultivated land probability, basic farmland number and other attributes, realizes spatial visualization and structured expression of cultivated land information, supports seamless docking with existing geographic information systems, provides intuitive and efficient decision support tools for cultivated land monitoring, resource management, policy making and the like, and significantly improves the practicability and landing value of the method. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the present application, the drawings needed to be used in the description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0024] Figure 1 The flowchart of the cultivated land recognition method based on modal fusion provided by the present application.

[0025] Figure 2 The structural diagram of the cultivated land recognition system based on modal fusion provided by the present application.

[0026] Figure 3 The structural diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0027] In the following detailed description of the specific steps of the cultivated land recognition method based on modal fusion, various embodiments of the present disclosure will be described more fully. The present disclosure can have various embodiments, and adjustments and changes can be made therein. However, it should be understood that there is no intention to limit various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives falling within the spirit and scope of various embodiments of the present disclosure.

[0028] In the following, the term "include" or "may include" used in various embodiments of the present disclosure indicates the presence of the disclosed functions, operations or elements, and does not limit the addition of one or more functions, operations or elements. In addition, as used in various embodiments of the present disclosure, the terms "include", "have" and their synonyms only mean to indicate a specific feature, number, step, operation, element, component or combination of the foregoing, and should not be understood as first excluding the presence or addition of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing or the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing.

[0029] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0030] Please refer to Figure 1 As shown in the method flow chart of a farmland recognition method based on modal fusion in an embodiment, the method comprises: S1: Collect image data of the to-be-recognized farmland containing RGB and multi-spectral bands by a UAV carrying a sensor, and simultaneously acquire GIS vector data and historical time-series images of the to-be-recognized farmland to form a basic data set.

[0031] In the specific embodiment, a UAV carrying a visible light sensor and a multi-spectral sensor is used to acquire image data containing RGB three-channel images and near-infrared, red edge and other bands according to the set flight parameters, so as to extract the color, texture features and vegetation index for calculation of the farmland, respectively.

[0032] At the same time, the GIS vector data of the land use status disclosed by the natural resources agency, the irrigation facility account of the agricultural agency, the road network of the surveying and mapping unit and the historical time-series images supplemented by Sentinel 2 L2A data are acquired, and data alignment and rule modeling processing are performed.

[0033] Finally, the basic data set is constructed and generated.

[0034] For example, a UAV carrying a visible light sensor (capturing RGB three-channel images) and a multi-spectral sensor (such as RedEdge-P, acquiring near-infrared, red edge bands) is used to collect image data according to the parameters of a flight height of 100-300 meters (balancing resolution and efficiency), an 80% heading overlap rate and a 65% lateral overlap rate, which is used to extract color, texture features and calculate vegetation index (such as NDVI, EVI). The SHP format land use status data acquired from the natural resources bureau, the irrigation facility account of the agricultural and rural bureau, the road network data (containing WGS84 coordinates and road grade attributes) of the surveying and mapping unit are used as GIS vector data; Sentinel 2 L2A satellite images (10-meter resolution, 13 bands) of the same period in the past three years are downloaded in batches through GoogleEarthEngine (GEE) to supplement the missing historical time-series data of the UAV, the satellite and the UAV images are unified to the UTMZone50N coordinate system through the same point matching, and the related policy files are parsed into structured data and stored in MySQL, thereby constructing a basic data set containing single-modal images and multi-modal auxiliary data.

[0035] S2: Correct, splice and segment the image data in the basic data set, and process the GIS vector data in the basic data set in a unified coordinate system and calculate the regularity of the field.

[0036] In the specific embodiment, first, the image data is preprocessed: radiation calibration and atmospheric correction, ortho correction and splicing, super pixel segmentation and other operations are sequentially performed, interference factors in the image data are eliminated, core features are extracted, and the image is converted into a feature data form suitable for cultivated land recognition.

[0037] Then, the GIS data is preprocessed: the coordinate system of the GIS vector data is unified, the regularity of the field is calculated, the spatial data has computability and reasoning ability, and accurate spatial information support is provided for subsequent cultivated land recognition.

[0038] Among them, the specific process of unifying the coordinate system of GIS vector data includes: Unifying the coordinate system of GIS vector data 1. Analyze the source data coordinate system: use GIS software (such as ArcGIS, QGIS) or programming library (such as GDAL, PyProj) to read the metadata of imported GIS vector data, determine the currently used coordinate system, and there may be many different coordinate systems such as WGS84, Beijing 54, Xi'an 80.

[0039] 2. Determine the target coordinate system: according to the overall requirements of the project and the spatial reference consistency requirements with other data such as unmanned aerial image, determine the target coordinate system after unified conversion, for example, a specific zone (such as UTMZone50N) in the UTM projection coordinate system.

[0040] 3. Perform coordinate conversion operation: If using GIS software, for example, QGIS, the source vector data layer can be loaded, and through the "save as" function, the corresponding coordinate reference system (CRS) is selected in the pop-up dialog box, and the target coordinate system is specified. The software will automatically perform coordinate transformation on the data based on the internal projection conversion algorithm, and save the converted data as a new vector file (such as shp format).

[0041] 4. If using GDAL library, use gdalwarp command in command line, such as gdalwarp -t_srs EPSG:32650 input.shp output_utm.shp, where -t_srs parameter is followed by EPSG code of target coordinate system (here, EPSG:32650 corresponding to UTMZone50N is taken as an example), and the input and output file path and name are specified. The coordinate conversion operation of batch vector data can be realized.

[0042] The calculation process of the field block regularity includes: Read the vector data geometry information: Use a programming language such as Python with Shapely, Fiona, and other libraries to load the converted field block vector data, extract the geometry vertex coordinate information of each field block polygon, and construct the corresponding ShapelyPolygon object for subsequent regularity calculation based on geometry operations.

[0043] Calculate the convex hull of the field block polygon: Call the convex_hull method provided by Shapely to calculate the convex hull of each field block Polygon object. The convex hull is the smallest convex polygon that contains all the vertices of the field block, which can approximately reflect the shape of the field block from the shape, but is more regular and smooth than the original field block polygon.

[0044] Calculate the area and regularity value: Get the area of the original field block polygon (through the area attribute of the Polygon object) and the area of the previously calculated convex hull polygon.

[0045] According to the regularity formula: regularity = field block actual area / convex hull area, calculate the regularity value of each field block. In Python, you can achieve this by simple division operation, but you need to pay attention to the special case of convex hull area may be zero (can be avoided by setting a small threshold to avoid division by zero error), for example: regularity = geometry.area / max(hull.area,1e-8), where geometry is the original field block polygon object, and hull is the corresponding convex hull polygon object.

[0046] Store the regularity result: Add the calculated regularity value of each field block as an attribute field to the corresponding vector data layer, which is convenient for subsequent use as a feature in the cultivated land recognition model. You can update the attribute table in GIS software or use libraries such as Fiona in Python to read and write the attribute structure of the vector file to write the regularity result to the corresponding field.

[0047] For example, pre-processing of the image data: first, apply the FLAASH algorithm through the ENVI or Py6S library, input the sensor metadata to complete the radiometric calibration and atmospheric correction, and convert the DN value to the true reflectance of the surface; then, use Pix4Dmapper combined with POS data and ground control points (GCP) for orthorectification to eliminate geometric distortion, realize image stitching through SIFT feature matching and Poisson fusion algorithm, and generate seamless DOM; finally, use the SLIC algorithm for superpixel segmentation, set n_segments=2000 and compactness=10, and segment the image into pixel clusters with similar color texture.

[0048] Then, for the GIS vector data, use the gdalwarp tool of the GDAL library to convert the WGS84 coordinate system data to the UTMZone50N projection coordinate system (such as the command: gdalwarp -t_srs EPSG:32650 input.shp output_utm.shp), extract the field block polygon geometric vertices based on the Shapely library, calculate the regularity of the field block (regularity = field block area / convex hull area, generate the convex hull through the ConvexHull function, and the formula is regularity = geometry.area / max(hull.area, 1e-8)), and store the result as an attribute field in the vector data.

[0049] S3: processing the basic data set using a preset single-modal visual perception model, outputting a preliminary detection result containing a probability map and boundary information, and using a preset multi-modal logical reasoning module to correct the preliminary detection result and generate a recognition result.

[0050] In the specific implementation, first, a single-modal visual perception model constructed by a trained U-Net++ or YOLOv8n-seg model is used to process the pre-processed image data, and a preliminary detection result of cultivated land is output, which includes a probability map and field block boundary information of the cultivated land.

[0051] Then, the multi-modal logical reasoning module is used to check and correct the single-modal detection result. Specifically, a multi-modal logical reasoning module is generated through the construction of spatial relationship reasoning, time series reasoning, and domain knowledge embedding units, and based on the multi-modal logical reasoning module, the single-modal detection result is analyzed and logically reasoned in depth in combination with multi-modal auxiliary data, the initial detection result is corrected, and finally a high-precision and interpretable cultivated land recognition result is obtained.

[0052] The training process of the single-modal visual perception model includes: Select U-Net++ or YOLOv8n-seg as the model architecture; determine the loss function, which can use a combination of cross-entropy and Dice loss, and set the class weight reasonably according to the sample class imbalance; configure the training parameters in detail, such as selecting the AdamW optimizer, setting the initial learning rate, batch size, training period, etc., and using the data enhancement pipeline to enhance the training data to improve the generalization ability of the model and the accuracy of cultivated land recognition.

[0053] Among them, the multi-modal logical reasoning module includes: The spatial relationship reasoning unit is used to construct a graph network with field block nodes and facility nodes as the core, calculate the edge features between nodes, and cover the distance from the road, the angle with the irrigation canal, the neighborhood cultivated land proportion, etc. The influence mechanism of spatial relationship on cultivated land probability is learned by means of graph convolution network (GCN) and other models, and the results are post-processed by the rule engine, such as appropriately increasing the cultivated land probability for areas with high rule degree and close to the road.

[0054] The time series reasoning unit is used to construct a time series feature vector, and the vector elements include NDVI mean, standard deviation, trend slope, fallow frequency, etc. The time series features are analyzed in depth by using Transformer and other models to obtain the time series consistency score of cultivated land, and the cultivated land recognition result is corrected accordingly. At the same time, for abnormal situations, auxiliary means such as SAR data are used for judgment.

[0055] The domain knowledge embedding unit is used to construct a knowledge graph containing entities such as field blocks, policy documents, and crop rotation periods and their relationships. The rule engine is used to modify and accurately label the cultivated land recognition results according to the pre-set rules, such as increasing the probability of low-probability results in basic farmland and performing automatic labeling operations in fallow areas.

[0056] For example, U-Net++ or YOLOv8n-seg is used as a single-modal visual perception model, the combination of cross-entropy and Dice loss is used as the objective function (the cultivated land class weight is set to 3 to deal with sample imbalance), Albumentations library is used for random rotation, Gaussian blur and other data enhancement, AdamW optimizer (initial learning rate 1e-4, cosine annealing decay) is used to train for 300 epochs, and a lightweight model is deployed on JetsonNano (parameter compression 40%, inference speed 15FPS). The model outputs a probability map for the preprocessed image, which is binarized by threshold 0.5, denoised by morphological closing operation, and the contour is extracted. Through non-maximum suppression (NMS, IoU threshold 0.3), redundant boundaries are filtered out, and the preliminary detection result is generated.

[0057] In the multi-modal logic reasoning module, the spatial relationship reasoning unit constructs a graph network based on PyG, the field node attributes include NDVI and regularity, the facility node (road, irrigation canal) attributes include type and length, the edge features calculate the distance from the road and the proportion of adjacent cultivated land, etc., learn the influence of spatial relationship on cultivated land probability through the GCN model (loss function MSE), and correct the results combined with the rule engine (such as the probability +0.2 when the regularity>0.6 and the distance from the road<800 meters); the time series reasoning unit extracts features such as NDVI mean and standard deviation in the past 3 years, inputs the Transformer model (d_model=64, 2 layers of Encoder) after StandardScaler standardization, outputs the time series consistency score (1-score as probability), triggers Sentinel-1 SAR data backscattering coefficient analysis (roughness>15 is judged as fallow cultivated land) for abnormal areas with score<0.3; the domain knowledge embedding unit constructs a knowledge graph based on Neo4j, executes rules such as "low probability results in basic farmland are forced to be improved" and "fallow area automatic labeling" through the Drools rule engine (such as rule weight 0.8), corrects the single-modal results combined with the rotation rule state transition matrix (such as corn->wheat probability 0.7), and finally generates high-precision recognition results through decision-level weighted fusion (weight [0.7, 0.2, 0.1], dynamic adjustment).

[0058] S4: The recognition results are visualized and vectorized to generate visual data and display.

[0059] In the specific embodiment, the final cultivated land recognition results are visualized and vectorized to intuitively display the cultivated land distribution and related attribute information, and the output formats include but are not limited to GeoJSON format data with cultivated land probability, basic farmland number and other attributes, which provides accurate and intuitive basis for subsequent cultivated land management and decision-making.

[0060] For example, using cv2 to render the probability map into pseudo-color (blue for low probability and red for high probability), and superimpose the field block boundary polygon to generate a visual image; convert the contour coordinates to GeoJSON format (including cultivated land probability, basic farmland number and other attributes).

[0061] In this embodiment, by constructing a multi-source heterogeneous data-driven intelligent recognition framework, high-precision analysis and dynamic monitoring of cultivated land spatial distribution are realized. In the data collection stage, the innovation integrates unmanned aerial vehicle multi-modal image and authoritative department spatio-temporal data, which not only obtains surface texture, vegetation index and other detailed features through RGB and multispectral sensors, but also integrates structured vector information such as land use status and irrigation facility distribution, and introduces Sentinel satellite time series image to construct a spatio-temporal continuous baseline, forming a three-dimensional data base covering spectrum, space and time. In the preprocessing stage, radiation correction, orthorectification and superpixel segmentation techniques are used to eliminate image distortion, and the regularity of the field block is quantified through coordinate system unification and convex hull analysis to establish an explicit association between geometric features and cultivated land attributes. The core recognition process adopts a two-stage reasoning mechanism: first, an improved U-Net++ deep learning model is used to extract cultivated land probability map and boundary information, then a multi-modal logical reasoning module is introduced, which analyzes the spatial dependency relationship between the field block and the road, irrigation facilities through the spatial relationship graph network, uses the time series Transformer to capture the vegetation index change rule, and embeds the cultivated land protection policy knowledge graph for rule constraint, realizing the organic integration of data-driven and domain knowledge. The final output includes probability distribution, attribute labeling and vectorized results, which can be directly connected to the national spatial planning system, significantly improving the automation level and decision-making scientificity of cultivated land monitoring, effectively addressing the recognition difficulties in complex scenarios such as cultivated land fragmentation and planting structure changes, and providing a full-process intelligent solution for cultivated land protection.

[0062] As Figure 2 shown below is an embodiment of a cultivated land recognition system based on modal fusion provided by the embodiments of the present disclosure. The system and the above-mentioned embodiments of the cultivated land recognition method based on modal fusion belong to the same inventive concept. Details not described in the embodiment of the cultivated land recognition system based on modal fusion can be referred to the above-mentioned embodiments of the cultivated land recognition method based on modal fusion.

[0063] A cultivated land recognition system based on modal fusion, comprising: a data collection module, a data preprocessing module, a cultivated land recognition module and a result output module.

[0064] The data collection module is configured to collect image data of the to-be-identified cultivated land containing RGB and multispectral bands by an unmanned aerial vehicle carrying a sensor, and simultaneously collect GIS vector data and historical time series images of the to-be-identified cultivated land to form a basic data set.

[0065] The data preprocessing module is configured to correct, splice and segment the image data in the basic data set, and perform unified coordinate system processing on the GIS vector data in the basic data set and calculate the regularity of the field block.

[0066] The cultivated land recognition module is configured to process the basic data set by using a preset single-modal visual perception model, output a preliminary detection result containing a probability map and boundary information, and correct the preliminary detection result by using a preset multi-modal logical reasoning module in combination with auxiliary data to generate a recognition result.

[0067] The result output module is configured to visualize and vectorize the final recognition result, generate visualized data, and display the visualized data.

[0068] The cultivated land recognition system based on modal fusion provided by the embodiment integrates images of unmanned aerial vehicles, GIS vector data, and historical time-series images to construct a multi-dimensional data set containing spectral features, spatial positions, time sequences, and domain knowledge. In combination with a single-modal visual perception model and a multi-modal logical reasoning module, spatial relationship reasoning, time-series dynamic analysis, and embedding of domain knowledge are cooperatively optimized, thereby significantly improving the accuracy, compliance, and adaptability to complex scenarios of cultivated land recognition, and providing technical support for digital management of agricultural resources.

[0069] Figure 3 A hardware structure schematic diagram of an electronic device for implementing various embodiments of the present application.

[0070] The cultivated land recognition method based on modal fusion provided by the embodiment of the present application can be applied to an electronic device. Those skilled in the art can understand that the electronic device structure involved in the embodiment of the present application does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the diagram, or combine certain components, or different component arrangements. In the embodiment of the present application, the electronic device includes but is not limited to a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described herein and / or claimed.

[0071] The electronic device can include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charge management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, a key, a camera, a display screen, and a SIM card interface, etc.

[0072] The processor can include one or more processing units, such as: the processor can include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video code, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0073] The processor can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0074] The processor can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory can save instructions or data that the processor has just used or repeatedly uses. If the processor needs to use the instruction or data again, it can be directly called from the memory. Avoiding repeated access reduces the waiting time of the processor, thus improving the efficiency of the system.

[0075] The external memory interface can be used to connect an external storage card, such as a MicroSD card, to realize the expansion of the storage capacity of the electronic device. The external storage card communicates with the processor through the external memory interface to realize the data storage function. For example, files such as music and video are saved in the external storage card.

[0076] The internal memory can be used to store computer executable program codes, which include instructions. The processor executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory. The internal memory can include a program storage area and a data storage area. The internal memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0077] The wireless communication function of the electronic device can be realized through an antenna, a wireless communication module, a modem processor, and a baseband processor, etc.

[0078] The wireless communication module can provide solutions for wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), and the like.

[0079] The electronic device can realize audio functions and the like through an audio module, a speaker, a receiver, a microphone, a headset interface, an application processor, and the like.

[0080] The electronic device can realize a shooting function through an ISP, a camera, a video codec, a GPU, a display screen, an application processor, and the like.

[0081] The electronic device can realize a display function through a GPU, a display screen, an application processor, and the like.

[0082] The GPU is a microprocessor for image processing, connected to a display screen and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor can include one or more GPUs that execute program instructions to generate or change display information.

[0083] The display screen is used to display images, videos, and the like. The display screen includes a display panel.

[0084] The electronic device realizes the method for identifying cultivated land based on modal fusion provided in the present application. The method includes the following steps: collecting RGB and multispectral images by a plurality of sensors of a UAV, and fusing GIS vector data and historical time-series images to construct a basic data set; performing data preprocessing by combining image correction, image splicing, image segmentation, GIS data coordinate unification, and field block regularity calculation; outputting a preliminary detection result by using a single-modal visual perception model, and correcting the preliminary detection result by using a multi-modal logical reasoning module containing spatial / time reasoning and embedded domain knowledge; and finally visualizing and vectorizing the result into GeoJSON format data, so as to obtain the beneficial effects of obtaining multi-dimensional complementary information, improving data quality and feature availability, comprehensively constraining space / time and domain rules to improve the cultivated land recognition precision and robustness, quantifying field block geometric features, and realizing structured information output to support efficient decision-making applications.

[0085] In the storage medium provided in the present application, a program product capable of realizing the method for identifying cultivated land based on modal fusion is stored.

[0086] The cultivated land recognition method based on modal fusion comprises: The cultivated land recognition method based on modal fusion comprises: The image data of the cultivated land to be recognized containing RGB and multispectral bands is collected by a sensor carried by a UAV, and GIS vector data and historical time-series images of the cultivated land to be recognized are acquired to form a basic data set; The image data in the basic data set is corrected, spliced and segmented, the GIS vector data in the basic data set is processed in a unified coordinate system, and the regularity of the field is calculated; The basic data set is processed by using a preset single-modal visual perception model to output a preliminary detection result containing a probability graph and boundary information, the preliminary detection result is corrected by using a preset multi-modal logical reasoning module in combination with auxiliary data to generate a recognition result; The final recognition result is visualized and vectorized to generate visualized data and is displayed.

[0087] In some possible implementation manners, the cultivated land recognition method based on modal fusion of the present disclosure can be implemented in the form of a program product, which includes program codes for causing a terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of the present specification when the program product is run on the terminal device.

[0088] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0089] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for cultivated land recognition based on modal fusion, characterized in that, The application relates to a farmland recognition method based on unmanned aerial vehicle (UAV) and a system thereof. The method comprises the following steps: collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; correcting, splicing and segmenting the image data in the basic data set, uniformly processing the GIS vector data in the basic data set in a coordinate system and calculating field block regularity; processing the basic data set by using a preset single-modal visual perception model to output a preliminary detection result containing a probability graph and boundary information, correcting the preliminary detection result by using a preset multi-modal logical reasoning module in combination with auxiliary data, and generating a recognition result; and performing visual and vector processing on the recognition result to generate visual data and display the visual data. The method comprises the following steps: collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be-recognized farmland, and forming a basic data set; collecting image data of to-be-recognized farmland containing RGB and multi-spectral bands by an UAV-mounted sensor, simultaneously acquiring GIS vector data and historical time-series images of the to-be ​ ​ 2. The modal fusion based cultivated land recognition method according to claim 1, characterized in that, ​ ​ ​ ​ 3. The modal fusion based cultivated land recognition method according to claim 2, characterized in that, ​ ​ 4. The modal fusion based cultivated land recognition method according to claim 3, characterized in that, ​ ​ ​ ​ ​ The regularity value corresponding to each field is calculated by the formula: regularity = actual area of the field / convex hull area; The calculated regularity value of each field is added to the corresponding vector data layer as an attribute field.

5. The modal fusion based cultivated land recognition method according to claim 4, characterized in that, The single-modal visual perception model is used to process the basic data set, and a preliminary detection result containing a probability map and boundary information is output, including: A trained U-Net++ or YOLOv8n-seg model is used to construct a single-modal visual perception model; The combined cross-entropy and Dice loss is used as the loss function, and the class weights are set reasonably according to the sample class imbalance; The AdamW optimizer is selected, and the initial learning rate, batch size, and training period are set for model training; The image data in the basic data set is input into the trained single-modal visual perception model, and a preliminary detection result of cultivated land is output, including the probability map of cultivated land and the boundary information of the field.

6. The modal fusion based cultivated land recognition method according to claim 5, characterized in that, The preset multi-modal logic reasoning module includes a spatial relationship reasoning unit, a time series reasoning unit, and a domain knowledge embedding unit; The spatial relationship reasoning unit is used to construct a graph network with field nodes and facility nodes as the core, calculate edge features between nodes, and the edge features include distance from road, angle with irrigation canal, and neighbor cultivated land proportion; the influence mechanism of spatial relationship on cultivated land probability is learned by means of graph convolution network, and the result is post-processed by rule engine; The time series reasoning unit is used to construct a time series feature vector, which includes NDVI mean, standard deviation, trend slope, and fallow frequency; the time series feature vector is analyzed in depth by using the Transformer model, and the time series consistency score of cultivated land is obtained, and the preliminary detection result is corrected according to the time series consistency score; The domain knowledge embedding unit is used to construct a knowledge graph containing field entities, policy document entities, and rotation period entities and their relationships, and the rule engine is used to forcibly correct and label the preliminary detection result according to the preset rules.

7. The modal fusion based cultivated land recognition method according to claim 6, characterized in that, The visualization data includes but is not limited to GeoJSON format data with cultivated land probability and basic farmland number.

8. A cultivated land recognition system based on modal fusion, characterized by, The system adopts the cultivated land recognition method based on modal fusion according to any one of claims 1 to 7; The system comprises: The data acquisition module is used to collect image data of the to-be-identified cultivated land containing RGB and multi-spectral bands by a UAV carrying sensors, and to obtain GIS vector data and historical time series images of the to-be-identified cultivated land, thereby forming a basic data set; The data preprocessing module is used to correct, stitch, and segment the image data in the basic data set, and to perform unified coordinate system processing on the GIS vector data in the basic data set and calculate the regularity of the field; The cultivated land recognition module is used to process the basic data set by using a preset single-modal visual perception model, output a preliminary detection result containing a probability map and boundary information, and correct the preliminary detection result by using a preset multi-modal logic reasoning module combined with auxiliary data to generate a recognition result; The result output module is used to visualize and vectorize the recognition result, generate visualization data, and display the visualization data.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method for identifying cultivated land based on modal fusion according to any one of claims 1 to 7 when executing the program.

10. A storage medium having stored thereon a computer program, characterized in that The computer program implements the steps of the method for identifying cultivated land based on modal fusion according to any one of claims 1 to 7 when executed by the processor.

Images