A farmland autumn harvest progress monitoring method and system based on image feature extraction

Abstract

The application belongs to the technical field of farmland remote sensing monitoring, and discloses a farmland autumn harvest progress monitoring method and system based on image feature extraction, which comprises the following steps: automatically acquiring farmland real scene images and preprocessing by calling a meteorological data platform interface at a fixed time; performing multi-modal feature decoupling extraction on the images to generate spectral, texture and edge morphological feature maps respectively, and generating an enhanced feature map by dynamically weighted fusion according to a real-time environmental state; inputting the enhanced feature map into a first multi-modal large model to generate a farmland state text description, and then outputting a structured crop type and a harvesting state by a second inference large model; jointly checking through time series mutation detection and GIS adjacent land spatial synchronicity verification to distinguish environmental interference from real harvesting behavior; and incrementally storing and visually displaying the corrected result. The application solves the problem that the recognition result is easily affected by interference such as light in a complex field environment and frequently jumps, and realizes full-process automatic and stable monitoring of the autumn harvest progress.

Description

Technical Field

[0001] This invention relates to the field of farmland remote sensing monitoring technology, and in particular to a method and system for monitoring the progress of autumn harvest in farmland based on image feature extraction. Background Technology

[0002] The development of smart agriculture has created an urgent need for real-time, automated, and precise monitoring of autumn harvest progress. Harvest progress is crucial foundational data for agricultural production management, grain yield estimation, disaster assessment, and agricultural policy formulation. Traditional monitoring methods primarily rely on manual reporting and sampling statistics, which suffer from drawbacks such as poor timeliness, limited coverage, strong subjectivity, and low data consistency. In recent years, with the widespread deployment of farmland real-scene image acquisition equipment and the operationalization of meteorological data platforms, image-based automated crop monitoring has become a development trend, providing a new technological path for objective and real-time monitoring of autumn harvest progress.

[0003] Existing technologies either rely on threshold discrimination based on single image features or simply stitch together multiple features with fixed weights. However, in complex field environments, the recognition results are easily affected by environmental factors such as changes in lighting, cloud cover, crop stubble, and ground shadows. This leads to frequent changes in monitoring data and insufficient stability, making it impossible to effectively distinguish between "real harvesting behavior" and "instantaneous environmental interference." As a result, business-level anomalies appear in the output results, such as "harvested yesterday, but reverted to unharvested today," which severely restricts the usability of automated monitoring systems in actual agricultural management scenarios. Summary of the Invention

[0004] The present invention aims to provide a method and system for monitoring the progress of autumn harvest in farmland based on image feature extraction, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for monitoring the progress of autumn harvest in farmland based on image feature extraction includes the following steps:

[0007] S1. Automatically call the meteorological data platform interface by triggering a task at a time to obtain the real-scene image metadata of each farmland station in the area to be monitored, remove duplicates according to the unique identifier of the station, retain the latest real-scene image of the farmland taken by each station and download it to the local machine, and perform standardized preprocessing on the farmland real-scene image to obtain a standardized farmland image.

[0008] S2. Perform multimodal feature decoupling extraction on the standardized farmland image, calculate and generate spectral feature map, texture feature map and edge morphology feature map respectively; and dynamically determine the fusion weight of each feature according to the real-time environmental state, perform weighted fusion on the obtained independent feature maps to generate an enhanced feature map;

[0009] S3. Input the enhanced feature map into the first multimodal large model and execute the preset first instruction to make the first multimodal large model generate text information describing the state of farmland based on the enhanced feature map;

[0010] S4. Input the text information into the second inference model and execute the preset second instruction, so that the second inference model outputs the identification results of crop type and harvest status according to the text information and the preset structured format; extract the values ​​of crop name and harvest status fields from the identification results through regular expressions as the identification results of crop type and harvest status;

[0011] S5. Obtain the harvest status identification results of the same site for multiple consecutive days, construct a time series sequence, calculate the amount of harvest status change between adjacent dates, and determine the time series mutation when the amount of harvest status change exceeds a preset threshold; for the site determined to be a time series mutation, query the amount of harvest status change of its GIS adjacent plots in the same time period. If the adjacent plots have not changed synchronously in the same direction, then correct the current identification result to the state before the mutation; otherwise, confirm it as a real harvesting behavior.

[0012] S6. The harvest status after verification and correction is stored incrementally in a structured data table created by date, using the site's unique identifier as the key, to ensure that only one record is retained for the same site on the same day, and then output to the visualization platform for display.

[0013] Preferably, step S2 specifically includes:

[0014] Three parallel feature extraction branches are set up: the spectral feature branch calculates the normalized vegetation index and soil-regulated vegetation index to generate a spectral feature map; the texture feature branch extracts texture parameters using gray-level co-occurrence matrix and local binary mode to generate a texture feature map; and the edge morphology branch extracts field contours and crop row features using edge detection and Hough transform to generate an edge morphology feature map.

[0015] The standardized farmland image is divided into a preset grid, and the brightness, cloud shadow ratio and stubble density of each grid are calculated. Global environmental parameters are statistically analyzed.

[0016] The current environment type is determined based on the preset decision tree, including normal lighting, strong light, cloudy and shadowy, and dense stubble environment;

[0017] Based on the preset mapping relationship between different environment types and feature weights, the fusion weights of spectral features, texture features, and edge morphology features are dynamically adjusted, and pixel-level weighted fusion is performed on each independent feature map to generate the enhanced feature map.

[0018] Preferably, the first instruction in step S3 includes preset agricultural image description rules, which guide the first multimodal large model to describe the crop morphology, ground condition and harvesting marks in the image in a realistic style, and exclude exaggeration.

[0019] Preferably, the second instruction in step S4 includes a preset structured output format rule, which is used to force the second inference model to output the recognition result according to the fixed template of "crop name: [value]; harvest status: [harvested / not harvested]".

[0020] Preferably, the timed triggering task in step S1 is a task that is automatically triggered at a preset time every day; after the timed triggering task is triggered, the meteorological data platform interface is called to obtain the real-scene image metadata of all farmland stations from the previous day.

[0021] Preferably, the incremental storage in step S6 specifically includes:

[0022] Extract the shooting date from the filenames of farmland scene images using regular expressions, and archive the image files according to the shooting date;

[0023] A structured data table is created for each shooting date, with the site's unique identifier serving as the unique key within the structured data table;

[0024] Perform insert or update operations on the identification results of the day: if the unique key does not exist in the table, add a new record; if it already exists, overwrite the original record to ensure that only one latest record is retained for each site each day.

[0025] Preferably, the visualization platform in step S6 is built on WebGIS, renders the harvest status of the site with different colors, and supports switching by date, linked table display, and real-scene image pop-up preview.

[0026] A farmland autumn harvest progress monitoring system based on image feature extraction includes:

[0027] The data acquisition layer is used to automatically call the meteorological data platform interface through timed triggering tasks to obtain real-scene image metadata of each farmland station in the area to be monitored, deduplicate by the unique identifier of the station, retain the latest real-scene image of the farmland taken by each station and download it to the local machine;

[0028] The image preprocessing and feature enhancement layer is used to perform standardized preprocessing and multimodal feature decoupling extraction on farmland scene images, respectively calculating and generating spectral feature maps, texture feature maps, and edge morphology feature maps; and dynamically determining the fusion weight of each feature based on the real-time environmental state, performing weighted fusion on the obtained independent feature maps to generate enhanced feature maps.

[0029] The dual-model collaborative recognition layer includes:

[0030] The first recognition module is used to call the first multimodal large model and execute the preset first instruction to generate text information describing the state of farmland based on the enhanced feature map;

[0031] The second recognition module is used to call the second inference model and execute the preset second instruction, so that it outputs the recognition result according to the preset structured format based on the text information; and extracts the values ​​of the crop name and harvest status fields from the recognition result through regular expressions as the recognition result of crop type and harvest status.

[0032] The spatiotemporal joint verification layer is used to obtain the harvest status identification results of the same site for multiple consecutive days, construct a time series, calculate the amount of change in harvest status between adjacent dates, and determine the time series mutation when the amount of change exceeds a preset threshold. For the site determined to be a time series mutation, query the amount of change in harvest status of its GIS adjacent plots in the same time period. If the adjacent plots have not changed synchronously in the same direction, the current identification result is corrected to the state before the mutation; otherwise, it is confirmed as a real harvesting behavior.

[0033] The incremental storage and visualization layer is used to incrementally store the verified and corrected harvest status, using the site's unique identifier as the key, and creating a structured data table by date. This ensures that only one record is retained for the same site on the same day, and the data is then output to the WebGIS platform for visualization.

[0034] Preferably, the image preprocessing and feature enhancement layer specifically includes:

[0035] The feature decoupling extraction module is configured with three parallel feature extraction branches, which are used to calculate vegetation index to generate spectral feature map, extract texture parameters to generate texture feature map, and extract edge morphology features to generate edge morphology feature map, respectively.

[0036] The environmental parameter calculation module is used to divide the standardized farmland image into a preset grid, and to calculate the brightness, cloud shadow ratio and stubble density of each grid to obtain environmental parameters, including global brightness, cloud shadow area ratio and stubble area ratio.

[0037] The environment type determination module is used to determine the current environment type based on a preset decision tree and environment parameters;

[0038] The dynamic weight fusion module is used to dynamically determine the fusion weight of each feature according to the preset mapping relationship between environment type and feature weight, and to perform weighted fusion on each independent feature map to generate the enhanced feature map.

[0039] Preferably, the spatiotemporal joint verification layer specifically includes:

[0040] The time-series sequence construction module is used to organize the identification results of the same site over multiple consecutive days in chronological order;

[0041] The mutation detection module is used to calculate the amount of change in harvest status between adjacent days and to detect temporal mutations based on a preset threshold.

[0042] The spatial adjacency verification module is used to query the adjacency relationship of GIS plots, obtain the change in harvest status of adjacent plots within the same time period, and verify whether the change is synchronous and in the same direction with the current site.

[0043] The state correction module is used to distinguish between environmental interference and actual harvesting behavior based on mutation detection results and spatial verification results, and to correct the identification results.

[0044] The beneficial effects of this technical solution compared to existing technologies are as follows:

[0045] (1) This technical solution uses multimodal feature decoupling extraction and environmental adaptive dynamic weight fusion to dynamically adjust the fusion weight of spectral, texture and edge morphology features according to the real-time environmental status (light intensity, cloud shadow ratio, stubble density), which fundamentally reduces the interference of light changes, cloud cover, crop stubble and ground shadow on the recognition results; at the same time, through the time-space joint verification mechanism, it uses the time sequence of multiple consecutive days to detect abrupt changes, and combines the spatial synchronization verification of adjacent plots in GIS, it can accurately distinguish between "real harvesting behavior" and "instantaneous environmental interference", automatically correct misjudgments caused by cloud shadow cover, etc., ensure that the monitoring data is stable and reliable, and eliminate the business-level anomaly of "harvested yesterday, but restored to unharvested today".

[0046] (2) By setting a timed trigger task, the meteorological data platform interface is automatically called to obtain real-time images of farmland. After standardized preprocessing, feature extraction, intelligent recognition, verification and correction, incremental storage and visualization are performed, forming a complete business loop. No manual intervention is required, and the autumn harvest progress data of the entire region is automatically updated every day.

[0047] (3) By adopting a dual-model collaborative reasoning architecture, the first multimodal big model generates realistic text descriptions based on enhanced feature maps, and the second reasoning big model outputs crop type and harvest status according to a preset structured format. Combined with the pre-processed image feature enhancement, the difficulty of recognizing complex farmland scenes such as variable light and dense crop stubble is reduced, so that stable and accurate recognition output can still be maintained in environments where a single model is prone to misjudgment. Attached Figure Description

[0048] Figure 1 This is a flowchart of the method of the present invention;

[0049] Figure 2 This is a system architecture diagram of the present invention; Detailed Implementation

[0050] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments:

[0051] like Figure 1 The method for monitoring the progress of autumn harvest in farmland based on image feature extraction, as shown, includes the following steps:

[0052] Step S1: Automated acquisition and preprocessing of farmland real-scene images

[0053] First, set the daily trigger time in the system configuration file, for example, 2:00 AM every day. The system uses the operating system's scheduled task to trigger the data collection program.

[0054] After the data acquisition program starts, it calls the application programming interface provided by the meteorological big data cloud platform via the HTTP protocol. The request parameters include: the time range is set to 00:00 to 24:00 of the previous day; the station type is limited to farmland real-scene monitoring station; the returned fields include station number, shooting time, latitude and longitude, and image file address.

[0055] The API returns a list of metadata in JSON format, which the system then parses to extract the image metadata for each site.

[0056] Since the same site may be photographed multiple times within a day, the system deduplicates images according to the following rules: The site's unique identifier (station number) is used as the grouping key; within each group, images are sorted in descending order of shooting time; only the record with the most recent shooting time is retained. After deduplication, the system downloads each image file one by one based on its address, saving them to the local storage directory while retaining the original naming rules. Simultaneously, a collection log is recorded, including the site, time, file path, and whether the collection was successful or failed.

[0057] The downloaded raw farmland scene images undergo standardization preprocessing, specifically the following steps are performed sequentially:

[0058] Standardized format: All images are converted to RGB color space and their resolution is uniformly scaled to 512×512 pixels to ensure consistency in subsequent processing.

[0059] Image denoising: A bilateral filtering algorithm is used. Bilateral filtering effectively preserves edge information while smoothing image noise, which is crucial for subsequent texture and edge feature extraction. The spatial domain parameter is set to 3, and the value domain parameter is set to 50.

[0060] Coordinate transformation: Raw latitude and longitude coordinates are usually stored in degrees, minutes, and seconds. The system automatically converts them to decimal degrees for easier WebGIS rendering. The conversion formula is: decimal degrees = degrees + minutes divided by 60 + seconds divided by 3600.

[0061] Invalid area removal: Based on GIS plot boundary data, non-farmland areas such as roads, ditches, and buildings in the image are marked as invalid pixels, and only valid farmland areas are retained for subsequent analysis.

[0062] After the above processing, a standardized farmland image is obtained, which serves as the input for subsequent steps.

[0063] Step S2: Multimodal feature decoupling extraction and dynamic weight fusion

[0064] S21: Feature Decoupling Extraction

[0065] Three completely independent parallel processing branches are set up, each branch does not interfere with the others, and the original semantic information of each feature is fully preserved. Through the parallel processing of the following three branches, the system obtains spectral feature maps, texture feature maps and edge morphology feature maps, which are used as inputs for subsequent dynamic weight fusion.

[0066] Spectral Feature Branch: Spectral features are primarily used to characterize the cover status of living crops in the field, serving as a key basis for distinguishing unharvested areas. This branch calculates the Normalized Difference Vegetation Index (NDI) and Soil-Regulated Vegetation Index (SDI) pixel-by-pixel based on the red and near-infrared band data of the image. The NDI is calculated as the difference between near-infrared and red band reflectance divided by their sum, with a value ranging from -1 to +1. Healthy, lush crops typically have a value greater than 0.6, while bare soil after harvesting usually has a value less than 0.2. The SDI incorporates a soil adjustment factor of 0.5 to reduce the influence of soil background. The system treats the NDI and SDI calculated for each pixel as independent channel values, combining them into a two-dimensional matrix of the same size as the original image, i.e., the spectral feature map, according to the original pixel arrangement. The value of each pixel in this feature map reflects the degree of living crop cover at that location.

[0067] Texture Feature Branch: Texture features are used to identify typical characteristics of semi-harvested conditions, such as crop stubble and crop residue. After crops are harvested, stubble and residue remain on the ground, significantly increasing the local texture complexity of the image. This branch uses a gray-level co-occurrence matrix (GLCM). A 15×15 pixel neighborhood window is taken centered on each pixel, and the GLCM is calculated in four directions: 0°, 45°, 90°, and 135°. The mean of these four directions is used as the texture parameter value for that pixel, including energy, contrast, entropy, and homogeneity. Simultaneously, a local binary mode is used, calculating the mean and variance of the texture spectrum in a circular neighborhood with a radius of 2 and 16 sampling points as auxiliary descriptors. The system uses the texture parameters calculated for each pixel as values ​​for independent channels, combining them according to the original pixel arrangement to form a two-dimensional matrix of the same size as the original image, i.e., the texture feature map. Among these, the entropy channel is most sensitive to texture complexity; the entropy value in stubble areas is higher than that in bare soil areas. Therefore, the entropy channel plays a crucial role in determining the semi-harvested state.

[0068] Edge morphology branch: Edge morphology features are used to extract field outlines and crop row structures, assisting in the accurate segmentation of farmland areas and the determination of harvest status. In unharvested farmland, crop rows are neat and continuous, while in harvested farmland, the row structure is disrupted. Specifically, the Canny edge detection algorithm is performed pixel-by-pixel on the standardized farmland image, with a low threshold of 80 and a high threshold of 160, resulting in a binary edge intensity map, where edge pixels are marked as 1 and non-edge pixels as 0. The edge detection results are then subjected to a Hough transform, and votes are accumulated in the parameter space to extract the crop row direction and spacing. The system uses the edge intensity value and row direction consistency value at each pixel location as channels, combining them according to the original pixel arrangement of the image to form a two-dimensional matrix of the same size as the original image, i.e., the edge morphology feature map. In this feature map, the edge intensity channel identifies the field outline and the position of the crop rows, while the row direction consistency channel reflects the regularity of the crop row arrangement within a local area.

[0069] S22: Dynamic Weight Fusion

[0070] Environmental parameter calculation: The standardized farmland image is uniformly divided into a preset number of grids, for example, 10 rows by 10 columns. For each grid region, a brightness parameter is calculated, which is the arithmetic mean of the pixel grayscale values ​​within the grid. A cloud shadow identification parameter is calculated, where a grid is marked as a cloud shadow grid when the proportion of pixels with grayscale values ​​below a preset shadow threshold exceeds 50%. A stubble retention identification parameter is calculated, where a grid is marked as a stubble retention grid when the texture entropy value within the grid exceeds a preset texture entropy threshold. Global environmental parameters are calculated, where global brightness is the average brightness of all grids, cloud shadow area proportion is the number of cloud shadow grids divided by the total number of grids, and stubble retention area proportion is the number of stubble retention grids divided by the total number of grids.

[0071] Environment type determination: Based on a preset decision tree. When the cloud shadow area accounts for more than 30%, it is determined to be a cloudy shadow environment; when the global brightness exceeds the preset brightness threshold and the cloud shadow area accounts for less than 30%, it is determined to be a strong light environment; when the stubble area accounts for more than 40%, it is determined to be a dense stubble environment; all other cases are determined to be normal lighting environments.

[0072] Dynamic weight configuration: The fusion weights are determined by looking up a table based on different environment types. Under normal lighting conditions, the spectral weight is 0.50, the texture weight is 0.30, and the edge weight is 0.20; under strong light conditions, the spectral weight is 0.30, the texture weight is 0.40, and the edge weight is 0.30; under cloudy or shadowy conditions, the spectral weight is 0.20, the texture weight is 0.45, and the edge weight is 0.35; under conditions with dense stubble, the spectral weight is 0.25, the texture weight is 0.55, and the edge weight is 0.20.

[0073] Weighted fusion: Pixel-level weighted summation is performed on each independent feature map according to dynamically determined weights to generate a global fused feature map. The fusion method is as follows: the value of the fused feature map at any pixel position is equal to the spectral weight multiplied by the value of the spectral feature map at that position, plus the texture weight multiplied by the value of the texture feature map at that position, plus the edge weight multiplied by the value of the edge feature map at that position.

[0074] Through this adaptive adjustment, the fused enhanced feature map can maintain optimal representational ability in various environments, providing a high-quality information foundation for subsequent large-scale model recognition.

[0075] Step S3: The first multimodal large model generates a text description of the farmland state.

[0076] A multimodal large model is used to understand the enhanced feature map and generate textual information describing the state of farmland. This embodiment uses the Gemma3 model, but those skilled in the art will understand that other multimodal large models with image understanding capabilities can also be used.

[0077] The input includes: the enhanced feature map output from step S2, and site auxiliary information such as site number, shooting time, and geographical location.

[0078] The system executes a pre-set first instruction, which is carefully designed to guide the model to provide a professional, objective, and realistic description focused on harvest-related features. The specific content of the first instruction is as follows: the model is required to provide a realistic description of a farmland scene photograph from the perspective of an agricultural expert, with a word count not exceeding one hundred; it is required to describe whether there are crops in the image, the morphological characteristics of the crops, whether there are signs of harvesting, and the state of the ground cover; it is required to avoid exaggeration and only objectively describe what is visible in the image. Guided by these instructions, the model can focus on visual cues directly related to the harvesting state, eliminating interference from irrelevant background information. For example, given an enhanced feature image of a harvested cornfield, the model might output the following description: "Lying corn stalks are visible in the field, the ground surface is largely bare, there are no green plants, and there are obvious harvesting marks and residual straw debris." Similarly, given an enhanced feature image of an unharvested rice paddy, the model might output: "The rice plants in the field are dense, green, upright and neat, with no obvious lodging or harvesting marks, and the ground surface is completely covered by the crop canopy."

[0079] Step S4: Structured Recognition and Result Extraction of the Second Large-Scale Inference Model

[0080] Following the text description output in step S3, the second large-scale reasoning model performs structured recognition. This embodiment uses the DeepSeek model, but those skilled in the art should understand that other large-scale language models with natural language understanding and reasoning capabilities can also be used.

[0081] The system executes a pre-set second instruction. This second instruction first sets the inference rules: if the description text indicates no crops in the image, it is determined that the crop has been harvested; if the description text indicates crops in the image, it is determined that the crop has not been harvested. Simultaneously, the second instruction forces the model to generate results according to a fixed output template. The template structure is as follows: the first line begins with "Crop Name:", followed by the specific crop name inferred by the model; the second line begins with "Harvest Status:", followed by a status indicator of "Harvested" or "Not Harvested". For example, when the input description text is "Locked corn stalks are visible in the plot, the ground surface is largely bare, and there are no green plants," the model will output the following according to the template: the first line "Crop Name: Corn", and the second line "Harvest Status: Harvested".

[0082] The text description output by the first model has filtered out a large amount of visual noise. The second model makes judgments based on clear semantics, reducing the recognition difficulty in complex farmland scenes. Through the forced formatting rules of the second instruction, the structured and parsable nature of the final result is ensured, avoiding the inconsistency in format that may occur when a single model directly outputs classification results.

[0083] Step S5: Timing-Spatial Joint State Verification

[0084] S51: Time Series Construction

[0085] For each farmland site, obtain the harvest status identification results for several consecutive days. The number of consecutive days should be no less than five. Arrange these results in chronological order to construct the time series sequence for that site. For example, the time series sequence for site A might be: Day 1, no harvest; Day 2, no harvest; Day 3, harvested; Day 4, harvested; Day 5, harvested.

[0086] S52: Mutation Detection

[0087] Calculate the change in harvest status between adjacent dates. In practice, unharvested can be quantified as a value of 0, and harvested as a value of 1. A change in status from 0 to 1 or from 1 to 0 indicates a change, and the change is 1. If the status is the same, the change is 0.

[0088] For a sequence of consecutive variable values, calculate its mean and standard deviation. If the change in state between the current day and the previous day exceeds the threshold obtained by adding twice the standard deviation to the mean, then it is determined that there is a temporal abrupt change at that station on the current day.

[0089] A typical manifestation of a temporal abrupt change is that the identification result suddenly changes to "harvested" on a certain day, while the identification results on the previous and following days were "unharvested". This isolated state jump is most likely a misjudgment caused by environmental factors such as cloud cover.

[0090] S53: Spatial Synchronization Verification

[0091] For sites identified as having a temporal abrupt change, instead of directly accepting or rejecting the current results, a spatial dimension is introduced for cross-validation.

[0092] Query the GIS database to obtain neighboring plots that are geographically connected to the current site and have the same crop type. Calculate the change in harvest status of these neighboring plots within the same time period.

[0093] Synchronization comparison is performed: If the difference between the state change of the current site and the state change of adjacent plots is less than a preset synchronization judgment threshold, and the two changes are in the same direction, then it is determined that a synchronous change in the same direction has occurred, confirming that the state change of the current site is a genuine contiguous harvesting operation. Conversely, if the difference exceeds the threshold or the change directions are inconsistent, it is determined that no synchronous change in the same direction has occurred, and the identification result of the current site is corrected to the state before the mutation occurred.

[0094] It should be noted that in an extreme case, where adjacent plots are initially unharvested and simultaneously covered by extensive cloud cover, each site may be mistakenly identified as harvested because it is not yet harvested. The system may temporarily classify this as a true contiguous harvest on the same day due to the detection of synchronous, unidirectional changes. However, this situation will be automatically identified and corrected during subsequent time-series verification: when the clouds dissipate and the image returns to normal the next day, the identification results for these sites will revert to unharvested, thus creating a state transition of "unharvested - harvested - unharvested." During subsequent time-series abrupt change detection, the system will capture this abnormal pattern that violates the irreversibility of harvesting, determine that there was a misjudgment record due to environmental interference the previous day, and automatically trigger a retrospective update operation on the stored data table for the corresponding date. Using the site's unique identifier as the key, the harvest status of that site will be corrected from harvested to unharvested, thereby ensuring the consistency and logical self-consistency between historical and current data.

[0095] Example of verification logic:

[0096] (1) Continuous harvesting scenario: The current site changes from unharvested to harvested, and a query reveals that adjacent plots also changed from unharvested to harvested at the same time, with the changes occurring synchronously and in the same direction. In this case, it is determined to be a real continuous harvesting operation, and the harvested identification result is retained.

[0097] (2) Cloud shadow occlusion scenario: The current site changes from unharvested to harvested, but the adjacent plots remain unharvested and do not change synchronously. In this case, the change of the current site is determined to be a misjudgment caused by cloud shadow occlusion, and its recognition result is corrected back to unharvested.

[0098] Step S6: Incremental Storage and WebGIS Visualization

[0099] S61: Date Extraction and Image Archiving

[0100] Extract the shooting date from the filenames of farmland scene images. Image filenames typically follow a fixed naming convention, containing eight consecutive digits indicating the shooting time. The system obtains the shooting date information by matching the pattern of these eight consecutive digits in the filename. Based on the extracted date, the system creates a corresponding archive directory, with the directory name containing the date information, and moves the corresponding image files to that directory.

[0101] S62: Incremental Updates of Structured Data Tables

[0102] The system creates a structured data table of recognition results for each day, which can use a comma-separated value format. The data table contains the following fields: station number, city, district / county, longitude, latitude, crop type of the original record, crop type identified by the system, harvest status, whether there is a real-scene image, and the storage path of the image file.

[0103] The incremental update logic is as follows: Read the existing data table file for the current day; use the station number as the unique identifier key for each record; for each newly generated identification result, check if the station number already exists in the table; if not, append a new record to the end of the table; if it already exists, overwrite the existing record corresponding to that station number with the new result. After the update is complete, sort and save the data in ascending order of station number. This ensures that each station retains only one latest record in the table each day, achieving automatic iterative updates of the daily autumn harvest progress.

[0104] Furthermore, the aforementioned incremental storage mechanism also supports retrospective correction of historical data. When the spatiotemporal joint verification of a subsequent date determines that the identification result of the previous day or an earlier date is misjudged and outputs the corrected harvest status, this step will, based on the correction result, perform an overwrite update operation on the stored structured data table for the corresponding date, using the site's unique identifier as the key, to replace the harvest status field of the misjudged record with the corrected value.

[0105] S63: WebGIS Visualization

[0106] The system uses the Flask Web framework to build backend services and Leaflet visualization components to implement WebGIS visualization on the front end.

[0107] Site rendering method: Different colored circular markers are used on the WebGIS visualization interface to distinguish the harvest status, for example, green indicates unharvested and red indicates harvested.

[0108] Interactive features: A date selection control is provided, allowing users to switch between viewing the overall harvest status distribution for different dates; a data table is located at the bottom of the WebGIS visualization interface, with table rows linked to markers on the WebGIS visualization interface. Clicking on a row in the table automatically locates the corresponding site in the WebGIS view; clicking on a site marker on the WebGIS visualization interface brings up an information window displaying detailed information such as site number, crop type, and harvest status, and includes a button to view a real-view image. Clicking this button will preview the original farmland real-view image taken on that day at that site.

[0109] Statistics and export functions: The system supports summarizing and statistically analyzing the percentage values ​​of the current harvest progress by administrative region (such as prefecture-level city or district / county), and can generate and export daily autumn harvest progress reports.

[0110] like Figure 2 The above-described system is a farmland autumn harvest progress monitoring system based on image feature extraction, which is used to implement the above method. The composition and function of each module of the system are described in detail below.

[0111] 1. Data Acquisition Layer

[0112] The data acquisition layer includes: a meteorological platform API interface module, which encapsulates the calling logic of the meteorological big data cloud platform application programming interface, and handles identity authentication, request sending, response parsing, and exception retries; a timed triggering and scheduling module, which automatically triggers the acquisition process at preset time points based on the operating system's timed task scheduler; a site deduplication and download module, which realizes deduplication by site identifier and batch image download, and supports breakpoint resume; and an acquisition log recording module, which records detailed information for each acquisition, including site, time, success or failure status, for easy operation and maintenance monitoring.

[0113] 2. Image preprocessing and feature enhancement layer

[0114] The image preprocessing and feature enhancement layer includes the following modules:

[0115] Image standardization module: Implements functions such as format unification, bilateral filtering for noise reduction, latitude and longitude coordinate transformation, and invalid region removal.

[0116] Feature decoupling extraction module: Configured with three independent parallel processing branches. The spectral branch calculates the normalized vegetation index and soil-regulated vegetation index pixel by pixel, and uses the two index values ​​at each pixel position as channels to combine them into a two-dimensional matrix of the same size as the original image, i.e., the spectral feature map; the texture branch takes a 15×15 pixel neighborhood window centered on each pixel, calculates the mean of the gray-level co-occurrence matrix in four directions as the energy, contrast, entropy, and homogeneity parameters of that pixel, and calculates the mean and variance of the local binary pattern texture spectrum in a circular neighborhood with a radius of 2 and 16 sampling points, and combines the texture parameters as channels to form a texture feature map; the edge branch performs Canny edge detection pixel by pixel to obtain a binary edge intensity map, and performs Hough transform to extract the line direction consistency, and combines the edge intensity and direction consistency as channels to form an edge morphology feature map.

[0117] Environmental parameter calculation module: Divides the image into a preset grid, calculates the brightness, cloud shadow ratio, and stubble density of each grid, and summarizes the environmental parameters. The environmental parameters include three indicators: global brightness, cloud shadow area ratio, and stubble area ratio.

[0118] Environment type determination module: Based on the preset decision tree rules, the module determines the current environment type according to the above environmental parameters. Possible types include normal lighting, strong light, cloudy shadow and dense stubble.

[0119] Dynamic weighted fusion module: Based on the determined environment type, the fusion weights of spectral features, texture features and edge morphology features are determined by looking up a table. Then, a weighted summation operation is performed on each independent feature map pixel by pixel to finally generate an enhanced feature map.

[0120] 3. Dual-model collaborative recognition layer

[0121] The dual-model collaborative recognition layer includes:

[0122] The first recognition module integrates the Gemma3 multimodal large model and loads preset first instructions, namely, agricultural realistic description prompts. This module receives the enhanced feature map generated by the dynamic weight fusion module as input, and outputs text information describing the state of farmland after calling the model.

[0123] The second recognition module integrates the DeepSeek inference model and loads preset second instructions, namely structured format constraint prompts. This module receives the text description output by the first recognition module as input, calls the model, and outputs crop names and harvest status organized according to a fixed template. It also extracts the values ​​of structured fields through text parsing.

[0124] 4. Spatiotemporal Joint Verification Layer

[0125] The spatiotemporal joint verification layer includes:

[0126] Time-series sequence construction module: Organizes the identification results of the same site over multiple consecutive days in chronological order to construct a time-series sequence.

[0127] Mutation detection module: Calculates the change in harvest status between adjacent dates, performs statistical analysis based on the mean and standard deviation of the change sequence, and detects time-series mutations that exceed the normal fluctuation range.

[0128] Spatial adjacency verification module: Query the adjacency relationships of plots stored in the GIS database, obtain the harvest status change information of plots adjacent to the current site, and verify whether synchronous and unidirectional changes have occurred between the two.

[0129] State Correction Module: This module combines the results of mutation detection and spatial verification to classify the identification results. If the misjudgment is confirmed to be caused by environmental interference, the result is corrected to the state before the mutation; if it is confirmed to be a genuine harvesting behavior, the original identification result is retained.

[0130] 5. Incremental Storage and Visualization Layer

[0131] The incremental storage and visualization layer includes:

[0132] Date Extraction and Archiving Module: Extracts date information from filenames, creates directories by date, and archives image files.

[0133] Incremental update module for data tables: Using the station number as a unique identifier, it performs operations to insert new records or overwrite existing records in the structured data table for the current day, ensuring data consistency and up-to-dateness.

[0134] WebGIS Visualization Module: Based on the Flask framework and Leaflet visualization component library, it realizes functions such as rendering the WebGIS visualization interface for site status, viewing by date switching, interactive linkage between data tables and WebGIS visualization interface markers, and pop-up preview of real-scene images.

[0135] Reporting and Early Warning Module: Supports summarizing and statistically analyzing the autumn harvest progress by administrative region, exporting report files, and automatically triggering early warning notifications when a region's progress lags behind and exceeds a preset threshold.

[0136] External API Interface Module: Provides application programming interfaces that conform to RESTful specifications, supporting data connection and sharing with external agricultural management platforms.

[0137] The specific implementation process is as follows:

[0138] Example 1: Performance Verification under Complex Weather Conditions

[0139] Verification scenario: Select real-world images of farmland in a certain area for 5 consecutive days, during which the following weather changes occurred: Day 1: Clear skies and normal lighting; Day 2: Cloudy with some areas obscured by cloud shadows; Day 3: Overcast with large areas covered by clouds; Day 4: Clear skies returned; Day 5: Clear skies and normal lighting.

[0140] Two schemes were set up for comparison: Scheme A uses a single model for direct recognition without feature enhancement and spatiotemporal verification; Scheme B uses the complete scheme of this invention.

[0141] The comparison results are as follows: With the actual autumn harvest progress gradually increasing from 30% on day 1 to 45% on day 5, Solution A, due to cloud cover on days 2 and 3, artificially inflated the reported progress to 52% and 61% respectively. After the clouds dissipated on day 4, the progress dropped back to 38%, exhibiting an abnormal data jump that rendered the solution unusable for business reference. Solution B, on the other hand, maintained the stability and accuracy of the reported progress throughout the entire process, with the reported progress trend consistent with the actual progress, and no abnormal jumps or inconsistencies were observed.

[0142] Solution A, lacking a protection mechanism against environmental interference, resulted in severe misjudgments under cloudy and overcast conditions. The solution of this invention employs environmentally adaptive weight fusion, automatically reducing the weight of spectral features and increasing the weight of texture and edge features in cloudy or shadowy environments, thus suppressing the interference of shadows on the recognition results. Simultaneously, the temporal verification mechanism detects inconsistencies between the recognition results on the 2nd and 3rd days and the stable states before and after, while spatial adjacency verification reveals that adjacent plots have not undergone synchronous state changes. Therefore, these are identified as environmental interference and automatically corrected to the state before the abrupt change, thereby ensuring stable and reliable data output throughout all time periods.

[0143] The above descriptions are merely embodiments of the present invention, and common knowledge such as specific technical solutions and / or characteristics are not described in detail here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the technical solutions of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for monitoring the progress of autumn harvest in a farmland based on image feature extraction, characterized in that, Includes the following steps: S1. Automatically call the meteorological data platform interface by triggering a task at a time to obtain the real-scene image metadata of each farmland station in the area to be monitored, remove duplicates according to the unique identifier of the station, retain the latest real-scene image of the farmland taken by each station and download it to the local machine, and perform standardized preprocessing on the farmland real-scene image to obtain a standardized farmland image. S2. Perform multimodal feature decoupling extraction on the standardized farmland image, and calculate and generate spectral feature map, texture feature map and edge morphology feature map respectively; Furthermore, the fusion weights of each feature are dynamically determined based on the real-time environmental state, and the obtained independent feature maps are weighted and fused to generate an enhanced feature map. S3. Input the enhanced feature map into the first multimodal large model and execute the preset first instruction to make the first multimodal large model generate text information describing the state of farmland based on the enhanced feature map; S4. Input the text information into the second inference model and execute the preset second instruction, so that the second inference model outputs the identification results of crop type and harvest status according to the text information and the preset structured format; extract the values ​​of crop name and harvest status fields from the identification results through regular expressions as the identification results of crop type and harvest status; S5. Obtain the harvest status identification results of the same site for multiple consecutive days, construct a time series sequence, calculate the amount of harvest status change between adjacent dates, and determine the time series mutation when the amount of harvest status change exceeds a preset threshold; for the site determined to be a time series mutation, query the amount of harvest status change of its GIS adjacent plots in the same time period. If the adjacent plots have not changed synchronously in the same direction, then correct the current identification result to the state before the mutation; otherwise, confirm it as a real harvesting behavior. S6. The harvest status after verification and correction is stored incrementally in a structured data table created by date, using the site's unique identifier as the key, to ensure that only one record is retained for the same site on the same day, and then output to the visualization platform for display.

2. The method for monitoring the progress of autumn harvest in farmland based on image feature extraction according to claim 1, characterized in that, Specifically, step S2 involves: Three parallel feature extraction branches are set up: the spectral feature branch calculates the normalized vegetation index and soil-regulated vegetation index to generate a spectral feature map; the texture feature branch extracts texture parameters using gray-level co-occurrence matrix and local binary mode to generate a texture feature map; and the edge morphology branch extracts field contours and crop row features using edge detection and Hough transform to generate an edge morphology feature map. The standardized farmland image is divided into a preset grid, and the brightness, cloud shadow ratio and stubble density of each grid are calculated. Global environmental parameters are statistically analyzed. The current environment type is determined based on the preset decision tree, including normal lighting, strong light, cloudy and shadowy, and dense stubble environment; Based on the preset mapping relationship between different environment types and feature weights, the fusion weights of spectral features, texture features, and edge morphology features are dynamically adjusted, and pixel-level weighted fusion is performed on each independent feature map to generate the enhanced feature map.

3. The method of claim 1, wherein the method comprises: The first instruction in step S3 includes preset agricultural image description rules, which guide the first multimodal large model to describe the crop morphology, ground condition and harvesting marks in the image in a realistic style, while excluding exaggerated rhetoric.

4. The method of claim 1, wherein the method comprises: The second instruction in step S4 includes a preset structured output format rule, which is used to force the second inference model to output the recognition result according to the fixed template of "crop name: [value]; harvest status: [harvested / not harvested]".

5. The method of claim 1, wherein the method comprises: The timed triggering task in step S1 is specifically a task that is automatically triggered at a preset time every day; after the timed triggering task is triggered, the meteorological data platform interface is called to obtain the real-scene image metadata of all farmland stations from the previous day.

6. The method of claim 1, wherein the method comprises: The incremental storage in step S6 specifically includes: Extract the shooting date from the filenames of farmland scene images using regular expressions, and archive the image files according to the shooting date; A structured data table is created for each shooting date, with the site's unique identifier serving as the unique key within the structured data table; Perform insert or update operations on the identification results of the day: if the unique key does not exist in the table, add a new record; if it already exists, overwrite the original record to ensure that only one latest record is retained for each site each day.

7. The method of claim 1, wherein the method comprises: The visualization platform in step S6 is built on WebGIS, uses different colors to render the harvest status of the site, and supports switching by date, linked table display, and real-scene image pop-up preview.

8. An image feature extraction-based farmland autumn harvest progress monitoring system, characterized in that, include: The data acquisition layer is used to automatically call the meteorological data platform interface through timed triggering tasks to obtain real-scene image metadata of each farmland station in the area to be monitored, deduplicate by the unique identifier of the station, retain the latest real-scene image of the farmland taken by each station and download it to the local machine; The image preprocessing and feature enhancement layer is used to perform standardized preprocessing and multimodal feature decoupling extraction on farmland scene images, respectively calculating and generating spectral feature maps, texture feature maps and edge morphology feature maps; Furthermore, the fusion weights of each feature are dynamically determined based on the real-time environmental state, and the obtained independent feature maps are weighted and fused to generate an enhanced feature map. The dual-model collaborative recognition layer includes: The first recognition module is used to call the first multimodal large model and execute the preset first instruction to generate text information describing the state of farmland based on the enhanced feature map; The second recognition module is used to call the second inference model and execute the preset second instruction, so that it outputs the recognition result according to the preset structured format based on the text information; and extracts the values ​​of the crop name and harvest status fields from the recognition result through regular expressions as the recognition result of crop type and harvest status. The spatiotemporal joint verification layer is used to obtain the harvest status identification results of the same site for multiple consecutive days, construct a time series, calculate the amount of change in harvest status between adjacent dates, and determine the time series mutation when the amount of change exceeds a preset threshold. For the site determined to be a time series mutation, query the amount of change in harvest status of its GIS adjacent plots in the same time period. If the adjacent plots have not changed synchronously in the same direction, the current identification result is corrected to the state before the mutation; otherwise, it is confirmed as a real harvesting behavior. The incremental storage and visualization layer is used to incrementally store the verified and corrected harvest status, using the site's unique identifier as the key, and creating a structured data table by date. This ensures that only one record is retained for the same site on the same day, and the data is then output to the WebGIS platform for visualization.

9. The system for monitoring the progress of the autumn harvest in a field based on image feature extraction as claimed in claim 8, characterized in that, The image preprocessing and feature enhancement layer specifically includes: The feature decoupling extraction module is configured with three parallel feature extraction branches, which are used to calculate vegetation index to generate spectral feature map, extract texture parameters to generate texture feature map, and extract edge morphology features to generate edge morphology feature map, respectively. The environmental parameter calculation module is used to divide the standardized farmland image into a preset grid, and to calculate the brightness, cloud shadow ratio and stubble density of each grid to obtain environmental parameters, including global brightness, cloud shadow area ratio and stubble area ratio. The environment type determination module is used to determine the current environment type based on a preset decision tree and environment parameters; The dynamic weight fusion module is used to dynamically determine the fusion weight of each feature according to the preset mapping relationship between environment type and feature weight, and to perform weighted fusion on each independent feature map to generate the enhanced feature map.

10. The system for monitoring the progress of the fall harvest in a field based on image feature extraction as claimed in claim 8, wherein, The spatiotemporal joint verification layer specifically includes: The time-series sequence construction module is used to organize the identification results of the same site over multiple consecutive days in chronological order; The mutation detection module is used to calculate the amount of change in harvest status between adjacent days and to detect temporal mutations based on a preset threshold. The spatial adjacency verification module is used to query the adjacency relationship of GIS plots, obtain the change in harvest status of adjacent plots within the same time period, and verify whether the changes occur synchronously and in the same direction with the current site. The state correction module is used to distinguish between environmental interference and actual harvesting behavior based on mutation detection results and spatial verification results, and to correct the identification results.

Images