Crop growth analysis system based on machine vision

Abstract

The invention discloses a crop growth analysis system based on machine vision, and relates to the technical field of agricultural intelligent monitoring and analysis, the system comprises a multi-source acquisition module, a disturbance identification module, a growth behavior modeling module, an environmental adaptability correction module and a feedback prediction module; wherein the multi-source acquisition module can synchronously acquire visible light, near-infrared and depth images, and generates a joint index in combination with environmental data; the disturbance identification module identifies abnormal areas such as wind disturbance and shielding based on the residual image and the image structure network; the growth behavior modeling module is used for extracting internode change, leaf surface tension and bifurcation angle characteristics by using a three-dimensional structure time sequence alignment and hidden change rate coding network; the environment adaptability correction module constructs a crop-environment double-domain mapping relation; the feedback prediction module outputs personalized intervention suggestions in combination with deviation analysis and crop variety embedding vectors; the system has the advantages of high modeling precision, strong adaptability, fast feedback response and the like, and is suitable for intelligent planting management of various crops.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural intelligent monitoring and analysis, and in particular to a crop growth analysis system based on machine vision. Background Art

[0002] Currently, identifying and assessing crop growth status is crucial in agricultural plantation management. Accurately and timely understanding of crop growth processes is crucial, especially in the context of precision agriculture and unmanned farms. Existing solutions often rely on extracting static features from two-dimensional images, such as crop color, morphological boundaries, or leaf count. However, these solutions often suffer from reduced recognition accuracy, limited feature representation, and weak model generalization capabilities when dealing with complex farmland backgrounds, seasonal variations, and varietal differences.

[0003] In addition, existing systems generally lack the three-dimensional spatial feature reconstruction and dynamic behavior recognition mechanism for the continuous growth process of crops, making it difficult to capture the detailed evolution characteristics of crops during the microscopic growth cycle; at the same time, some methods fail to fully consider non-ideal factors such as structural occlusion, self-deformation and environmental disturbances between plants in farmland scenes during the analysis process, resulting in obvious deviations in growth trend assessment.

[0004] Therefore, there is an urgent need for an analysis system that has the ability to fuse heterogeneous information, perform adaptive modeling based on cross-period image sequences, and identify multi-scale growth behaviors in combination with the physical process of crop growth, so as to improve the accuracy and versatility of crop growth status assessment and thus realize intelligent monitoring and management of the entire life cycle of farmland. Summary of the Invention

[0005] The purpose of the present invention is to provide a crop growth analysis system based on machine vision to address the shortcomings of the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a crop growth analysis system based on machine vision, comprising: Multi-source acquisition module, used to collect multi-mode image data under different spectra during the crop growth cycle, and automatically mark the mapping relationship between the acquisition time and meteorological data; The disturbance recognition module is used to perform multi-view sparse matching on multi-modal image data, construct a three-dimensional temporal structure map of the crop plants, and combine the reconstruction residual model to identify image distortion areas caused by wind disturbance, occlusion, or plant posture changes; The growth behavior modeling module is used to extract the internode elongation rate, leaf tension distribution, and bifurcation angle variation characteristics of crops based on a three-dimensional time series structure diagram, and to construct a cross-stage growth trend model using a latent rate coding mechanism; The environmental adaptability correction module is used to dynamically adjust the growth trend model based on the measured temperature and humidity, light intensity, and soil conditions of the target plot, and establish a crop-environment dual-domain mapping relationship; The feedback prediction module is used to output early warnings based on the degree of deviation between the current growth trend model and the standard growth dynamics curve, and to generate intervention recommendation signals based on the target crop varieties.

[0007] Preferably, the multi-source acquisition module specifically includes: An out-of-situ viewing angle acquisition architecture is constructed using an integrated multispectral camera array. This architecture simultaneously captures visible light, near-infrared, and depth images during each sampling cycle. The main control system dynamically adjusts the exposure ratio and spectral channel weights to adapt to the varying reflectance of different crop leaves. At the time of collection, the temperature, light intensity, and wind speed parameters recorded by the local micrometeorological collection unit are bound through a high-precision timestamp mechanism to generate a joint index table of image sampling frames and environmental conditions; A spectral consistency calibration model is constructed, dynamic weighting is performed based on the difference in inter-pixel reflection intensity between the near-infrared channel and the visible light channel, and a spatial correlation regularization network is used to jointly encode multimodal images.

[0008] Preferably, the disturbance identification module includes: A multi-baseline imaging strategy is used to perform low-overlap sparse matching of multi-view images. The geometric distribution of natural plant feature points in unstructured images is used to construct a cross-view matching graph. The multi-scale SIFT-HOAD joint descriptor is used to improve the robustness of leaf tip and stem node recognition. The structural alignment residual graph is constructed using the sparse point cloud reconstruction results and the known crop skeleton template. The Euclidean deviation and connectivity break index between key structural units are calculated, and a weighted residual energy function is introduced as a dynamic adjustment factor for the distortion sensitivity threshold. The residual map is spatially analyzed through a convolutional residual comparison network to automatically identify abnormal reconstruction areas caused by wind disturbance, uneven lighting, and plant tilt.

[0009] Preferably, the disturbance identification module further includes: A perturbation graph structure network built based on the structural residual graph takes each 3D structural key point as a graph node. The node attributes include local geometric offset, brightness gradient direction and texture complexity index, and the graph edge relationship is defined based on the residual connection weight; An improved direction-aware graph convolutional network is used to iterate the graph structure, integrate spatially heterogeneous disturbance information, extract topological coherence features between regions, and dynamically mark regions with high probability of topological breaks as candidate anomaly regions. The candidate abnormal region is fused with the structural integrity score map, and the abnormal region is refined through a dual-path residual inversion mechanism.

[0010] Preferably, the growth behavior modeling module includes: Based on the three-dimensional time series structure diagram, a cross-time point cloud repositioning algorithm was used to align the key nodes of each plant in time series. The internode length change, leaf tension relaxation coefficient, and bifurcation angle vector transfer rate between consecutive frames were calculated to generate the original growth feature tensor. The growth feature tensor is dimensionally mapped and reconstructed by an adjustable nonlinear nested kernel function to enhance the nonlinear interaction expression between features, and the stage transition boundary of the growth pattern is identified by multidimensional dynamic discriminant clustering; A time-oriented latent rate coding network is constructed to encode the original feature sequence into an implicit growth rate parameter sequence, and the model learning rate is dynamically adjusted through the differential information entropy control mechanism.

[0011] Preferably, the environmental adaptability correction module includes: The multi-source time series fusion of the measured environmental factors of the target plot is performed to construct a multi-dimensional environmental state vector including temperature, humidity, light intensity and soil conductivity indicators; The environmental state vector is input into a coupled orthogonal feature transformation module, and its semantic layer change trend and numerical layer perturbation response are extracted respectively through a dual-channel transformation network, and a dual-domain correlation matrix between the crop growth feature domain and the environmental state domain is constructed; Based on the constructed dual-domain correlation matrix, a weight regulation submodule of the growth trend model is introduced, and the conditional weight mapping mechanism is used to recalibrate the dynamic feature channels in the model layer by layer.

[0012] Preferably, the feedback prediction module includes: The latent rate series output by the current growth trend model is matched with the standard growth dynamics curve on a time-by-time basis, and a weighted displacement-micro-change difference dual-index modeling strategy is adopted to calculate the structural deviation value and dynamic offset curve. Construct a crop variety embedding vector library, generate multidimensional variety gene codes based on crop genetic traits and variety historical growth records, and input them into an adversarial intervention inference network to establish an optimal intervention path mapping model between the standard curve and abnormal deviations; Based on the current deviation value and the intervention path of the corresponding crop variety, an intervention recommendation signal is output, including adjusting water and fertilizer strategies, shading control duration or preventive pesticide application methods, and accompanied by timing execution recommendations.

[0013] In the above technical solution, the technical effects and advantages provided by the present invention are: 1. This invention effectively improves the robustness of crop structure recognition and the accuracy of trend modeling in complex field environments by constructing an integrated multi-source acquisition architecture, a disturbance recognition mechanism, a growth behavior modeling network, and an environmental adaptability feedback loop. The system not only dynamically acquires multispectral imagery and micrometeorological data but also enables in-depth modeling of internode elongation, tension evolution, and bifurcation angles based on three-dimensional structural changes, significantly enhancing responsiveness to early-stage growth anomalies.

[0014] 2. Compared with existing technical approaches that rely on static images or rule-based judgments, the present invention has stronger environmental adaptability, real-time early warning capabilities, and intelligent intervention decisions. It can generate personalized management suggestions for different crop varieties and form executable closed-loop control, thereby realizing intelligent perception, trend prediction, and precise regulation of the crop growth process. It has broad prospects for promotion and application and significant economic and agricultural management value. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0016] Figure 1 This is a mind map of the system modules of the present invention. DETAILED DESCRIPTION

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0018] For examples, see Figure 1 As shown, the crop growth analysis system based on machine vision described in this embodiment includes: Multi-source acquisition module, used to collect multi-mode image data under different spectra during the crop growth cycle, and automatically mark the mapping relationship between the acquisition time and meteorological data; The disturbance recognition module is used to perform multi-view sparse matching on multi-modal image data, construct a three-dimensional temporal structure map of the crop plants, and combine the reconstruction residual model to identify image distortion areas caused by wind disturbance, occlusion, or plant posture changes; The growth behavior modeling module is used to extract the internode elongation rate, leaf tension distribution, and bifurcation angle variation characteristics of crops based on a three-dimensional time series structure diagram, and to construct a cross-stage growth trend model using a latent rate coding mechanism; The environmental adaptability correction module is used to dynamically adjust the growth trend model based on the measured temperature and humidity, light intensity, and soil conditions of the target plot, and establish a crop-environment dual-domain mapping relationship; The feedback prediction module is used to output early warnings based on the degree of deviation between the current growth trend model and the standard growth dynamics curve, and to generate intervention recommendation signals based on the target crop varieties.

[0019] This embodiment provides a multi-source acquisition module, which is a key component of the machine vision-based crop growth analysis system described in the present invention. It is used to perform high-quality acquisition of multi-channel image data and environmental parameter mapping during the crop growth cycle, thereby achieving adaptive data acquisition for different crops at different growth stages and under different environmental conditions, and providing a highly reliable data foundation for subsequent three-dimensional structure modeling, disturbance identification, and growth behavior modeling.

[0020] This multi-source acquisition module primarily involves the construction of a heterogeneous view acquisition architecture, joint indexing modeling of image and environmental data, and the implementation of a spectral consistency calibration mechanism. Each step is described in detail below.

[0021] This implementation utilizes an integrated multispectral camera array, specifically comprising at least one visible light camera, one near-infrared camera, and a depth camera based on structured light or Time-of-Flight (TOF) principles. These cameras are physically distributed on a capture stand at preset angles, forming a heterogeneous viewpoint acquisition array. Data is synchronized between the cameras using a clock synchronization module to ensure consistent acquisition timestamps, and they are all connected to a central control system.

[0022] During each sampling cycle, the main control system issues unified acquisition commands to the camera array, activating each spectral channel camera simultaneously within a very short time interval to collect multimodal image data for the current crop area. Because the reflectance characteristics of different crop leaves vary significantly across different spectral bands (particularly the near-infrared and red-edge bands), the main control system incorporates a dynamic exposure and channel weighting mechanism. Using a preset or real-time crop spectral reflectance model, the system dynamically adjusts the exposure time, gain factor, and pixel mapping ratio of each camera, maximizing the information content of each channel image while minimizing saturated or underexposed areas.

[0023] In addition, the differences in perspectives of multiple cameras arranged in different locations provide multi-viewpoint redundant information, which is beneficial for subsequent three-dimensional reconstruction and depth completion processing, especially when facing complex backgrounds (such as overlapping branches and leaves, low-contrast textures, etc.), improving image stability and structural integrity.

[0024] To enhance the expressiveness of image data in time series analysis and environmental modeling, this module further implements deep binding between images and environmental states.

[0025] Specifically, during each sampling process, the system records the exact moment of image acquisition through a high-precision timestamp mechanism, and simultaneously retrieves the environmental data recorded by the local micrometeorological acquisition unit deployed on the target site, including but not limited to: air temperature, relative humidity, light intensity (radiant power per unit area), wind speed and direction, soil conductivity and humidity and other parameters.

[0026] The acquisition module uses the timestamp of the image sampling frame as an index key, forming a set of data pairs with the meteorological data within the corresponding time period, which are stored in the system's joint index table. This index table has the following structured attributes: {image frame number, acquisition time, spectrum type, acquisition angle, and environment vector}, where the environment vector is a state descriptor composed of multidimensional real numbers.

[0027] This mechanism not only ensures the precise correspondence between images and environmental data on the time axis, but also provides a combination of input features that can be used for training subsequent growth behavior models, helping to establish a mapping model between crop growth responses and environmental disturbances.

[0028] In multispectral imaging systems, due to factors such as differences in sensor response curves and synchronization errors, images from different channels may be geometrically and spectrally inconsistent. To address this, this implementation design employs a spectral consistency calibration model, which includes two key steps: First, the visible and near-infrared images are aligned (using cross-correlation matching of local image patches). Then, for each image pixel, the difference in reflectance intensity ΔR across different channels is calculated. A spectral response surface is constructed based on ΔR. An empirical response model based on crop type and stage characteristics is introduced to generate a weighting matrix W(x, y). This matrix dynamically adjusts the channel outputs to suppress the influence of noise and improve the consistency of the response of the same physical structure across different channels.

[0029] Before multimodal image fusion, a Spatial Correlation Regularization Network (SCR-Net) is used to jointly encode multiple channel images. This network inputs normalized image patches from each channel and outputs uniformly encoded feature maps. The network architecture includes a multi-scale receptive field extraction layer, a cross-channel attention fusion module, and an edge-preserving loss function to preserve structural information and optimize the reconstruction of spatial relationships.

[0030] Ultimately, the fused output image feature map not only has response consistency at the spectral level, but also maintains geometric alignment characteristics at the spatial level, providing high-quality input data for subsequent three-dimensional temporal structure modeling.

[0031] The disturbance recognition module involved in this embodiment is an important component used in the present invention for stability analysis and abnormal area identification of three-dimensional time-series structure diagrams of crops. It aims to effectively detect image reconstruction distortion areas caused by factors such as wind disturbance, uneven lighting, structural occlusion, or changes in plant posture, thereby ensuring the accuracy of subsequent growth behavior modeling and trend analysis.

[0032] The disturbance identification module comprehensively adopts low-overlap sparse matching, multi-scale joint feature extraction, structural residual modeling, improved graph convolutional network and residual inversion mechanism. Its specific implementation process is described as follows.

[0033] To address the problems of complex structural layers, severe occlusion, and sparse leaf texture in natural crop planting environments, this implementation prioritizes the use of a multi-baseline imaging strategy for image acquisition, that is, collecting multiple sets of view image data under large viewing angle differences, thereby enhancing the ability to acquire depth information.

[0034] Based on this, the module uses a low-overlap sparse matching algorithm to extract and match feature points from multi-view images. Unlike conventional high-density matching strategies, this method uses natural plant feature points (such as leaf tips, leaf margins, and internode intersections) in the captured images, combining their spatial distribution characteristics in unstructured images to construct a cross-view matching graph. This graph is represented using a graph theory structure, with nodes representing feature points and edges representing the viewpoint matching relationships between images.

[0035] To improve matching robustness, this module uses a multi-scale SIFT-HOAD joint descriptor. SIFT (Scale-Invariant Feature Transform) extracts local gradient distributions, while HOAD (Histogram of Angular Derivative) enhances the representation of curved structures (such as the curvature of leaf edges). The combination of these two significantly improves matching stability for areas with unique crop structures (such as curvature at stem nodes and sharp corners at leaf tips).

[0036] Through the above processing, the system obtains a set of sparse three-dimensional point cloud structures containing the main crop skeleton information.

[0037] To identify distorted areas during reconstruction, this implementation uses the sparse point cloud to perform structural alignment with a built-in standard crop skeleton template. This template, generated using a pre-trained model, contains geometric features such as internode length and leaf growth angle for standard varieties.

[0038] During the alignment process, the module calculates the Euclidean spatial deviation of each structural unit (such as internode connection segment and leaf centerline) and introduces connectivity break index to judge the change of structural continuity.

[0039] In order to further improve the sensitivity of distorted area identification, the system constructs a structural alignment residual map, which records the spatial deviation and topological continuity score of each structural unit from the standard model.

[0040] On the residual graph, the system introduces a weighted residual energy function, which is defined as follows: ;in To be practical, is the template point, is the weight factor related to the structural importance, Fracture score, is an adjustable disturbance sensitivity factor. This energy function serves as the basis for adjusting the distortion sensitivity threshold, enabling the system to have adaptive anomaly recognition capabilities.

[0041] This implementation further introduces a convolutional residual difference (CNN) network to perform spatial deconstruction and regional aggregation on the structural alignment residual map. The network structure includes three convolutional layers, two residual connection layers, and a dual-branch output structure, which are used to identify: Localized areas of high deviation (e.g., wind-induced blade deflection); Global structural discontinuity areas (e.g., structural discontinuities caused by occlusion); The overall tilt area of ​​the plant posture.

[0042] The network is supervised and trained based on real distorted samples in the training set, and the output is a binary abnormal region mask map, which is used to guide subsequent graph structure modeling.

[0043] To enhance spatial structure analysis capabilities, this module further constructs a Disturbance Graph Network (DGN). Its principle is to treat each key point of the 3D structure as a graph node, and the node attributes include: Local geometric offset value (from the residual map); Brightness gradient direction (reflecting lighting changes); Texture complexity index (used to judge structural stability).

[0044] Graph edges are constructed through residual connection weights, taking into account spatial proximity and historical growth coherence.

[0045] On this perturbation graph structure, the system deploys an improved Directional-Aware Graph Convolutional Network (Directional-GCN), which has the following capabilities: Preserve spatial orientation information; Extract topological coherence features between regions; Enhance the contrast in properties between fractured and stable areas.

[0046] The output of network training is the "topology fracture probability" of each node, and the system automatically marks the candidate abnormal areas based on the probability threshold.

[0047] Finally, the system fuses the candidate anomaly regions with the aforementioned structural integrity score map, which is jointly weighted based on the residual map and the graph neural network output.

[0048] Subsequently, a dual-path residual inversion mechanism was introduced. This mechanism constructs two independent inversion paths based on the original residual map channel and the topological fracture scoring channel. Each path consists of a residual suppression layer, a structure reconstruction layer, and an edge recovery layer. Finally, by weighted fusion of the outputs of the two paths, the boundaries of the abnormal regions are refined and false positives are eliminated, achieving high-precision identification of disturbed areas.

[0049] This embodiment provides a growth behavior modeling module for accurately extracting the microstructural growth behavior of crops from three-dimensional temporal structure diagrams, enabling deep modeling and phase identification of temporal trends. Based on geometric traits such as internode length, leaf tension, and bifurcation angle changes, this module utilizes a multidimensional feature interaction mechanism and a time-oriented implicit modeling framework to achieve growth trend modeling, phase boundary identification, and predictive behavior abstraction.

[0050] To accurately capture the geometric changes of crops throughout their growth cycle, this module first analyzes the input 3D temporal structure graph frame by frame. This structure graph is the cleaned point cloud sequence output by the disturbance recognition module, representing the 3D spatial structure of the crop at multiple consecutive time points.

[0051] The system introduces a key point alignment mechanism based on the Cross-Temporal Point Relocation (CTPR) algorithm. The steps are as follows: For each plant, the system extracts its key structural node set (including stem nodes, bifurcation points, main leaf tips, etc.) and constructs the point set structure of each frame; A spatial graph matching algorithm is used to match and relocate key nodes of adjacent frames with distance stability and topological similarity as the cost function to generate an inter-frame alignment index table. Using the alignment index, the key structures between consecutive frames are differentiated to obtain: Internode length change (ΔL); The leaf surface tension relaxation coefficient (T_relax) is defined as the combined index of the normal change and curvature change rate per unit area; The bifurcation angle vector transfer rate (A_diff) is calculated by the change of the spatial angle vector sequence.

[0052] The above-mentioned geometric change parameters constitute the structural response characteristic sequence of the crop in this time window. The system organizes them into original growth characteristic tensors with time dimension, structure dimension and type dimension as input for subsequent modeling.

[0053] Considering that the growth behavior of crops exhibits nonlinear and multi-factor coupled characteristic changes at different stages, this module performs dimension mapping and structural reconstruction on the original growth feature tensor to improve the feature expression capability.

[0054] Specifically, the system constructs a set of adjustable nonlinear nested kernel functions , n represents the total kernel function; each kernel function acts on a different projection dimension of the tensor, and the response range and intensity of the kernel function are dynamically adjusted through an adjustable parameter set.

[0055] For example, the radial basis kernel is used for the tension coefficient sequence , where x, x′ represent two growth feature vector samples in the input space, usually representing the values ​​of a certain structural feature (such as tension relaxation coefficient) at different time points or on different plants; Represents the kernel width control parameter (often called "bandwidth"), which determines the sensitivity of the function to distance changes; a periodic kernel is used for angle changes , where Represents the periodic scale parameter, which determines the "tightness" of the function period or the periodic sensitivity to angle difference: the system adaptively selects kernel combinations based on growth feature categories and crop varieties, and reconstructs the multi-kernel mapping results into tensors.

[0056] In the reconstructed feature space, the system deploys a set of Multi-Dimensional Adaptive Clustering (MDAC) algorithms, the core of which are: Construct a similarity graph based on feature local density and directional consistency; A self-adjusting cluster radius strategy is used to achieve highly sensitive identification of characteristic mutation points across stages; The cluster label sequence of each time point is output, and the system determines the position of the stage transition boundary through differential analysis.

[0057] This module effectively distinguishes multiple stages of structural evolution patterns such as growth period, rapid expansion period, stabilization period, and stagnation period, providing a structural basis for trend analysis and regulation.

[0058] To further abstract the intrinsic growth trends of crops, this module constructs a temporal latent rate encoder (TLRE) network. This network aims to encode the original growth feature tensor sequence into a temporally continuous and semantically interpretable sequence of latent growth rate parameters.

[0059] The TLRE network structure includes: Input embedding layer: maps the multi-channel growth feature tensor to a fixed-dimensional representation space; Temporal encoder layer: uses stacked bidirectional LSTM or Transformer modules to model temporal dependencies; Latent rate generation layer: outputs the mean and standard deviation of the latent variable at each time step in the form of normal distribution parameters, and uses reparameterization techniques to generate the latent growth rate zt; Supervised calibration layer: If there is a manual labeling stage, alignment loss can be introduced to assist training.

[0060] To further optimize model convergence and generalization performance, TLRE integrates a Differential Entropy Regulator (DER) mechanism, specifically: In each training cycle, the system calculates the information entropy difference of the latent rate sequence output by the model; If the current entropy growth rate exceeds the threshold, the system automatically reduces the learning rate and freezes some low-variable channels; If the entropy is too low, it means that the model is in an overfitting or oscillating state. In this case, the gradient amplitude should be increased appropriately or random perturbations should be injected.

[0061] This mechanism forms a learning rate self-regulating loop through entropy increase and control feedback, ensuring that the model has good adaptability and explanatory power for complex crop growth trends.

[0062] This embodiment provides an environmental adaptability correction module to address growth trend model drift caused by heterogeneous environmental conditions within crop planting plots. By constructing an environmental state vector, a multi-channel feature transformation mechanism, and a dual-domain weight mapping model, this module enables dynamic adaptive adjustment of the growth analysis model to complex environmental perturbations, enhancing the system's generalization capabilities and deployment stability.

[0063] During actual crop cultivation, microclimate conditions (such as temperature, humidity, and light intensity) and soil physical and chemical properties (such as electrical conductivity and moisture content) across different plots directly impact crop growth. Therefore, to achieve model adaptability, it is first necessary to accurately extract and structure the environmental factors of the target plot.

[0064] This module is deployed in the multi-source environment collection unit on the system side and collects the following environmental factor data: Air temperature (unit: °C), collection frequency: every 10 minutes; Relative humidity (unit: %RH), collection frequency: every 10 minutes; Light intensity (unit: W / m²), acquisition frequency: per minute; Soil electrical conductivity (unit: dS / m), sampling frequency: every hour; Optional parameters: wind speed, rainfall, CO2 concentration, etc.

[0065] To enhance time consistency, this module aligns all environmental data based on timestamps and performs multi-source time series fusion processing. The fusion algorithm includes: Sliding average within the time window (smoothing out short-term fluctuations); Missing point interpolation (based on spline or linear completion); Noise removal (introducing Z-score to remove outliers).

[0066] Finally, the environmental data of a plot within a specified time window is represented as a multidimensional environmental state vector ,in Indicates temperature, Indicates humidity, Indicates the light intensity, Represents soil conductivity, etc. The environmental state vector is used as input to the feature transformation module of the next stage.

[0067] To explicitly couple the growth model with the environmental state, this implementation utilizes a Coupled Orthogonal Feature Transformer (COFT) module. This module uses a dual-channel neural network structure to extract both the semantic and numerical perturbation responses of the environmental state vector, thereby establishing a mapping mechanism with the crop structural feature domain.

[0068] This module consists of two core subpaths: Semantic Trajectory Channel: Normalize the environment state vector; Use 1D convolution-residual structure to extract trend evolution path across time window; The output result is the trend embedding vector .

[0069] Perturbation Sensitivity Channel: Perform local differential calculation on the environment state vector to capture high-frequency disturbance components; Input to a multilayer perceptron (MLP) to extract nonlinear perturbation responses; Output perturbation embedding vector .

[0070] The two channel results are concatenated by linear transformation to generate a joint environment representation At the same time, the system extracts the structural feature representation tensor of the current crop from the structural behavior modeling module of the main model. , R is a set of real numbers; the crop growth feature vector is generated by global average pooling and embedding dimensionality reduction .

[0071] This module further constructs the crop-environment dual domain correlation matrix , defined as follows: ;in is the learnable weight matrix, is a normalized activation function (such as ReLU or Sigmoid). This matrix is ​​used to measure the high-dimensional mapping relationship between the crop structure domain and the environmental state domain, and serves as an intermediate variable input for the weight control mechanism.

[0072] In order to achieve real-time adaptation of model behavior to differences in the external environment, this module introduces a conditional weight mapping submodule to adjust the response strength of each dynamic feature channel in the main model.

[0073] The mechanism takes the dual-domain correlation matrix A as input and implements feature recalibration through the following processing path: Input A into the channel attention mapping network (Channel Attention Block) and output the channel weight vector ; Structural behavior characteristic tensor in the main model ; Apply channel-by-channel scaling: ; Updated feature tensor The original structural information is retained while enhancing the response capability to more sensitive channels under the current environmental conditions of the site.

[0074] In addition, this module can be optionally equipped with a historical offset feedback mechanism. When a certain plot of land shows significant model error drift over a continuous period of time (for example, the predicted trend continues to deviate from the measured value), the system can reversely adjust the gradient path of the weight generation process to dynamically suppress misleading signal interference.

[0075] This implementation provides a feedback prediction module for generating intelligent intervention recommendations based on deviations between current crop growth trends and standard growth patterns. This module leverages latent rate time series analysis, crop variety embedding vector representation, and adversarial intervention reasoning to construct a closed-loop control path from deviation detection to the output of executable agronomic instructions, enhancing the system's predictive decision-making capabilities and variety adaptability.

[0076] In the system of the present invention, the growth behavior modeling module outputs a sequence of latent rate parameters of crops in a continuous period. This sequence is indexed by time and records the structural growth rate, tension evolution or geometric expansion trend of the crop at each observation time point, denoted as Zt; At the same time, the system database stores a set of standard growth dynamics curves for multiple varieties, multiple plots, and multiple stages. These curves are obtained through historical real measurement data and modeling methods, expressing the growth trajectory under ideal conditions, denoted as St; This module uses a weighted displacement and local gradient deviation (WDLGD) dual-index modeling strategy to compare the two time series mentioned above and calculate their structural deviation magnitude and dynamic deviation curve.

[0077] Weighted Displacement Index (D1): reflects the degree of deviation from the overall trend and is defined as: ;in The importance weights of different time periods are usually increased according to key growth stages (such as jointing stage, heading stage, etc.). represents the implicit rate parameter sequence The value at the i-th time point in ; represents the value of the i-th time point in the set of standard growth kinetic curves, and m is the total number of time points; Micro-change difference index (D2): reflects the local fluctuation difference and is defined as: ; The two together constitute the deviation state representation vector , used to drive the input channel of the subsequent intervention reasoning network.

[0078] In addition, this module constructs a deviation dynamic curve image and converts the Zt−St difference map into a two-dimensional tensor expression through a temporal graph encoder to assist the generation task of the adversarial model.

[0079] Because different crop varieties vary widely in their genetic characteristics, growth cycles, and environmental response strategies, unified intervention strategies are often insufficiently applicable. Therefore, this implementation introduces a multidimensional Crop Variety Embedding Vector (CVEV) mechanism to provide a foundation for personalized modeling at the variety level.

[0080] The process of constructing the product embedding is as follows: Collect basic information on the following dimensions for each crop variety: Genetic trait coding (such as drought resistance coefficient, light and temperature sensitivity); Historical growth data (tension, internode length, etc. curves from multiple plots); Environmental response parameters (e.g., drought-sensitive period, high light stress window); A nested feature fusion is performed on the feature vectors of each dimension, and a three-layer fully connected network and normalization processing are used to obtain the standardized embedding vector Vp. This vector is used as the individualized feature condition input and is input into the intervention strategy reasoning network together with the deviation indicator vector.

[0081] The intervention network is an improved Adversarial Crop Advisory Network (ACAN), which consists of a generator Gθ and a discriminator Dϕ. The training objective is: Generator G: Inputs the bias vector + the variety embedding vector and outputs the intervention measure recommendation vector A=[a1,a2,...,ah], where h is the total number of vectors; such as irrigation adjustment value, shading duration, pesticide pre-spraying timing, etc. Discriminator D: determines whether the generated results conform to the expert standard recommendations or the historical real intervention path; Wasserstein-GAN or Gradient Penalty mechanism is used during training to improve stability. The final output vector A will be decoded into natural language suggestions or device control instructions.

[0082] After obtaining the intervention vector A, this module performs the following parsing: Intervention parameter decoding: Mapping numeric vectors into operation terms. For example: a1=0.8⇒It is recommended to increase the irrigation frequency to 80% of the current base; a2=30⇒It is recommended to extend the daily shading time to 30 minutes; a3=0.6⇒The recommended application time window is 0.6 days in advance.

[0083] Generate a list of recommended actions to take, such as: Table 1 Generate a list of recommended executable operations

[0084] Generate intervention command interface output, which can be sent to agricultural terminal control platforms or intelligent agricultural machinery systems, such as water and fertilizer integrated machines, sunshade net controllers, etc.

[0085] The system automatically records whether the intervention measures have achieved the target adjustment effect based on the tracking changes of subsequent growth trends. If deviation regression is not achieved, the system enters the next round of intervention optimization stage to continue learning and correcting strategies.

[0086] Example 2: To demonstrate the practicality and technical advantages of the system described in this invention in a real agricultural setting, this application deployed a complete crop growth analysis system based on this invention using actual farmland at a specific location as an example. The corn variety "Zhengdan 958" was selected as the target crop for the experiment. The experimental period was 60 days, covering the main growth stages (seedling emergence to tasseling) from day 10 to day 70 after sowing.

[0087] Image acquisition device: Install an integrated multispectral acquisition array, each set includes a visible light camera (RGB), a near-infrared camera (NIR) and a structured light depth camera; Environmental data collection equipment: Deploy temperature and humidity sensors, light intensity sensors, and soil conductivity sensors, with data collection frequencies of 10 minutes, 1 minute, and 1 hour, respectively; Sampling frequency: The image acquisition interval is 6 hours / time, and the environmental data is synchronized in real time; Total data volume: A total of approximately 720 sets of image samples and approximately 36,000 pieces of environmental data were collected.

[0088] After implementing the integrated ex situ acquisition architecture, the average structural similarity index (SSIM) of the RGB and NIR image fusion was improved from the original 0.61 to 0.87 after being processed by the SCR-Net spectral consistency assessment model; Compared with the control group (traditional single-channel visible light acquisition), the IOU in leaf surface contour recognition was improved by 19.4%.

[0089] The system uses the SIFT-HOAD joint descriptor to extract feature points. Compared with the traditional SIFT descriptor, the feature matching accuracy under wind interference conditions is improved from 78.6% to 92.1%; The convolutional residual comparison network achieved an F1-score of 0.91 for identifying wind-obstructed areas, an improvement of 26.7% compared to the traditional pixel threshold-based method.

[0090] Through the cross-period point cloud relocation algorithm, the daily inter-node change trend was successfully extracted; After building the TLRE latent rate encoding model, the system's RMSE for predicting growth trends in a 7-day forecast window was 0.032, significantly better than the unmodeled baseline (RMSE of 0.089); Multidimensional dynamic clustering is used to automatically identify growth acceleration, deceleration and stagnation periods, with an accuracy rate of 87%.

[0091] The system calculated the deviation between the current trend and the standard curve in real time and found that the experimental plots experienced growth delay due to water stress on the 43rd day; The intervention reasoning module outputs the suggestion of "increase irrigation frequency + reduce light exposure". After subsequent implementation, the latent variable rate value rebounded from the 46th day, and the trend deviation converged; Compared with the control plots that did not use this system, the average yield of the experimental group increased by 12.8%, of which the disease control rate increased by 18.5%.

[0092] As shown in the table, the control experiment is summarized: Table 2 Summary of control experiments

[0093] This embodiment is deployed and verified based on real agricultural scenarios. The results show that the system of the present invention can effectively obtain stable multi-modal images and multi-source environmental data in natural planting environments; the proposed disturbance recognition method can accurately identify wind disturbances and plant occlusion areas, reducing misidentification; the growth behavior modeling module has the ability to abstract implicit structural changes and adapt to different growth stages; the intervention prediction module can issue effective intervention suggestions in early growth deviations, and the feedback control closed loop is clear and has good practical executability. Ultimately, it achieves comprehensive beneficial effects such as improving trend modeling accuracy, enhancing environmental adaptability, and increasing crop yield and health levels.

[0094] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A crop growth analysis system based on machine vision, characterized by: include: Multi-source acquisition module, used to collect multi-mode image data under different spectra during the crop growth cycle, and automatically mark the mapping relationship between the acquisition time and meteorological data; The disturbance recognition module is used to perform multi-view sparse matching on multi-modal image data, construct a three-dimensional temporal structure map of the crop plants, and combine the reconstruction residual model to identify image distortion areas caused by wind disturbance, occlusion, or plant posture changes; The growth behavior modeling module is used to extract the internode elongation rate, leaf tension distribution, and bifurcation angle variation characteristics of crops based on a three-dimensional time series structure diagram, and to construct a cross-stage growth trend model using a latent rate coding mechanism; The environmental adaptability correction module is used to dynamically adjust the growth trend model based on the measured temperature and humidity, light intensity, and soil conditions of the target plot, and establish a crop-environment dual-domain mapping relationship; The feedback prediction module is used to output early warnings based on the degree of deviation between the current growth trend model and the standard growth dynamics curve, and to generate intervention recommendation signals based on the target crop varieties.

2. The machine vision-based crop growth analysis system according to claim 1, characterized in that: The multi-source acquisition module specifically includes: An out-of-situ viewing angle acquisition architecture is constructed using an integrated multispectral camera array. This architecture simultaneously captures visible light, near-infrared, and depth images during each sampling cycle. The main control system dynamically adjusts the exposure ratio and spectral channel weights to adapt to the varying reflectance of different crop leaves. At the time of collection, the temperature, light intensity, and wind speed parameters recorded by the local micrometeorological collection unit are bound through a high-precision timestamp mechanism to generate a joint index table of image sampling frames and environmental conditions; A spectral consistency calibration model is constructed, dynamic weighting is performed based on the difference in inter-pixel reflection intensity between the near-infrared channel and the visible light channel, and a spatial correlation regularization network is used to jointly encode multimodal images.

3. The machine vision-based crop growth analysis system according to claim 1, characterized in that: The disturbance identification module includes: A multi-baseline imaging strategy is used to perform low-overlap sparse matching of multi-view images. The geometric distribution of natural plant feature points in unstructured images is used to construct a cross-view matching graph. The multi-scale SIFT-HOAD joint descriptor is used to improve the robustness of leaf tip and stem node recognition. The structural alignment residual graph is constructed using the sparse point cloud reconstruction results and the known crop skeleton template. The Euclidean deviation and connectivity break index between key structural units are calculated, and a weighted residual energy function is introduced as a dynamic adjustment factor for the distortion sensitivity threshold. The residual map is spatially analyzed through a convolutional residual comparison network to automatically identify abnormal reconstruction areas caused by wind disturbance, uneven lighting, and plant tilt.

4. The machine vision-based crop growth analysis system according to claim 3, characterized in that: The disturbance identification module further includes: A perturbation graph structure network built based on the structural residual graph takes each 3D structural key point as a graph node. The node attributes include local geometric offset, brightness gradient direction and texture complexity index, and the graph edge relationship is defined based on the residual connection weight; An improved direction-aware graph convolutional network is used to iterate the graph structure, integrate spatially heterogeneous disturbance information, extract topological coherence features between regions, and dynamically mark regions with high probability of topological breaks as candidate anomaly regions. The candidate abnormal region is fused with the structural integrity score map, and the abnormal region is refined through a dual-path residual inversion mechanism.

5. The machine vision-based crop growth analysis system according to claim 1, characterized in that: The growth behavior modeling module includes: Based on the three-dimensional time series structure diagram, a cross-time point cloud repositioning algorithm was used to align the key nodes of each plant in time series. The internode length change, leaf tension relaxation coefficient, and bifurcation angle vector transfer rate between consecutive frames were calculated to generate the original growth feature tensor. The growth feature tensor is dimensionally mapped and reconstructed by an adjustable nonlinear nested kernel function to enhance the nonlinear interaction expression between features, and the stage transition boundary of the growth pattern is identified by multidimensional dynamic discriminant clustering; A time-oriented latent rate coding network is constructed to encode the original feature sequence into an implicit growth rate parameter sequence, and the model learning rate is dynamically adjusted through the differential information entropy control mechanism.

6. The machine vision-based crop growth analysis system according to claim 1, characterized in that: The environmental adaptability correction module includes: The multi-source time series fusion of the measured environmental factors of the target plot is performed to construct a multi-dimensional environmental state vector including temperature, humidity, light intensity and soil conductivity indicators; The environmental state vector is input into a coupled orthogonal feature transformation module, and its semantic layer change trend and numerical layer perturbation response are extracted respectively through a dual-channel transformation network, and a dual-domain correlation matrix between the crop growth feature domain and the environmental state domain is constructed; Based on the constructed dual-domain correlation matrix, a weight regulation submodule of the growth trend model is introduced, and the conditional weight mapping mechanism is used to recalibrate the dynamic feature channels in the model layer by layer.

7. The machine vision-based crop growth analysis system according to claim 1, characterized in that: The feedback prediction module includes: The latent rate series output by the current growth trend model is matched with the standard growth dynamics curve on a time-by-time basis, and a weighted displacement-micro-change difference dual-index modeling strategy is adopted to calculate the structural deviation value and dynamic offset curve; Construct a crop variety embedding vector library, generate multidimensional variety gene codes based on crop genetic traits and variety historical growth records, and input them into an adversarial intervention inference network to establish an optimal intervention path mapping model between the standard curve and abnormal deviations; Based on the current deviation value and the intervention path of the corresponding crop variety, an intervention recommendation signal is output, including adjusting water and fertilizer strategies, shading control duration or preventive pesticide application methods, and accompanied by timing execution recommendations.

Images