A corn whole growth period growth monitoring method based on fixed monocular vision

Abstract

The application discloses a corn whole growth period growth monitoring method based on fixed monocular vision, which comprises the following steps: an RGB image of a fixed area of a corn field is collected by a fixedly installed monocular camera and is transmitted to a remote server; an AI deep learning target detection network pre-trained on the server is used to identify corn plants in the image and determine the phenological stage, and a plant mask and the corresponding phenological stage are output; different corn growth index algorithms are designed for different phenological stages, and corn growth index values are calculated; and a single plant corn growth change curve is generated according to the calculated corn growth index values. The method has the advantages of low cost, high precision and full automation, and is suitable for precise corn management.

Description

Technical Field

[0001] This invention relates to the field of maize growth monitoring technology, specifically to a method for monitoring maize growth throughout its entire growth period based on fixed monocular vision. Background Technology

[0002] As an important food crop in my country, the dynamic monitoring of maize's growth throughout its entire growth cycle is crucial for precise field management, yield prediction, and variety selection. Traditional maize field growth monitoring mainly relies on manual measurement of plant height, leaf count, stem diameter, ear length, and ear diameter, which is labor-intensive, lacks timeliness, and makes it difficult to obtain continuous dynamic data. While multispectral / hyperspectral UAVs or satellite remote sensing can achieve large-scale monitoring, their spatial resolution is limited, failing to reach the individual plant level, and they are constrained by weather, cloud cover, and flight plans, making it difficult to achieve high-frequency daily observations.

[0003] This invention proposes a technical solution of "fixed monocular camera + remote server-side AI + similar triangle monocular height and distance measurement + multi-phenological stage and multi-indicator joint extraction (including ear length and ear diameter) + single plant level cross-day matching" to achieve low-cost, high-precision and continuous monitoring of tall maize plants throughout their entire growth period. Summary of the Invention

[0004] This application provides a method for monitoring maize growth throughout its entire growth period based on fixed monocular vision, in order to solve the above-mentioned technical problems existing in current maize field growth monitoring methods.

[0005] According to the first aspect, one embodiment provides a method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision, the method comprising: RGB images of a fixed area of ​​cornfields are captured by a fixed-installation monocular camera and transmitted to a remote server. The AI ​​deep learning object detection network pre-trained on the server is used to identify corn plants in the image and determine their phenological stage, and output the plant mask and its corresponding phenological stage. Differentiated maize growth index algorithms were designed for different phenological stages, and maize growth index values ​​were calculated. Based on the calculated maize growth index values, individual plant-level maize growth change curves were generated.

[0006] Furthermore, an AI deep learning object detection network pre-trained on the server is used to identify corn plants in the image and determine their phenological stage, outputting the plant mask and its corresponding phenological stage, specifically including: Professional annotation tools were used to perform polygon segmentation and annotation on the complete outline of each corn plant in the image, and a phenological stage category label was assigned to the corresponding plant, including seedling stage, growth stage, and ear-bearing stage. The labeled images are used to construct training and testing sets, and the YOLOv8-seg network model is trained and tested using the training and testing sets. By inputting the acquired images into the trained YOLOv8-seg network model, the model outputs the mask of each maize plant and its corresponding phenological stage category.

[0007] Furthermore, for different phenological stages, differentiated maize growth index algorithms were designed, and the maize growth index values ​​were calculated, specifically including: The phenological period includes the seedling stage, the growth stage, and the heading stage; a. For the seedling stage: Segment the leaf instances within the plant's covered area and count the number of leaves N; b. For the growth period: A monocular similar triangle localization algorithm is used to detect the ordinate of the plant root (y_root) and the ordinate of the top (y_top) in the image. The plant height H is calculated using the formula: H = H_cam·(y_root − y_top) / (y_root − cy), where H_cam is the vertical height from the optical center to the ground, and cy is the ordinate of the principal point. An ellipse is fitted within the root mask, and the stem thickness D = d_px·λ is calculated based on the pixel minor axis d_px and the pixel-world scale λ = H / (y_root −y_top). c. Regarding the heading stage: Detect corn ears within the plant cover and count the number of ears M. Based on the monocular similar triangle positioning algorithm, the ear height is calculated as: H_ear = H_cam·(y_root − y_ear) / (y_root − cy), where y_ear is the ordinate of the top of the ear; Ear length estimation: Within the ear mask area, extract the pixel length L_px along the main axis of the ear, and combine it with the scale λ_ear = H_ear / (y_root − y_ear) to calculate the true ear length L = L_px·λ_ear; Ear thickness estimation: Extract the pixel diameter W_px at the maximum width of the ear mask, and calculate the true ear thickness W = W_px·λ_ear.

[0008] Furthermore, leaf instance segmentation is performed within the plant masking area, specifically including: A lightweight U-Net network is used for instance segmentation of the blades.

[0009] Furthermore, the detection of corn ears within the plant cover specifically includes: An improved YOLOv8-nano network was obtained by adding a P2 small target layer and embedding a CBAM attention module on the basis of YOLOv8-nano. The improved YOLOv8-nano network was then used for corn ear detection.

[0010] Furthermore, the pixel length L_px is extracted along the main axis of the ear of grain, specifically including: The principal component analysis (PCA) method was used to obtain the pixel length L_px of the spikelet principal axis.

[0011] Furthermore, the pixel diameter W_px is extracted at the maximum width of the ear mask, specifically including: Calculate the minimum bounding rectangle of the mask outline; the shorter side is W_px.

[0012] Furthermore, based on the calculated maize growth index values, a single-plant-level maize growth change curve is generated, specifically including: For the current day's image and the previous day's image, perform single-plant level matching based on the IoU of the plant mask and the spatial position offset threshold Δp; For successfully matched plants, calculate the changes in maize growth: Δphenological stage, ΔN, ΔH, ΔD, ΔM, ΔH_ear, ΔL, ΔW; where Δphenological stage: change in phenological stage; ΔN: change in leaf height; ΔH: change in plant height; ΔD: change in stem diameter; ΔM: change in ear number; ΔH_ear: change in ear height; ΔL: change in ear length; ΔW: change in ear diameter. The changes in corn growth were written into the database, and the corn growth curve was updated.

[0013] Furthermore, for the current day's image and the previous day's image, single-plant-level matching is performed based on the IoU of the plant mask and the spatial position offset threshold, specifically including: Calculate the Euclidean distance d between the center points of the plants on the two days before and after; Calculate the masking overlap (IoU) of the plants on the two days before and after; If the IoU is greater than the preset threshold and d is less than the spatial position offset threshold, they are determined to be the same plant. The spatial position offset threshold is determined by calibration test. The Hungarian algorithm or greedy matching is used to handle one-to-many conflicts, ensuring the uniqueness of the match.

[0014] Furthermore, for successfully matched plants, the changes in maize growth vigor are calculated: Δphenological stage, ΔN, ΔH, ΔD, ΔM, ΔH_ear, ΔL, and ΔW, specifically including: ΔPhenological period = Current phenological period code minus the previous day's phenological period code; Phenological periods are represented by numerical codes: seedling stage code is 1, growing season code is 2, and heading stage code is 3; ΔN = the number of leaves on the current day N_t minus the number of leaves on the previous day N_{t-1}; where N is the number of leaves counted during the seedling stage, reflecting the speed of leaf differentiation in seedling plants. The larger ΔN is, the more vigorous the leaf growth. ΔH = plant height H_t on the current day minus plant height H_{t-1} on the previous day, reflects the daily growth rate of the plant during the growing season and is an important basis for judging the plant's growth vitality; ΔD = stem diameter D_t on the current day minus stem diameter D_{t-1} on the previous day, reflects the rate at which the plant stem thickens during the growing season and is closely related to the plant's resistance to lodging; ΔM = the number of ears on the current day M_t minus the number of ears on the previous day M_{t-1}, reflecting the occurrence of ears during the heading stage; ΔH_ear = the ear height of the day H_ear_t minus the ear height of the previous day H_ear_{t-1}, the index reflects the change in the position of the ear on the stem; ΔL = the length of the ear on the current day L_t minus the length of the ear on the previous day L_{t-1}, which reflects the elongation rate of the ear during the grain-filling and milk-ripe stages. A positive value indicates that the ear is still growing, while a value close to 0 indicates that the ear length is basically fixed. ΔW = the diameter of the ear on the current day W_t minus the diameter of the ear on the previous day W_{t-1}, reflects the rate of ear thickening during the grain-filling period and is closely related to the degree of grain filling.

[0015] This application provides a method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision, which has the following beneficial effects: 1) Low cost: The single-lens camera costs around 100 yuan, the terminal does not require a GPU, and the power consumption of the whole device is less than 2W; 2) High precision: Based on the principle of similar triangles, the plant height error is ≤3%, the stem diameter error is ≤4%, the ear length error is ≤5%, and the ear diameter error is ≤5%. 3) Full growth period: Covering the seedling stage, growth stage, and heading stage, with abundant indicators, including increased ear length and diameter; 4) Single plant level: It can track the growth evolution of any single corn plant, providing data for precision fertilization, irrigation, and yield estimation; 5) Simple deployment: Only one H_cam measurement is required, without the need for vanishing point calibration or ground references; 6) Automation: Daily scheduled data collection and remote server-side AI processing require no manual intervention; 7) Scalable: The remote server microservice architecture supports online horizontal scaling. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision, as provided in one embodiment of the present invention; Figure 2 The hardware architecture diagram of a method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision is provided in one embodiment of the present invention. Figure 3 This is a schematic diagram of the field camera installation in a method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision, provided as an embodiment of the present invention (including the labeling of the optical center-ground height H). Figure 4 The phenological period determination network structure diagram (YOLOv8-seg) in a maize growth monitoring method based on fixed monocular vision provided in an embodiment of the present invention. Figure 5 This is a geometric model diagram of monocular similar triangle height measurement in a method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision, provided in one embodiment of the present invention. Figure 6 A geometric model diagram of monocular estimation of ear length and ear diameter in a maize growth monitoring method based on fixed monocular vision provided in an embodiment of the present invention; Figure 7 This is an example diagram of single-plant matching and growth curve in a maize growth monitoring method based on fixed monocular vision provided in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0018] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0019] The first embodiment of this invention provides a method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision. This method utilizes a fixedly installed monocular camera to continuously acquire images of the maize field, and a server-side AI algorithm deployed on a remote server automatically extracts growth indicators and tracks individual plants throughout the entire growth period—from seedling stage to tasseling stage. The overall flowchart is shown below. Figure 1 As shown. The hardware architectures that support this method are as follows. Figure 2 As shown, it specifically includes: The edge acquisition device is only used for periodic shooting and uploading of raw RGB images; the edge acquisition device does not contain a GPU and does not run any AI inference models locally. The remote server is used to receive raw images and perform phenological stage determination, growth index extraction, single plant matching, and time series storage; the remote server deploys server-side AI inference microservices via Docker containerization, supporting online horizontal scaling. A communication network is used for encrypted data transmission between the edge acquisition device and the remote server; the communication network adopts 4G LTE Cat-1, NB-IoT or 5G RedCap; The visualization and interface module, deployed on a remote server, is used to provide growth curve query and alarm services.

[0020] Hardware parameters: Camera: 5 MP industrial camera, fixed focus 4 mm, installation height 2.8 m, downward viewing angle 30°; Edge Computing: Raspberry Pi Zero 2W + 64GB TF Card + 20W Solar Panel + 3 18650 Batteries, daily power consumption 1.8 Wh, can work continuously for 5 days in cloudy or rainy weather; Communication: 4G LTE Cat-1 module, MQTT over TLS, peak upload rate 10 Mbps; Remote server: Alibaba Cloud gn6i instance (NVIDIA T4 GPU), Ubuntu 20.04, Docker + Kubernetes, InfluxDB 2.x, Grafana 9.x.

[0021] The following is combined with Figure 1 The first embodiment of the present invention provides a detailed description of a method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision.

[0022] like Figure 1 As shown, in step S100, RGB images of a fixed area of ​​cornfield are captured by a fixedly installed monocular camera and transmitted to a remote server.

[0023] In this embodiment, as Figure 3 As shown, RGB images of a fixed area of ​​cornfields are captured daily at time T using a fixed-installation monocular camera.

[0024] like Figure 1 As shown, in step S200, the AI ​​deep learning object detection network pre-trained on the server is used to identify corn plants in the image and determine their phenological stages, and output the plant mask and its corresponding phenological stage.

[0025] In this embodiment, a trained server-side AI deep learning object detection network is used to segment each corn plant in the image to obtain plant masks, and the phenological stage to which it belongs is determined based on the morphological features and color and texture features of the masks: seedling stage, growth stage or heading stage.

[0026] In this embodiment, phenological period determination is performed using YOLOv8-seg, such as... Figure 4 As shown, the input is 640×640, the output category is head_stage∈{seedling, growing, earring}, the training set has 5200 field images, and mAP@0.5 = 0.93.

[0027] Regarding "phenological period determination," this embodiment does not simply rely on the deep learning network to directly output classification results, but instead adopts a two-level determination mechanism of "deep learning as the main criterion + mask feature as the auxiliary criterion": 1) First, the corn plants in the image are detected and segmented using the trained instance segmentation network such as YOLOv8-seg. The network also outputs the preliminary phenological stage category and confidence score. 2) Secondly, for plants with low confidence or in the phenological transition stage, the system further extracts the morphological features of the plant mask (such as aspect ratio, contour complexity, and ear protrusion detection) and color and texture features (such as HSV color distribution and GLCM texture parameters), and performs fine-grained verification and correction through preset rules or lightweight classifiers. 3) Finally, the network output and feature analysis results are fused to form the final phenological period determination.

[0028] In this embodiment, the training of the phenological period determination model adopts single-plant-level instance segmentation and annotation, and the specific annotation method is as follows: Labeled objects: Each corn plant in each image.

[0029] Labeling content: Perform polygon segmentation labeling (masking) on ​​the complete outline of each corn plant, and assign a phenological stage category label to the plant.

[0030] Category tags: seedling, growing, and earring, corresponding to numerical codes 0, 1, and 2 respectively.

[0031] Number of annotations: The training set contains 5,200 field images, with a total of approximately 15,600 corn plants annotated (an average of 3 plants per image).

[0032] Labeling tools: Professional labeling tools such as X-AnyLabeling or LabelMe are used, and the labeling is completed by trained agricultural technicians. After labeling, cross-validation is performed to ensure that the accuracy rate is not less than 98%.

[0033] In this way, the model learns during training to output a mask for a given image region of a plant and its corresponding phenological stage category. Therefore, the model can simultaneously detect multiple corn plants in a single image and output a phenological stage label for each plant.

[0034] Data caching: The original image, mask, and shooting timestamp of each corn plant are associated and uploaded to a remote server and written to a time-series database. like Figure 1 As shown, in step S300, a differentiated maize growth index algorithm is designed for different phenological stages, and the maize growth index value is calculated. Based on the calculated maize growth index value, a single-plant-level maize growth change curve is generated.

[0035] The above steps specifically include: S310, Growth Index Calculation: The corresponding algorithm is executed on the remote server based on the phenological stage. a) Seedling stage: Segment the leaf instances within the plant's covered area and count the number of leaves N.

[0036] In this embodiment, blade segmentation uses lightweight U-Net with IoU = 0.87.

[0037] b) Growth period: like Figure 5 As shown, a monocular similar triangle localization algorithm is used to detect the ordinate y_root of the plant root and the ordinate y_top of the top in the image; The plant height H is calculated using the formula: H = H_cam·(y_root − y_top) / (y_root − cy), where H_cam is the vertical height from the optical center to the ground, and cy is the ordinate of the principal point. An ellipse is fitted within the root mask, and the stem thickness D = d_px·λ is calculated using the pixel minor axis d_px combined with the pixel-world scale λ = H / (y_root− y_top).

[0038] c) Ear-bearing period: Detect corn ears within the plant cover and count the number of ears M. The ear height H_ear = H_cam·(y_root − y_ear) / (y_root − cy) is calculated using the same monocular similar triangle positioning algorithm as the plant height, where y_ear is the ordinate of the top of the ear; like Figure 6 As shown, the ear length is estimated as follows: within the ear mask area, the pixel length L_px is extracted along the main axis of the ear, and the true ear length L = L_px·λ_ear is calculated by combining the scale λ_ear = H_ear / (y_root − y_ear). like Figure 6 As shown, ear thickness estimation: extract the pixel diameter W_px at the maximum width of the ear mask, and calculate the true ear thickness W = W_px·λ_ear.

[0039] In this embodiment, ear detection: A P2 small target layer is added to YOLOv8-nano and a CBAM attention module is embedded. The training set consists of 5000 maize ear-bearing images. Data augmentation includes random lighting and occlusion. After training, mAP@0.5 = 0.91, and the inference time per image is ≤ 30 ms (T4 GPU).

[0040] The specific implementation method for improving the ear detection model is as follows: 1. Improvement Goals and Background Corn ears are typically small targets in images (with a small pixel size, approximately 50×150 pixels) and are easily affected by factors such as leaf occlusion and changes in lighting. In the original YOLOv8-nano network structure, the smallest detection layer is layer P3 (downsampling factor of 8), which has limited detection capability for small targets like ears. Therefore, this embodiment introduces a higher resolution layer P2 (downsampling factor of 4) in the backbone to enhance the feature extraction capability for small targets.

[0041] 2. Location and connection method for adding P2 small target layers Layer P2 is derived from the shallow feature map of the backbone, and is added in the following way: Location of introduction: After the second C2f module in Backbone (i.e. the layer before the original network used to generate the P3 layer), the feature map size of this layer is 1 / 4 of the input image (if the input is 640×640, then the P2 feature map size is 160×160), containing rich spatial detail information.

[0042] Feature extraction: After extraction, the P2 features are further processed by an additional C2f module to enhance their semantic expressiveness while maintaining high spatial resolution.

[0043] Feature fusion: The processed P2 features are downsampled to the size of the P3 layer (1 / 8) through a downsampling operation (such as convolution with a stride of 2), and then concatenated with the original P3 features to achieve the fusion of shallow detailed information and mid-level semantic information.

[0044] Multi-scale detection head: A top-down path is added to the Neck section to input the fused P2 features along with the original P3, P4, and P5 features into the detection head, forming a four-scale detection structure of P2, P3, P4, and P5, which are responsible for the detection of targets of different sizes.

[0045] 3. Embedding location and connection method of CBAM attention module CBAM (Convolutional Block Attention Module) is used to enhance the representation of key information in feature maps in both channel and spatial dimensions. The specific embedding method is as follows: Embedding location: Before the feature map of each detection layer (P2, P3, P4, P5) is input into the Detect layer, a CBAM module is embedded respectively; Connection method: Taking layer P2 as an example, after feature map fusion in the Neck layer, it first goes through a convolutional layer for channel adjustment, and then is input into the CBAM module. The CBAM module performs channel attention weighting and spatial attention weighting in sequence, outputting an enhanced feature map. This feature map is then fed into the Detect layer for target classification and localization prediction.

[0046] Mechanism of action: The channel attention module adaptively weights each channel of the feature map to highlight the semantic features related to the ear; the spatial attention module weights the spatial position of the feature map to strengthen the response of the region where the ear is located and suppress the interference of background and leaf occlusion.

[0047] 4. Network Structure Diagram For ease of understanding, the improved network structure can be summarized as follows: Backbone: CSPDarknet, outputting feature maps at four levels: P2 (1 / 4), P3 (1 / 8), P4 (1 / 16), and P5 (1 / 32); Neck: PANet structure, adding processing and fusion paths for P2 features, forming a two-way feature fusion from top to bottom and bottom to top; Attention: Embed a CBAM module after the output feature map of each detection layer; Head: Four-scale detection heads (P2, P3, P4, P5), responsible for detecting small, medium, and large targets respectively.

[0048] 5. Training and Reasoning Training data: 5000 images of maize at the ear-bearing stage, with bounding boxes labeled on the ears; data augmentation included random lighting adjustment, random occlusion, and random rotation. Inference performance: On T4 GPU, single-image inference time is ≤ 30 ms, and mAP@0.5 reaches 0.91.

[0049] In summary, the "adding a P2 small target layer on top of YOLOv8-nano" in this embodiment specifically refers to: extracting a feature map (P2) with a resolution of 1 / 4 of the input image from the second C2f module of the backbone; after processing by the C2f module, it is downsampled and concatenated with the P3 feature to form a four-scale detection structure containing P2, P3, P4, and P5. The "embedding of the CBAM attention module" specifically refers to: sequentially embedding the CBAM module before inputting the feature maps of each scale (P2, P3, P4, P5) into the Detect layer. This module performs channel attention weighting and spatial attention weighting in sequence to enhance the model's feature representation of the ear of fruit.

[0050] In this embodiment, the principal axis of the ear mask is extracted by using PCA principal component analysis to obtain the pixel length L_px of the ear principal axis.

[0051] The specific process for extracting the pixel length of the main axis of the ear using principal component analysis is as follows: 1. Input data Within the ear detection bounding box output by the ear detection model (YOLOv8-nano + P2 layer + CBAM), a pixel-level mask for the ear is obtained through instance segmentation. This mask is a binary image, where a pixel value of 1 represents the ear region and 0 represents the background. The image coordinates (x, y) of all pixels in the mask are extracted to form a point set S = {(x_i, y_i) | i = 1,2, ..., n}, where n is the total number of pixels contained in the ear mask.

[0052] 2. Principal component analysis calculates the principal axis direction. The specific steps for performing principal component analysis on the point set S are as follows: Calculate the mean coordinates (x_mean, y_mean) of the point set, which is the average of the coordinates of all pixels; Construct a covariance matrix C with a dimension of 2×2, and calculate the deviation of each pixel relative to the mean and the correlation between the deviations. Find the eigenvalues ​​and eigenvectors of the covariance matrix, where the eigenvalues ​​λ1≥λ2>0 and the corresponding eigenvectors v1 and v2 are orthogonal to each other. Take the eigenvector v1 corresponding to the largest eigenvalue λ1 as the main axis direction of the ear, which is the length direction of the ear; v2 is the secondary direction perpendicular to the main axis, corresponding to the width direction of the ear.

[0053] 3. Calculate the principal axis pixel length Project all pixels in the point set S onto the principal axis direction v1. The projection value is calculated as follows: for each pixel (x_i, y_i), calculate the vector between it and the mean point, and then perform a dot product with the principal axis unit vector to obtain the projection value t_i. The difference between the maximum value t_max and the minimum value t_min of the projection is the pixel length of the ear of fruit projected on the main axis, expressed by the formula: L_px = t_max t_min; L_px is the pixel length along the main axis of the ear of fruit, corresponding to the actual length of the ear of fruit (to be converted to scale).

[0054] 4. Supplementary Explanation If the ear of corn is curved in the image, the direction of the principal axis reflects the overall orientation of the ear, and the projected length is approximately the "unfolded length" of the ear. For severely curved ears, it is possible to consider performing PCA analysis on segments of the ear and then summing the results. However, in this embodiment, the degree of curvature of the corn ear is limited, and the overall PCA method already meets the accuracy requirements.

[0055] The advantages of this method are: it does not require manual specification of the ear endpoints, automatically adapts to the tilt posture of the ear, the calculation process is stable, and it is less affected by changes in ear shape.

[0056] In this embodiment, the maximum width of the ear mask is extracted by calculating the minimum bounding rectangle of the mask outline, and the shorter side is W_px.

[0057] In this embodiment, the on-site calibration of monocular height measurement, diameter measurement, ear length measurement, and ear diameter measurement is as follows: 1) The vertical distance H_cam from the optical center to the ground was measured with a steel tape measure and found to be 2.75 m (error ±1 cm). 2) The camera intrinsic parameters were obtained using Zhang's calibration method: f = 2048 px, cx = 640, cy = 360, and the distortion is negligible; 3) On-site measurement of 50 corn plants: The average height measured manually was 201.4 cm, the average height measured systematically was 200.7 cm, and the RMSE was 2.1 cm. The average stem diameter measured manually was 25.2 mm, and the average stem diameter measured systematically was 25.5 mm, with an RMSE of 1.3 mm. The average length of the spikelet measured manually was 18.7 cm, and the average length measured systematically was 18.5 cm, with an RMSE of 0.9 cm. The average spikelet diameter measured manually was 45.3 mm, and the average spikelet diameter measured systematically was 45.0 mm, with an RMSE of 1.1 mm.

[0058] S320, calculation of single-plant matching and growth change, details are as follows: S321, perform single-plant level matching between the current day's image and the previous day's image based on the IoU of the plant mask and the spatial position offset threshold Δp.

[0059] In this embodiment, single-plant matching is the core technical step for achieving continuous growth tracking, and its parameters are obtained as follows: 1. Calculation of plant masking IoU The mask is derived from the output of the instance segmentation network in the phenological stage determination step, and is a binary image (pixel value 1 represents the plant area, and 0 represents the background).

[0060] For the plant masks in the images from the previous two days, calculate the IoU pairwise, which is defined as the intersection area of ​​the two masks divided by the union area.

[0061] The IoU value is obtained directly through pixel-level comparison of the mask, without the need for additional calibration.

[0062] 2. Obtaining the spatial position offset threshold Δp Δp was determined through prior calibration experiments, taking into account the following factors: a) Camera stability: The fixed camera experiences slight positional changes during daily shooting. The maximum pixel shift is measured through calibration experiments. b. Plant growth displacement: Slight tilting of the corn stalks during growth leads to changes in the position of the top projection. Statistical analysis of images of 50 corn plants over 7 consecutive days showed a mean pixel offset of 12 pixels and a standard deviation of 6 pixels. c. Plant spacing constraint: Estimate the minimum pixel spacing between adjacent plants based on the image resolution and the average plant spacing in the field (approximately 30 cm), and ensure that Δp is less than this value to avoid mismatches.

[0063] Based on the above analysis, we take "mean plus three standard deviations" as Δp, that is, Δp = 12 pixels plus (3 multiplied by 6 pixels) equals 30 pixels. This value shows the best balance between matching accuracy and recall in the experiment.

[0064] 3. Matching decision-making process: Calculate the Euclidean distance d between the center points of the plants on the two days before and after; Calculate the mask overlap (IoU); If the IoU is greater than 0.5 and d is less than 30 pixels, then they are determined to be the same plant. The Hungarian algorithm or greedy matching is used to handle one-to-many conflicts, ensuring the uniqueness of the match.

[0065] S322, for successfully matched plants, calculate the changes in maize growth: Δphenological period, ΔN, ΔH, ΔD, ΔM, ΔH_ear, ΔL, ΔW.

[0066] The calculation methods for each indicator of corn growth change are as follows: 1. Phenological periods (phenological changes) Phenological stages are represented by numerical codes: seedling stage is coded as 1, growing stage as 2, and heading stage as 3. The Δphenological stage is calculated by subtracting the previous day's phenological stage code from the current day's phenological stage code.

[0067] Calculation Example: If the plant was in the seedling stage the previous day (code 1) and in the growing stage on the current day (code 2), then Δphenological period = 21 = 1, indicating that the phenological period has advanced by one stage. If the phenological period has not changed, then Δphenological period = 0. If the plant has entered the heading stage from the growing stage, then Δphenological period = 1. Under this coding rule, the value of Δphenological period is only 0 or 1, which conforms to the irreversible biological law of maize phenological periods.

[0068] 2. ΔN (Change in the number of leaves) ΔN = Number of leaves on the current day N_t minus the number of leaves on the previous day N_{t-1}.

[0069] Where N represents the number of leaves counted during the seedling stage. This indicator reflects the rate of leaf differentiation in seedling plants; the larger the ΔN, the more vigorous the leaf growth.

[0070] 3. ΔH (change in plant height) ΔH = Plant height H_t on the current day minus plant height H_{t-1} on the previous day, in centimeters.

[0071] This indicator reflects the daily growth rate of plants during their growth period and is an important basis for judging the growth vitality of plants.

[0072] 4. ΔD (Change in stem diameter) ΔD = Daily stem diameter D_t minus the previous day's stem diameter D_{t-1}, in millimeters.

[0073] This indicator reflects the rate at which the plant's stem thickens during the growing season and is closely related to the plant's resistance to lodging.

[0074] 5. ΔM (Change in ear number) ΔM = Number of ears of grain on the current day M_t minus Number of ears of grain on the previous day M_{t-1}.

[0075] This indicator reflects the occurrence of ears during the tasseling stage. Since the number of ears per corn plant is usually no more than 2, and ears do not disappear once they appear, ΔM is set to 0 or 1.

[0076] 6. ΔH_ear (change in ear height) ΔH_ear = the ear height H_ear_t of the day minus the ear height H_ear_{t-1} of the previous day, in centimeters.

[0077] This indicator reflects the change in the position of the ear of fruit on the stem. Since the node position of the ear is fixed once the ear is formed, the ear height should remain stable after the ear appears, and ΔH_ear usually tends to be close to 0; if a positive value appears, it may indicate that the ear has not yet fully stabilized or that there is a measurement error.

[0078] 7. ΔL (change in ear length) ΔL = the length of the ear on the current day L_t minus the length of the ear on the previous day L_{t-1}, in centimeters.

[0079] This indicator reflects the elongation rate of the ear during the grain-filling and milk-ripe stages. A positive value indicates that the ear is still growing, while a value close to 0 indicates that the ear length is basically fixed.

[0080] 8. ΔW (Change in ear diameter) ΔW = the diameter of the ear on the current day W_t minus the diameter of the ear on the previous day W_{t-1}, in millimeters.

[0081] This indicator reflects the rate of ear thickening during the grain-filling period and is closely related to the degree of grain filling.

[0082] S323: Write the changes in corn growth into the database and update the corn growth curve.

[0083] Example of single-plant matching and growth curve: Figure 7 As shown, the curve illustrates the changes in maize plant height from 0 to 110 days with the growth period / tasseling period.

[0084] Other relevant parameter settings in this embodiment: 1) Time series database: Using InfluxDB 2.x, measurement = maize_status, tags = {plot_id, plant_id}, fields = {stage, N, H, D, M, H_ear, L, W, temperature, humidity}; Data retention strategy: 2 years for original images, 5 years for indicator data.

[0085] The meanings of the fields are explained below: 1. Measurement (name of measurement instrument) maize_status: This table is used to store data on the growth status of corn plants.

[0086] 2. Tags (used for quick searching and grouping) plot_id: The plot number is used to identify the experimental plot or field area to which the corn plant belongs.

[0087] plant_id: Plant ID, used to uniquely identify a single corn plant.

[0088] 3. Fields (stores the actual measurement data) stage: phenological stage, which can be seedling stage, growth stage or heading stage (corresponding to codes 1, 2, 3).

[0089] N: Number of leaves (seedling stage), unit is leaves.

[0090] H: Plant height (during growth period), in centimeters.

[0091] D: Stem diameter (during growth period), in millimeters.

[0092] M: Number of ears (ear-bearing period), unit is individual ears.

[0093] H_ear: Ear height (earing stage), in centimeters.

[0094] L: Ear length (ear-bearing period), in centimeters.

[0095] W: Ear diameter (at the heading stage), in millimeters.

[0096] Temperature: Ambient temperature, in degrees Celsius, is collected synchronously by edge sensors.

[0097] humidity: Ambient humidity, in percentage, is collected synchronously by edge sensors.

[0098] This data structure design supports efficient querying by plot and by individual plant, facilitating the subsequent generation of individual plant growth curves and population statistical analysis.

[0099] 2) End-to-Server Communication and Security: Communication protocol: HTTPS POST, JSON payload; Local caching during network outages: JPEG images from the last 7 days are stored in a loop at the edge, and automatically resumed when the network is restored; Security: TLS 1.3, JWT authentication, Bucket-level ACL.

[0100] 3) Growth visualization Grafana Dashboard: Line graph: ΔH as a function of DOY; Line graph: ΔL as a function of DOY; Line graph: ΔW as a function of DOY; Heat map: Average ear length in different plots; Alarm: When ΔL < 0.2 cm / day for 3 consecutive days, the "ear growth stagnation" alarm is triggered.

[0101] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A method for monitoring the growth of maize throughout its entire growth period based on fixed monocular vision, characterized in that, The method includes: RGB images of a fixed area of ​​cornfields are captured by a fixed-installation monocular camera and transmitted to a remote server. The AI ​​deep learning object detection network pre-trained on the server is used to identify corn plants in the image and determine their phenological stage, and output the plant mask and its corresponding phenological stage. Differentiated maize growth index algorithms were designed for different phenological stages, and maize growth index values ​​were calculated. Based on the calculated maize growth index values, individual plant-level maize growth change curves were generated.

2. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 1, characterized in that, The system utilizes a pre-trained AI deep learning object detection network on the server to identify corn plants in images and determine their phenological stages, outputting plant masks and their respective phenological stages, specifically including: Professional annotation tools were used to perform polygon segmentation and annotation on the complete outline of each corn plant in the image, and a phenological stage category label was assigned to the corresponding plant, including seedling stage, growth stage, and ear-bearing stage. The labeled images are used to construct training and testing sets, and the YOLOv8-seg network model is trained and tested using the training and testing sets. By inputting the acquired images into the trained YOLOv8-seg network model, the model outputs the mask of each maize plant and its corresponding phenological stage category.

3. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 1, characterized in that, Differentiated maize growth index algorithms were designed for different phenological stages, and the maize growth index values ​​were calculated. Specifically, this included: The phenological period includes the seedling stage, the growth stage, and the heading stage; a. For the seedling stage: Segment the leaf instances within the plant's covered area and count the number of leaves N; b. For the growth period: A monocular similar triangle localization algorithm is used to detect the ordinate of the plant root (y_root) and the ordinate of the top (y_top) in the image. The plant height H is calculated using the formula: H = H_cam·(y_root − y_top) / (y_root − cy), where H_cam is the vertical height from the optical center to the ground, and cy is the ordinate of the principal point. An ellipse is fitted within the root mask, and the stem thickness D = d_px·λ is calculated based on the pixel minor axis d_px and the pixel-world scale λ = H / (y_root − y_top). c. Regarding the heading stage: Detect corn ears within the plant cover and count the number of ears M. Based on the monocular similar triangle positioning algorithm, the ear height is calculated as: H_ear = H_cam·(y_root − y_ear) / (y_root − cy), where y_ear is the ordinate of the top of the ear; Ear length estimation: Within the ear mask area, extract the pixel length L_px along the main axis of the ear, and combine it with the scale λ_ear =H_ear / (y_root − y_ear) to calculate the true ear length L = L_px·λ_ear; Ear thickness estimation: Extract the pixel diameter W_px at the maximum width of the ear mask, and calculate the true ear thickness W = W_px·λ_ear.

4. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 3, characterized in that, Leaf instance segmentation is performed within the plant masking area, specifically including: A lightweight U-Net network is used for instance segmentation of the blades.

5. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 3, characterized in that, Detecting corn ears within a plant cover specifically includes: An improved YOLOv8-nano network was obtained by adding a P2 small target layer and embedding a CBAM attention module on the basis of YOLOv8-nano. The improved YOLOv8-nano network was then used for corn ear detection.

6. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 3, characterized in that, Extracting the pixel length L_px along the main axis of the ear, specifically including: The principal component analysis (PCA) method was used to obtain the pixel length L_px of the spikelet principal axis.

7. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 3, characterized in that, Extracting the pixel diameter W_px at the maximum width of the mask specifically includes: Calculate the minimum bounding rectangle of the mask outline; the shorter side is W_px.

8. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 1, characterized in that, Based on the calculated maize growth index values, a single-plant-level maize growth change curve is generated, specifically including: For the current day's image and the previous day's image, perform single-plant level matching based on the IoU of the plant mask and the spatial position offset threshold Δp; For successfully matched plants, calculate the changes in maize growth: Δphenological stage, ΔN, ΔH, ΔD, ΔM, ΔH_ear, ΔL, ΔW; where Δphenological stage: change in phenological stage; ΔN: change in leaf height; ΔH: change in plant height; ΔD: change in stem diameter; ΔM: change in ear number; ΔH_ear: change in ear height; ΔL: change in ear length; ΔW: change in ear diameter. The changes in corn growth were written into the database, and the corn growth curve was updated.

9. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 8, characterized in that, For the current day's image and the previous day's image, single-plant-level matching is performed based on the IoU of the plant mask and the spatial position offset threshold, specifically including: Calculate the Euclidean distance d between the center points of the plants on the two days before and after; Calculate the masking overlap (IoU) of the plants on the two days before and after; If the IoU is greater than the preset threshold and d is less than the spatial position offset threshold, they are determined to be the same plant. The spatial position offset threshold is determined by calibration test. The Hungarian algorithm or greedy matching is used to handle one-to-many conflicts, ensuring the uniqueness of the match.

10. The method for monitoring maize growth throughout its entire growth period based on fixed monocular vision as described in claim 8, characterized in that, For successfully matched plants, calculate the changes in maize growth vigor: Δphenological stage, ΔN, ΔH, ΔD, ΔM, ΔH_ear, ΔL, and ΔW, specifically including: ΔPhenological period = Current phenological period code minus the previous day's phenological period code; Phenological periods are represented by numerical codes: seedling stage code is 1, growing season code is 2, and heading stage code is 3; ΔN = the number of leaves on the current day N_t minus the number of leaves on the previous day N_{t-1}; where N is the number of leaves counted during the seedling stage, reflecting the speed of leaf differentiation in seedling plants. The larger ΔN is, the more vigorous the leaf growth. ΔH = plant height H_t on the current day minus plant height H_{t-1} on the previous day, reflects the daily growth rate of the plant during the growing season and is an important basis for judging the plant's growth vitality; ΔD = stem diameter D_t on the current day minus stem diameter D_{t-1} on the previous day, reflects the rate at which the plant stem thickens during the growing season and is closely related to the plant's resistance to lodging; ΔM = the number of ears on the current day M_t minus the number of ears on the previous day M_{t-1}, reflecting the occurrence of ears during the heading stage; ΔH_ear = the ear height of the day H_ear_t minus the ear height of the previous day H_ear_{t-1}, the index reflects the change in the position of the ear on the stem; ΔL = the length of the ear on the current day L_t minus the length of the ear on the previous day L_{t-1}, which reflects the elongation rate of the ear during the grain-filling and milk-ripe stages. A positive value indicates that the ear is still growing, while a value close to 0 indicates that the ear length is basically fixed. ΔW = the diameter of the ear on the current day W_t minus the diameter of the ear on the previous day W_{t-1}, reflects the rate of ear thickening during the grain-filling period and is closely related to the degree of grain filling.

Images