Field crop early-stage identification monitoring method based on three-section type ensemble learning

Abstract

The invention discloses a field crop early-stage identification monitoring method based on three-section type integrated learning, and the method comprises the steps: obtaining a field high-angle image, carrying out the preprocessing after the judgment of the completeness, and obtaining a crop foreground image based on the preprocessed field high-angle image; obtaining the crop area and the crop amount at the corresponding time based on the crop foreground image obtained after the crop is transplanted; calculating a color space histogram of the crop foreground image according to a time sequence, judging whether the crop amount reaches a separable threshold value or not based on the color space histogram, and if yes, dividing the crop foreground image into early-stage pictures; obtaining a first-level crop category based on the early-stage picture, and performing associative storage with the early-stage picture; and inputting the early pictures into a secondary crop classification model to obtain a crop category judgment result and a growth maturity prediction result. According to the invention, efficient and accurate crop identification and growth monitoring are realized by integrating multiple machine learning algorithms and combining multi-source data.

Description

Technical Field

[0001] The present invention belongs to the field of smart agriculture and agricultural information technology, and in particular relates to an early identification and monitoring method for field crops based on three-stage integrated learning. Background Art

[0002] At present, the development of facility agriculture is becoming increasingly unmanned and intelligent, and digital sensors are widely used to monitor planting conditions. However, in the field of field agriculture, there is still a lack of solutions to achieve the monitoring of planting conditions. Therefore, the development of crop monitoring technology suitable for field agriculture will help to ensure the balance of supply and demand in the grain market and stable prices. In field agricultural production management, the efficiency of manual identification and monitoring of crops is low because seedlings of different types of crops have similar appearances, dense planting, and multiple targets in the field of view. In addition, traditional manual detection methods rely on human expertise and judgment, which are prone to errors and inaccuracies. Therefore, improving the efficiency and accuracy of crop identification and growth monitoring in the early stages has always been a technical difficulty.

[0003] Although traditional field planting can reflect the large-scale crop planting situation in various regions through satellite remote sensing systems, detailed agricultural conditions still need to be understood by field investigators to understand the crop conditions of each plot. Therefore, a rapid crop identification method suitable for field agriculture is urgently needed.

[0004] In the prior art, there is a method and device for monitoring the growth of crop cultivation in facilities based on a hyperspectral camera and a fiber optic spectrometer, which can accurately track the growth status of crop cultivation in facilities in real time; a seedling growth prediction method based on environmental parameters and physiological parameters, which uses a model to determine whether environmental regulation is needed, can improve the economic benefits of seedling cultivation in facilities; a leaf hyperspectral image classification and regression method based on deep learning, which realizes the classification of crop leaf hyperspectral images through a multi-scale cascade convolutional neural network; a green vegetable identification and prediction method based on digital image processing and linear regression, which realizes the automatic identification and cutting of green vegetable plant images in a natural environment, extracts relevant leaf color parameters, and establishes a prediction of changes in images and environmental factors. With the rapid development of agricultural engineering information equipment, approximate solutions have solved the problem of crop identification in field agriculture from scratch, but in the face of non-standard crops and complex production environments, current technologies have the following defects:

[0005] Research focuses on crop classification, while ignoring the early stages of crops. In fact, crop classification is more effective if it is achieved as early as possible, but due to the difficulty of early classification, this reality has not received sufficient attention. As a result, there is a lack of research on methods for identifying field crop species in the early stages.

[0006] Most of the existing patents are based on high-performance spectral cameras to collect images to achieve high recognition accuracy. If this type of patent system is to be applied, additional equipment needs to be installed on the edge of each independent field. However, due to the expensive equipment and high cost of technical application, it is difficult to promote and apply it on a large scale. Therefore, there is an urgent need for a low-cost field crop detection system to replace manual visits and questionnaires in traditional field agriculture, and to assist government departments in realizing unified supervision and management planning of regional agriculture.

[0007] Existing patents usually use linear regression to predict crop growth trends. However, the growth changes of most crops are nonlinear, and it is difficult to accurately reflect the actual situation by relying solely on linear models. Summary of the invention

[0008] In order to solve the above technical problems, the present invention proposes an early identification and monitoring method for field crops based on three-stage integrated learning to solve the problems existing in the above-mentioned prior art.

[0009] To achieve the above object, the present invention provides a method for early identification and monitoring of field crops based on three-stage integrated learning, comprising:

[0010] Obtaining a field overhead picture, preprocessing it after judging whether it meets the integrity, obtaining a crop foreground image based on the preprocessed field overhead picture, and storing the crop foreground image and the corresponding shooting time;

[0011] Calculating a color space histogram of the crop foreground image according to the time sequence of the shooting time, calculating the crop amount based on the color space histogram, judging whether the crop amount reaches a separable threshold, and if so, dividing the crop foreground image into an early picture;

[0012] Obtaining a primary crop category based on the early picture and storing it in association with the early picture;

[0013] A fusion secondary crop classification model is constructed, and the early images are input into the secondary crop classification model to obtain crop category judgment results and growth maturity prediction results.

[0014] Optionally, the aerial view picture of the field is obtained at fixed time intervals. If it does not meet the integrity requirement, a reshoot request is issued until the aerial view picture of the field meets the requirement, and shooting of the current node is stopped.

[0015] Optionally, the process of obtaining a crop foreground image includes:

[0016] After the overhead field pictures are subjected to noise removal and light equalization processing, color space conversion is performed; the converted pictures are preliminarily segmented by the Otsu threshold method, the optimal boundary is determined by the minimum cut algorithm, and a segmented image is obtained based on the optimal boundary and the preliminary segmentation result; the noise of the segmented image is removed by morphological operations, and the crop foreground image is obtained by connected component analysis.

[0017] Optionally, the crop amount is calculated based on the color space histogram, connected component analysis is performed on the crop foreground image obtained after crop transplanting, and individual crops are preliminarily identified, the sum of the areas of each connected component is calculated to obtain the crop area, and the crop amount is obtained by counting the number of pixels of each crop.

[0018] Optionally, the early image is input into a neural network built based on a visual transformer and a self-attention mechanism to obtain a primary crop category.

[0019] Optionally, the fused secondary crop classification model is composed of a multidimensional backbone network and a verification classification network; the multidimensional backbone network includes a macro classification structure and a micro recognition structure; and the verification classification network is composed of an integrated CNN classifier.

[0020] Optionally, the macro classification structure identifies the early pictures through a convolutional neural network to obtain the planting spacing information;

[0021] The microscopic recognition structure is composed of a target detection algorithm to detect key features of crops.

[0022] Optionally, the early images are input into a fusion secondary crop classification model to obtain planting spacing information through a macroscopic classification structure; crop feature recognition results and crop growth period sequence are obtained through a microscopic recognition network;

[0023] The planting spacing information, crop feature recognition results, and crop growth period sequence are input into the model verification classification network to obtain crop category judgment results and growth maturity prediction results.

[0024] Optionally, the verification classification network is connected to multiple different deep learning networks.

[0025] Compared with the prior art, the present invention has the following advantages and technical effects:

[0026] The present invention first uses a field monitoring camera to obtain overhead pictures at fixed time intervals, judges the integrity of the image and extracts the area of ​​interest, judges whether there are crops and records the timestamp, dynamically analyzes the crop quantity and stores the data. The color histogram is used to judge whether the crop quantity reaches the separable threshold to form an early image data set. The visual transformer is used to perform primary crop classification based on a neural network with a self-attention mechanism, and the crops are divided into leafy vegetables and fruit vegetables. According to the classification results, the microscopic crop characteristics and macroscopic planting spacing are identified, and the growth period characteristics are analyzed in combination with time information. Finally, multidimensional data fusion is performed through a convolutional neural network (CNN), a crop identification model and a growth maturity prediction model are constructed, and accurate identification and prediction of crop categories and growth conditions are achieved.

[0027] The present invention supplements the current research gap in the field of early identification of field crops and realizes digital monitoring of crop growth throughout the entire process. This innovative method greatly improves the efficiency of field agricultural management and can effectively replace traditional field surveyors, thereby greatly reducing the management cost of field agriculture. This invention provides a new way for the management and efficiency of field agriculture, which has important practical significance and broad application prospects;

[0028] The present invention overcomes an important problem in the current field of field crop detection: reliance on expensive equipment, such as hyperspectral cameras, near-infrared cameras, etc. The high cost of these devices limits their application in this field. The present invention is based on the real environment of the field, does not require any additional equipment, and combines basic monitoring cameras with desktop computers to achieve efficient and accurate identification of crop early stages and prediction of each growth period, thereby providing a practical and economical method for detecting field crops;

[0029] The present invention simplifies the complexity of traditional feature selection methods by integrating deep learning algorithms, and effectively simulates the nonlinear changes in crop characteristic growth. Through this system, farmers can directly conduct autonomous monitoring, understand the growth of field crops in the early stages, and then choose the optimal water and fertilizer irrigation strategy to achieve timely and effective agricultural management, ultimately increasing crop yields and planting benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0032] Figure 2 It is a schematic diagram of the module structure of an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the ViT self-attention network workflow of an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of the YOLOX target detection network detection process according to an embodiment of the present invention;

[0035] Figure 5 1 is a diagram of the model training process of an embodiment of the present invention, (a) is the curve corresponding to the extended Epoch=200, (b) is the curve corresponding to two rounds of Epoch=100, (c) is the curve corresponding to three rounds of Epoch=100, (d) is the Mobilenet loss function curve, (e) is the Resnet50 loss function curve, (f) is the VGG loss function curve, and (g) is the ViT loss function curve;

[0036] Figure 6 : These are the results of the model empirical verification of the embodiments of the present invention, (a) is the curve corresponding to the extended Epoch = 200, (b) is the curve corresponding to two rounds of Epoch = 100, (c) is the curve corresponding to three rounds of Epoch = 100, (d) is the accuracy of Mobilenet, (e) is the accuracy of Resnet50, (f) is the accuracy of VGG, and (g) is the accuracy of ViT. DETAILED DESCRIPTION

[0037] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Embodiment 1

[0040] like Figure 1-6 As shown, this embodiment provides a method for early identification and monitoring of field crops based on three-stage integrated learning, including:

[0041] S1. Use a surveillance camera installed in the field to obtain aerial images of the field at fixed time intervals and record the current timestamp. The camera position is fixed to ensure the stability and comparability of continuous images.

[0042] S2. Preprocess the acquired overhead images, including noise removal and illumination equalization. Common methods include Gaussian filtering and adaptive histogram equalization to improve image quality. Then, convert the RGB image to other color spaces, such as HSV (hue, saturation, brightness). By selecting a color space common to crops, it is easier to separate the foreground and background. First, the image is preliminarily segmented using the Otsu threshold method. The Otsu method determines the optimal threshold by maximizing the inter-class variance and divides the image into foreground and background. Further, the image segmentation method based on the graph cut is finely segmented. The graph cut method represents the image as a graph, where nodes represent pixels and edge weights represent the similarity between pixels. The optimal boundary between the foreground and the background is found through the minimum cut algorithm. Finally, the segmentation results are post-processed, including morphological operations (such as dilation and erosion) to remove small noise areas, and connected component analysis is used to retain the largest foreground area, ensuring the coherence and integrity of the segmentation results.

[0043] S3. When the crop seedlings are transplanted into the cultivation area of ​​the greenhouse, the biomass (pixel value of the characteristic color interval) of the foreground image is calculated. If the biomass reaches the minimum recognizable level or the specific characteristics are visible after the seeds germinate, then the transplanting timestamp T0 is recorded. Connected component analysis is performed on the crop foreground image obtained from S2, and each connected area is marked and counted to preliminarily identify each crop individual. Connected component analysis can effectively distinguish between separate crops. First, the area of ​​each connected component is calculated, and the crop amount is determined by counting the number of pixels of each crop. The area calculation formula is:

[0044]

[0045] Among them, A i represents the area of ​​the i-th connected component, and n is the total number of connected components. The crop area and quantity obtained in the above steps are combined with the timestamp information for dynamic statistical analysis. Use time series analysis methods (e.g., moving average and exponential smoothing) to monitor the trend of crop quantity. Finally, display the statistical results in the form of charts, such as using a line chart to show the trend of crop quantity over time, for easy observation and analysis.

[0046] S4. Calculate the HSV, RGB, and CMYK color histograms of the (foreground region of interest) ROI area according to the above formula. When the crop volume reaches the separable threshold, record the image as a separable early image and save the timestamp Tn. This process is used to determine the specific time when the crop growth reaches the classification conditions.

[0047] S5. Use a neural network based on a vision transformer (ViT) and a self-attention mechanism to perform primary crop classification. Classify field crops into common leafy vegetables and fruiting vegetables to improve the accuracy and efficiency of subsequent processing. The specific steps include standardizing the input image, extracting timestamps and leaf area information, and preliminarily classifying crops through the ViT model. The output result is the crop category label (leafy vegetables or fruiting vegetables) and its corresponding image data, which will serve as the input of S6.

[0048] S6. Based on the recognition results of S5, identify microscopic crop features, such as leaf shape and color changes, from leafy vegetables and fruit and vegetable crops respectively; at the same time, detect the macroscopic planting spacing to evaluate the planting density. Combined with time information, calculate the time series data of crops, and analyze the crop characteristics and changes in each growth period. The output results include crop feature recognition results, planting spacing information, and crop growth period time series. These detailed feature information and growth data will be used for model construction in S7.

[0049] S7, based on the data provided by S6, uses Convolutional Neural Network (CNN) to perform multi-dimensional data fusion to generate common crop identification models and crop growth and maturity prediction models. The fused data includes flower, fruit and leaf pictures, planting spacing and growth period data, and the CNN model training ensures accurate identification of crop types and growth conditions. The final output is a secondary crop identification model, and provides the most suitable prediction results for crops and their growth conditions, helping users to accurately identify and predict crop categories and growth conditions.

[0050] S8 accurately and quickly gives various crop categories and maturity predictions based on the multivariate model constructed in step S7. The model can provide crop classification information at an early stage and predict future growth and maturity, helping growers optimize management decisions.

[0051] Through the above steps, the present invention can realize the rapid identification and growth monitoring of field crops in the early stage, has the characteristics of high efficiency, accuracy and intelligence, and provides strong technical support for large-scale agricultural production.

[0052] The present invention is realized by a field crop early stage rapid identification and growth monitoring system based on a cascade neural network. The system first uses a field monitoring camera to obtain overhead pictures at fixed time intervals, judges the integrity of the image and extracts the region of interest (ROI), judges whether there are crops and records the timestamp, dynamically analyzes the crop quantity and stores the data. The color histogram is used to judge whether the crop quantity reaches the separable threshold to form an early image data set. The visual transformer (ViT) is used to perform primary crop classification based on a neural network with a self-attention mechanism, and the crops are divided into leafy vegetables and fruit vegetables. According to the classification results, microscopic crop characteristics and macroscopic planting spacing are identified, and the growth period characteristics are analyzed in combination with time information. Finally, multidimensional data fusion is performed through a convolutional neural network (CNN), a crop identification model and a growth maturity prediction model are constructed, and accurate identification and prediction of crop categories and growth conditions are achieved.

[0053] The system structure diagram is as follows: Figure 2 As shown, it mainly includes the following modules:

[0054] (1) Data acquisition hardware module, mainly composed of field monitoring cameras and local area network, mainly collects data can be divided into image parameters and device parameters. Image parameters include: resolution, timestamp, clarity, color deviation, storage format and image size, etc.; device parameters include: battery power, setting number, location information, time, protocol number, available storage space size.

[0055] (2) Image acquisition module software module: First, the field monitoring camera records the entire field through overall macro shooting, and then performs image integrity detection. If the image is damaged or blurred, it requests another image acquisition until a valid image acquisition is completed. Then, the foreground segmentation is performed on the effective clear image to extract the effective planting area, and it is determined whether crops appear. If crops appear, the transplanting timestamp T0 is recorded, and finally the time information and ROI area image are uploaded to the database for storage.

[0056] (3) The primary crop classification module is mainly composed of a digital image processing system and a ViT self-attention network. First, the crop ROI image and the corresponding time series information after transplantation are obtained from the database. Then, the image color space histogram is used to determine whether the crop reaches the threshold for separation. If the crop reaches the threshold for separation, the timestamp Tn is refreshed and then sent to the ViT self-attention network to distinguish whether it is a leafy vegetable or fruit vegetable primary crop. Finally, the time information, primary crop category, and ROI area image are uploaded to the database for storage.

[0057] The ViT self-attention network divides the input early stage field crop images into image slices at certain area sizes. Then the divided image blocks are combined into sequences, and the combined one-dimensional sequences are passed into the algorithm's unique multi-head self-attention for feature extraction, and finally classified using image spatial features. Similar to the traditional crop recognition network, the entire self-attention network is divided into two parts: feature extraction part and classification network.

[0058] The feature extraction part includes the crop information transformation layer and the information extraction layer. The crop information transformation layer takes the input image size of 224×224 and divides the image into fixed-size pixel blocks. The pixel block size is 16×16, so each image will generate 196 pixel blocks, that is, the input sequence length is 196, and the length of each pixel block is 16×16×3=768. The dimension of the linear projection layer is 768×N (N=768), so the dimension of the input after passing through the linear projection layer is still 196×768, that is, there are a total of 196 block features, and the dimension of each block feature is 768. According to the location of each block feature, the position encoding is added, so the dimension of the total input feature is 197×768. The information extraction layer is an attention encoder. This layer extracts features from each feature block. The feature block position encoding will interact with other features and integrate the features of other image sequences. Finally, the self-attention features after extracting features using the multi-head attention mechanism structure are used as the basis for the subsequent classification network.

[0059] The classification network is a multi-head neural network, which stacks 4 layers of one-dimensional crop information input. The dimension is first enlarged to 197×768×4, and then reduced to 197×768 through the fully connected FC layer, and then the image space convolution vector is calculated. Finally, the category of the self-attention feature is calculated through the regularized connection layer LN. The specific formula is as follows:

[0060]

[0061] In the formula, To input early stage field crop images, is a two-dimensional pixel block, C is the number of channels, P is the pixel block size, there are N pixel blocks in total, N = HW / P2. y is the crop image label. In the first-level crop classification model here, leafy vegetables and fruit and vegetable crops are distinguished based on the remote contour and the overall planting method.

[0062] The first-level crop classification model uses image color space threshold detection as the input to identify early crop planting conditions, and introduces the concept of shooting timestamps to supplement the temporal information of static images. The backbone network is a fruit and vegetable classification model based on the self-attention mechanism. It uses knowledge distillation and model pruning to migrate neural networks and achieve fast computing in field edge devices. This design shares computing power with the computing core and avoids the fitting of the second-level classification model caused by massive data.

[0063] (4) The secondary crop classification module mainly uses the YOLOV8 network to perform data fusion on micro-feature target recognition and macro-distance detection, thereby constructing a deep convolutional neural network classifier to extract crop ROI images and time series information from the database, and then identify specific crop categories through a deep learning classification network, and make crop growth and maturity predictions.

[0064] The secondary crop classification model consists of a multidimensional backbone network and a verification classification network. The multidimensional backbone network includes two parts: macro classification and micro identification. Macro classification uses a convolutional neural network to identify the overall content of the image, realize crop growth period identification, calculate vegetation coverage and estimate the stress of the crop population. The micro identification network is composed of a target detection algorithm, which combines agricultural experience and knowledge to detect key crop characteristics, such as crop flowering and fruiting, estimate crop height and width, and detect planting methods. The specific process is as follows:

[0065] First, crop ROI images and related time series information containing early stage images are extracted from the database. This time series information is obtained by recording image data at different time points during crop growth to ensure that the key time nodes of crop growth can be captured. The extracted images are identified for specific crop categories through a deep learning classification network.

[0066] During the recognition process, the secondary crop classification module constructs a CNN classification model through data fusion, and the output result format is: [category 1 confidence 0.5, category 2 confidence 0.38, category 3 confidence 0.1, ...]. If the recognition result category 1 is consistent with the crop's own growth period, the recognition result is considered correct; if the recognition result does not meet the requirements, category 2 is matched according to the confidence level to ensure the accuracy of the classification.

[0067] The model verification classification network is composed of an integrated CNN classifier, which integrates multiple lightweight convolutional neural networks. The verification model is connected to multiple different deep learning networks, and the multi-network MLP decision voting method fully integrates time, microscopic and macroscopic image information to accurately give the current state of the crop. The results are corrected by coupling multi-dimensional data, and finally provide information such as specific crop types, crop growth periods, crop flowering and fruiting conditions, crop coverage, and whether there is water stress, to ensure the accuracy and efficiency of crop management.

[0068] (5) The model construction module is mainly composed of two types of deep learning models, namely CNN convolutional neural network (YOLOV8) and ViT self-attention mechanism neural network.

[0069] First, the crop images collected by the online system are segmented through the foreground image to generate timestamp information, standardized image size and clarity. Among them, the first batch of images require manual sample annotation according to the task type, and are subdivided into primary crop categories such as leafy crops, fruit and vegetable crops, etc., secondary categories such as the names of various crops such as tomatoes, lettuce, cucumbers, peppers, etc., and crop feature categories such as the specific phenological period of tomatoes, which is divided into early, flowering and fruiting, mature, and harvesting periods. According to the target detection task, labels such as tomato florets, immature tomatoes, and mature tomatoes are marked according to the characteristics of tomato flowers and fruits, and the north-south and east-west ridge directions of crops are marked according to the crop planting method, and the spacing between planted crops is calculated by detecting the position of each ridge.

[0070] Then, the labeled images are fused and enhanced, and the number of image samples is expanded to avoid overfitting of the model training with small sample data. The image data is then uploaded to a high-performance server for network training, so that the model loss function reaches maximum saturation, and a model evaluation test is performed. Finally, if the comprehensive recognition accuracy of the model meets the standard, the model is lightweight and deployed on the central control dispatch computer. If the comprehensive recognition accuracy of the model does not meet the standard, the weight file is backed up and stored, and the model is trained again.

[0071] In this technology, 60% of the data set is used as a training set, 20% as a validation set, and 20% as a test set during model training. The update model is divided into a classification network and a target detection network according to its task. The commonly used indicators in the classification network are calculated as follows:

[0072]

[0073] In the formula, TP is the number of correct recognition, FP is the number of missed recognition, TN is the number of correct recognition for negative samples, FN is the number of false positives; F1score is the harmonic mean.

[0074] Another type of target detection network uses the average recognition rate and Precision-Recall curve as the main evaluation method for the detection task (AP value, PR curve for short), the formula is as follows:

[0075]

[0076] Where n is the total number of model detection boxes, AP IoU＝i represents the model detection accuracy under the condition of confidence IoU=i, P k Represents the calculation accuracy of the kth detection box, R k Represents the recall rate of the k-th detection box.

[0077] For multi-target detection tasks, mAP is used as the evaluation method, which means calculating AP for all categories and then taking the average. The formula is as follows:

[0078]

[0079] In summary, the calculation accuracy, calculation efficiency and other characteristics are fully considered, and the PR curve, AP, mAP in target detection and the accuracy, precision, recall rate and F1score in the classification model are used as evaluation indicators. The classification model takes effect when the overall recognition accuracy of the classification network is greater than 90% and the difference between the accuracy, recall rate and F1score is less than 5%, and the overall mAP of the target detection model is greater than 80% and the difference between the AP of various target features is less than 10%. The recognition model takes effect. Finally, the model number is generated and entered into the model database and the model is deployed online.

[0080] Data acquisition: The data used in this example is divided into three parts and installed in the field test site of the Central Agricultural Demonstration Base in the Huanghuaihai Irrigation Area. Agricultural monitoring stations and high-definition surveillance cameras are installed in the field, through which images of crops can be obtained from all angles throughout their growth period. The case study object selected Zhongmai 578, which is suitable for a wide planting area, immune to a variety of common diseases and pests, and has a high yield per mu. Morphologically, the seedlings are semi-upright, strong, cold-resistant, strong tillering ability, and high ear formation rate. The plant type is compact, heading early, and the ear layer is thick. The plant height is about 80 cm and has strong resistance to lodging. From September 2021 to April of the following year, wheat field crop images were collected at 9 am and 3 pm respectively.

[0081] Data processing: This example acquired more than 6,670 enhanced images of monitoring sites, which were acquired from sowing in October to May of the following year, and a total of 672 overhead wheat images of the experimental field were taken at regular intervals. This study used LabelImg to manually label the images, with 15,680 labels for stressed water-deficient (dry) wheat and 12,870 labels for healthy wet (wet) wheat, for a total of 28,550 labels. In order to restore the illumination changes in the natural environment and solve the problem of asymmetric quantities between various samples, 8 types of data enhancement preprocessing were performed for the small number of labels in the wheat images, including color balance, brightness adjustment, grayscale balance enhancement, left and right mirroring, color enhancement, image slicing, lateral deformation, and longitudinal deformation. In addition, according to different visual tasks, 60% were randomly selected as training sets, 20% as validation sets, and 20% as test sets.

[0082] In terms of model construction: In this example, a three-stage cascade neural network of "feature-crop-growth period" is constructed through multi-site monitoring images, wheat macro-aerial photography and wheat feature side photography data. The first-level crop classification module and the growth period recognition of featureless images both use ViT-16 based on the architecture of the self-attention mechanism, and compare MobileNet, ResNet and Vgg based on the classic deep learning CNN architecture. Each model is trained for 200 rounds (Epoch), including 50 rounds of frozen training and 150 rounds of thaw training. The second-level crop classification module uses the YOLOV8s model. In order to explore the potential performance of the network, based on the original network, three optimization training strategies are adopted, namely, enhanced training (Adam optimization and cosine annealing learning rate), extended training, and multi-round training. The experiment uses the Pytorch1.10 deep learning framework, and the hardware accessories are as follows: CPU is Intel(R) Core(TM) i7-10700, GPU is Nvidia GeForce GTX 1660Ti, computing acceleration environment is CUDA10.1CUDNN7.5.0, development software environment is Pycharm 2020, and operating system is Windows 10. The training parameters are set as follows: the byte width is dynamically set to 8 or 4 according to the network size, the momentum factor is set to 0.95, the cosine annealing attenuation coefficient is 0.005, the initial learning rate is 0.001, and the model training uses Adam optimization and cosine annealing to dynamically optimize the learning rate.

[0083] Example verification results: In the classification task, in the early stage of training, the main network parameters are frozen, the model has fewer learnable parameters, and the loss value is relatively large. Among them, the Vgg and ViT networks have the fastest loss value drop during the frozen training process due to the characteristics of complex networks and many parameters. The corresponding curve of the training process is attached. Figure 5. When the model training enters the unfreezing training, a large number of backbone network parameters are activated, which is manifested as a jump in the total loss value. The ViT model with the self-attention mechanism can smoothly enter the unfreezing training, making the loss value lower than the original best performing Vgg model. After 185 rounds of training, the model learning rate reaches saturation. Finally, the loss of the ViT network training set is 0.0480, and the loss of the validation set is 0.0045. Among them, the ViT model has the best classification effect, and the model recognition is accurate as shown in the attached figure. Figure 6 , the test results show that the accuracy of this method for positive samples is 99.71%, the average recall rate is 99.71%, the average precision is 99.72%, and the harmonic mean score is 99.71%. In the target detection task, the training results show that the average mAP accuracy is improved with each additional round of training. By adding the first round, it is improved by 0.08% compared with the first round of training, among which the recognition accuracy of stressed water-deficient wheat (dry) is improved by 0.06%, and the recognition accuracy of healthy wet wheat (wet) is improved by 0.1%. After three rounds of training, the final accuracy reaches 99.71%, which is 0.03% higher than the second round of training, among which the recognition accuracy of stressed water-deficient wheat (dry) is improved by 0.04%, and the recognition accuracy of healthy wet wheat (wet) is improved by 0.02%. Increasing the training Epoch can effectively improve the model accuracy, but after multiple rounds of training, with the increase of training Epoch, the computing power consumption cost increases greatly, and the corresponding training model benefits become lower.

[0084] The method proposed in this embodiment aims to realize automatic monitoring and planting planning of crops by quickly and accurately identifying crop types through real-time monitoring of crop growth conditions. This method can not only improve farmers' crop planting efficiency, but also help improve the overall level of agricultural production. The three-stage integrated learning method integrates multiple machine learning algorithms and combines multi-source data to achieve efficient and accurate crop identification and growth monitoring.

[0085] The present invention adopts a lightweight integrated convolutional neural network model for crop growth trends. The algorithm optimizes the cumbersome feature selection process in traditional machine learning and realizes an end-to-end processing flow, from collecting images of crops in their early stages to accurately identifying crop categories, and then predicting the key time nodes of each crop growth period from image timestamps.

[0086] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for early identification and monitoring of field crops based on three-stage ensemble learning, characterized in that: The following steps are involved: Obtaining a field overhead picture, preprocessing it after judging whether it meets the integrity requirement, obtaining a crop foreground image based on the preprocessed field overhead picture, and storing the crop foreground image and the corresponding shooting time; Calculating a color space histogram of the crop foreground image according to the time sequence of the shooting time, calculating the crop amount based on the color space histogram, judging whether the crop amount reaches a separable threshold, and if so, dividing the crop foreground image into an early picture; Obtaining a primary crop category based on the early picture and storing it in association with the early picture; A fusion secondary crop classification model is constructed, and the early images are input into the secondary crop classification model to obtain crop category judgment results and growth maturity prediction results.

2. The method for early identification and monitoring of field crops based on three-stage integrated learning according to claim 1, characterized in that: The aerial view of the field is obtained at fixed time intervals. If it does not meet the integrity requirements, a reshoot request is issued until the aerial view of the field meets the requirements, and shooting of the current node is stopped.

3. The method for early identification and monitoring of field crops based on three-stage integrated learning according to claim 1, characterized in that: The process of obtaining the crop foreground image includes: After the overhead field pictures are subjected to noise removal and light equalization processing, color space conversion is performed; the converted pictures are preliminarily segmented by the Otsu threshold method, the optimal boundary is determined by the minimum cut algorithm, and a segmented image is obtained based on the optimal boundary and the preliminary segmentation result; the noise of the segmented image is removed by morphological operations, and the crop foreground image is obtained by connected component analysis.

4. The method for early identification and monitoring of field crops based on three-stage integrated learning according to claim 1, characterized in that: The crop amount is calculated based on the color space histogram, the crop foreground image obtained after crop transplanting is subjected to connected component analysis, and individual crops are preliminarily identified. The sum of the areas of each connected component is calculated to obtain the crop area, and the crop amount is obtained by counting the number of pixels of each crop.

5. The method for early identification and monitoring of field crops based on three-stage integrated learning according to claim 1, characterized in that: The early images are input into a neural network built based on a visual transformer and a self-attention mechanism to obtain the first-level crop category.

6. The method for early identification and monitoring of field crops based on three-stage integrated learning according to claim 1, characterized in that: The fused secondary crop classification model is composed of a multidimensional backbone network and a verification classification network; the multidimensional backbone network includes a macro classification structure and a micro recognition structure; the verification classification network is composed of an integrated CNN classifier.

7. The method for early identification and monitoring of field crops based on three-stage integrated learning according to claim 6, characterized in that: The macro classification structure recognizes the early pictures through a convolutional neural network to obtain the planting spacing information; The microscopic recognition structure is composed of a target detection algorithm to detect key features of crops.

8. The method for early identification and monitoring of field crops based on three-stage integrated learning according to claim 7, characterized in that: The early images are input into a fusion secondary crop classification model to obtain planting spacing information through a macroscopic classification structure; crop feature recognition results and crop growth period sequence are obtained through a microscopic recognition network; The planting spacing information, crop feature recognition results, and crop growth period sequence are input into the model verification classification network to obtain crop category judgment results and growth maturity prediction results.

9. The method for early identification and monitoring of field crops based on three-stage integrated learning according to claim 6, characterized in that: The verification classification network is connected to a plurality of different deep learning networks.

Images