Automatic crop number counting method
The crop images are collected by drones and identified and corrected using neural network models, which solves the problems of low manual counting efficiency and large errors, and achieves efficient and accurate automatic counting of crop plants.
Patent Information
- Application Number
- CN202411567586.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2025-03-18
AI Technical Summary
The prior art has low efficiency and large errors in the number of crops manually, making it difficult to achieve fine agricultural management and improve crop yields.
The drone is used to obtain the training image and target image of the crop, and the images are identified through the neural network model, and the adjusted Euclidean distance between the predicted position points are calculated, and the predicted position points that are too close are combined and the number of predicted plants is corrected.
It improves the efficiency and accuracy of crop plant counting, reduces manual intervention, avoids repeated counting, and significantly improves the accuracy of prediction results.
Smart Images
Figure CN119649207A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of crop counting, and in particular to a method for automatically counting the number of crop plants. Background Art
[0002] The number of crop plants directly affects the management and decision-making of agricultural production. It is an important indicator in the agricultural field to evaluate crop growth conditions, predict yields, and optimize planting density. Accurately obtaining the number of crop plants has important theoretical significance and application value for achieving precision agricultural management and increasing crop yields. At present, the acquisition of crop plant numbers mainly relies on manual counting. Manual counting is not only inefficient, but also often has large errors, especially in large-scale planting. Summary of the invention
[0003] In view of the above problems in the prior art, the present invention provides a method for automatically counting the number of crop plants, which solves the problem of low efficiency and large error in manually obtaining the number of crop plants in the prior art.
[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0005] A method for automatically counting the number of crop plants is provided, comprising the steps of:
[0006] S1, obtain crop training images and target images through UAV;
[0007] S2, using training images to train the neural network model to obtain a plant automatic counting model based on P2PNet;
[0008] S3, inputting the target image into the automatic plant counting model for identification, and obtaining the predicted position points of the crops and the corresponding predicted number of plants;
[0009] S4. Calculate the adjusted Euclidean distances between all predicted position points through the automatic plant counting model, and determine whether each adjusted Euclidean distance is greater than the set minimum distance threshold. If so, the automatic plant counting model merges the two predicted position points corresponding to the adjusted Euclidean distance into one and corrects the predicted number of plants. Otherwise, keep the data of the predicted number of plants.
[0010] In this solution, drones are used to collect massive image data, which greatly improves data collection efficiency and coverage. The automatic plant counting model can process plant images of different scales and densities. Compared with manual technology, it improves work efficiency, reduces manual intervention, and ensures the accuracy of plant counting. In addition, step S4 can avoid the two predicted position points being too close, avoid repeated counting, and improve the accuracy of the prediction results.
[0011] Furthermore, the construction method of the plant automatic counting model is:
[0012] S2.1. Preprocess the training image and cut the training image into multiple sub-images according to the distribution of crops. The number of crop plants in the multiple sub-images is moderate and the number of crop plants is kept within a certain range. Setting the number of crop plants in the sub-images in this way is conducive to the learning of the automatic plant counting model.
[0013] S2.2. Use the Labelme tool to mark the crops in each sub-image and divide the multiple sub-images into training set, validation set and test set.
[0014] S2.3. Optimize the multi-scale feature fusion module of the P2PNet model as a plant automatic counting model. The multi-scale feature fusion module can capture feature information at different scales, making the neural network model more suitable for crops with different densities and distributions.
[0015] S2.4, input the training set into the automatic plant counting model, and use the Adam optimizer to optimize the parameters of the automatic plant counting model through adaptive learning rate and momentum adjustment, train the automatic plant counting model, accelerate model convergence and improve training stability. The Adam optimizer is used to dynamically adjust the learning rate to ensure that the automatic plant counting model converges stably during training and achieves the best performance.
[0016] S2.5. Use the validation set and test set to evaluate the performance of the automatic plant counting model.
[0017] Furthermore, the ratio of the number of sub-images in the training set, validation set, and test set is 8:1:1.
[0018] Furthermore, between step S2.3 and step S2.4, data enhancement is performed on the sub-images in the training set. Data enhancement includes random cropping, random flipping in the horizontal and vertical directions, random angle rotation, random scaling, and random changes in brightness, contrast, saturation, and hue to increase data diversity and improve the robustness of the model.
[0019] Furthermore, the multi-scale feature fusion module processes the sub-images in the following manner:
[0020] S2.31. Due to the large difference in plant scales at different growth stages, in order to better capture multi-scale features, the VGG16_bn backbone network of the P2PNet model is used to extract feature maps of multiple scales from sub-images, and a multi-scale feature fusion module is used for feature fusion. The multiple feature maps include three scales: H / 4×W / 4, H / 8×W / 8, and H / 16×W / 16.
[0021] S2.32. Use the maximum pooling layer and nearest neighbor interpolation to downsample or upsample all feature maps of size H / 4×W / 4 and all feature maps of size H / 16×W / 16 to size H / 8×W / 8, respectively.
[0022] S2.33. All feature maps are concatenated in the depth direction to obtain concatenated feature maps, and a 1×1 convolutional layer is used to fuse the concatenated feature maps to reduce the dimension while retaining the key features.
[0023] Furthermore, when the plant automatic counting model is trained, its loss function in and are classification loss and positioning loss respectively, and λ2 is the weight factor. The combination of classification loss and positioning loss improves the overall performance of the model in classification and positioning.
[0024] Furthermore, in order to solve the problem of class imbalance in classification tasks, the model pays more attention to samples that are difficult to classify. The classification loss is expressed as:
[0025]
[0026] Where M is the number of all the suggested points obtained by the neural network model for processing the sub-image, N is the number of suggested points classified as crops, is the predicted value of the confidence score of the i-th suggestion point, λ1 is the weighting factor of the negative suggestion point, c s (i) is the true value of the confidence score of the i-th proposed point, γ is the adjustment factor, which enhances the importance of misclassified samples. The proposed point is the candidate point of the crop in the sub-image.
[0027] Furthermore, the expression of positioning loss is:
[0028]
[0029] Among them, p i Mark the position of the i-th dot; is the position of the i-th recommended point; ∥·∥ 2 is the Euclidean distance, and N is the number of proposed points classified as crops.
[0030] Furthermore, the adjusted Euclidean distance d between the predicted position points of two adjacent crops is adj for;
[0031]
[0032] Where x1, y1 and x2, y2 are the coordinates of the two crop prediction position points respectively, k is the adjustment constant, ρ is the density adjustment factor, which indicates the density of plants in a certain area. The distance threshold can be dynamically adjusted according to the density of points in the local area, so that points can be screened more strictly in dense areas, while the screening conditions can be relaxed in sparse areas; n is the actual number of crop plants in the sub-image, H is the height of the image, and W is the width of the image.
[0033] Further, step S2.5 includes:
[0034] S2.51, input the validation set into the plant automatic counting model to obtain the predicted number of plants, and calculate the mean square error between the predicted number of plants and the actual number of plants marked;
[0035] S2.52, using mean square error as the performance evaluation method for the automatic plant counting model, and obtaining multiple automatic plant counting models by adjusting model hyperparameters and training strategies;
[0036] S2.53, using the test set to obtain the mean square error of multiple plant automatic counting models;
[0037] S2.54. Select the automatic plant counting model with the smallest mean square error as the final optimized automatic plant counting model.
[0038] The present invention discloses a method for automatically counting the number of crop plants, which has the following beneficial effects:
[0039] The plant automatic counting model trained by the present invention through training images can process plant images of different scales and densities, and compared with manual technology, it improves work efficiency, reduces manual intervention, and also ensures the accuracy of plant counting. In addition, step S4 can avoid the two predicted position points being too close, avoid the occurrence of repeated counting, and improve the accuracy of the prediction result. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is a flow chart of the method for automatically counting the number of crop plants; DETAILED DESCRIPTION
[0041] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0042] refer to Figure 1 This embodiment provides a method for automatically counting the number of crop plants, comprising the steps of:
[0043] S1. Use drones to obtain crop training images and target images. Image acquisition should be done during periods with good lighting conditions to ensure image quality.
[0044] S2. Use training images to train the neural network model to obtain an automatic plant counting model.
[0045] S3. Input the target image into the automatic plant counting model for identification, and obtain the predicted position points of the crops and the corresponding predicted number of plants.
[0046] S4. Calculate the adjusted Euclidean distances between all predicted position points through the automatic plant counting model, and determine whether each adjusted Euclidean distance is greater than the set minimum distance threshold. If so, the automatic plant counting model merges the two predicted position points corresponding to the adjusted Euclidean distance into one and corrects the predicted number of plants. Otherwise, keep the data of the predicted number of plants.
[0047] Specifically, the adjusted Euclidean distance between the predicted position points of two adjacent crops is d adj ;
[0048]
[0049]
[0050] Where x1, y1 and x2, y2 are the coordinates of the two crop prediction position points respectively, k is the adjustment constant, n is the actual number of crop plants in the sub-image, H is the height of the image, and W is the width of the image; ρ is the density adjustment factor, which indicates the density of plants in a certain area. The distance threshold can be dynamically adjusted according to the density of points in the local area, so that points can be screened more strictly in dense areas, while the screening conditions can be relaxed in sparse areas, ensuring the accuracy of the prediction and the reliability of the results.
[0051] Specifically, the method for constructing the automatic plant counting model is as follows:
[0052] S2.1. Preprocess the training image and cut the training image into multiple sub-images according to the distribution of crops. The number of crop plants in the multiple sub-images is moderate and kept within a certain range. Setting the number of crop plants in the sub-images in this way is conducive to the learning of the automatic plant counting model.
[0053] S2.2. Use the Labelme tool to mark the crops in each sub-image and divide the multiple sub-images into training set, validation set and test set.
[0054] In this implementation, the ratio of the number of sub-images in the training set, validation set, and test set is 8:1:1. And the sub-images in the training set are data enhanced. Data enhancement includes random cropping, random flipping in horizontal and vertical directions, random angle rotation, random scaling, random changes in brightness, contrast, saturation, and hue to increase data diversity and improve the robustness of the model.
[0055] S2.3. Optimize the multi-scale feature fusion module of the P2PNet model as a plant automatic counting model, where the plant automatic counting model uses VGG16_bn as the backbone network to extract features. The multi-scale feature fusion module can capture feature information at different scales, making the neural network model more suitable for crops of different densities and distributions.
[0056] In this implementation, the multi-scale feature fusion module processes the sub-images in the following manner:
[0057] S2.31. Due to the large difference in plant scales at different growth stages, in order to better capture multi-scale features, the VGG16_bn backbone network of the P2PNet model is used to extract feature maps of multiple scales from sub-images, and a multi-scale feature fusion module is used for feature fusion. The multiple feature maps include three scales: H / 4×W / 4, H / 8×W / 8, and H / 16×W / 16.
[0058] S2.32. Use the maximum pooling layer and nearest neighbor interpolation to downsample or upsample all feature maps of size H / 4×W / 4 and all feature maps of size H / 16×W / 16 to size H / 8×W / 8, respectively.
[0059] S2.33. All feature maps are concatenated in the depth direction to obtain concatenated feature maps, and a 1×1 convolutional layer is used to fuse the concatenated feature maps to reduce the dimension while retaining the key features.
[0060] S2.4, input the training set into the automatic plant counting model, and use the Adam optimizer to optimize the parameters of the automatic plant counting model through adaptive learning rate and momentum adjustment to train the automatic plant counting model. The Adam optimizer is used to dynamically adjust the learning rate to ensure that the automatic plant counting model converges stably during training and achieves the best performance.
[0061] S2.5. Use the validation set and test set to evaluate the performance of the automatic plant counting model.
[0062] Specifically, step S2.5 includes:
[0063] S2.51. Input the validation set into the automatic plant counting model to obtain the predicted number of plants, and calculate the mean square error between the predicted number of plants and the actual marked number of plants.
[0064] S2.52. The mean square error is used as the performance evaluation method for the automatic plant counting model, and multiple automatic plant counting models are obtained by adjusting the model hyperparameters and training strategies.
[0065] S2.53. Use the test set to obtain the mean square error of multiple plant automatic counting models.
[0066] S2.54. Select the automatic plant counting model with the smallest mean square error as the final optimized automatic plant counting model.
[0067] As a further solution of this embodiment, when the plant automatic counting model is trained, its loss function in and are classification loss and positioning loss respectively, and λ2 is the weight factor. The combination of classification loss and positioning loss improves the overall performance of the model in classification and positioning.
[0068] In order to solve the problem of class imbalance in classification tasks, the model pays more attention to samples that are difficult to classify. The expression of classification loss is:
[0069]
[0070] Where M is the number of all the suggested points obtained by the neural network model for processing the sub-image, N is the number of suggested points classified as crops, is the predicted value of the confidence score of the i-th suggestion point, λ1 is the weighting factor of the negative suggestion point, c s (i) is the true value of the confidence score of the i-th proposed point, and γ is the adjustment factor, which enhances the importance of misclassified samples. The proposed point is the candidate point classified as a crop in the sub-image, and the candidate point is a pixel point in a certain area.
[0071] The expression of positioning loss is:
[0072]
[0073] Among them, p i Mark the position of the i-th dot; is the position of the i-th recommended point; ∥·∥ 2 is the Euclidean distance, and N is the number of proposed points classified as crops.
[0074] Although the specific implementation of the invention is described in detail in conjunction with the drawings, it should not be understood as limiting the scope of protection of this patent. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.
Claims
1. A method for automatically counting the number of crop plants, characterized in that: Includes steps: S1, obtain crop training images and target images through UAV; S2, using the training images to train the neural network model to obtain an automatic plant counting model; S3, inputting the target image into the automatic plant counting model for identification, and obtaining the predicted position points of the crops and the corresponding predicted number of plants; S4. Calculate the adjusted Euclidean distances between all predicted position points through the automatic plant counting model, and determine whether each adjusted Euclidean distance is greater than the set minimum distance threshold. If so, the automatic plant counting model merges the two predicted position points corresponding to the adjusted Euclidean distance into one and corrects the predicted number of plants. Otherwise, keep the data of the predicted number of plants.
2. The method for automatically counting the number of crop plants according to claim 1, characterized in that: The construction method of the plant automatic counting model is as follows: S2.1, preprocessing the training image, and cutting the preprocessed training image into multiple sub-images according to the distribution of crops; S2.2, use the Labelme tool to mark the crops in each sub-image, and divide the multiple sub-images into training set, validation set and test set; S2.3, optimize the multi-scale feature fusion module of the P2PNet model as an automatic plant counting model; S2.4, inputting the training set into the automatic plant counting model, and using the Adam optimizer to optimize the parameters of the automatic plant counting model through adaptive learning rate and momentum adjustment, and training the automatic plant counting model; S2.
5. Use the validation set and test set to evaluate the performance of the automatic plant counting model.
3. The method for automatically counting the number of crop plants according to claim 2, characterized in that: The ratio of the number of sub-images in the training set, validation set, and test set is 8:1:
1.
4. The method for automatically counting the number of crop plants according to claim 2, characterized in that: Between step S2.3 and step S2.4, data enhancement is performed on the sub-images in the training set.
5. The method for automatically counting the number of crop plants according to claim 2, characterized in that: The multi-scale feature fusion module processes sub-images in the following ways: S2.31, the VGG16_bn backbone network of the P2PNet model extracts feature maps of multiple scales from the sub-image, and uses the multi-scale feature fusion module to perform feature fusion, where the multiple feature maps include three scales: H / 4×W / 4, H / 8×W / 8, and H / 16×W / 16; S2.32, use the maximum pooling layer and nearest neighbor interpolation to downsample or upsample all feature maps of size H / 4×W / 4 and all feature maps of size H / 16×W / 16 to size H / 8×W / 8 respectively; S2.
33. All feature maps are concatenated in the depth direction to obtain concatenated feature maps, and the concatenated feature maps are fused using a 1×1 convolutional layer.
6. The method for automatically counting the number of crop plants according to claim 2, characterized in that: When training the automatic plant counting model, its loss function in and are classification loss and positioning loss respectively, and λ2 is the weight factor.
7. The method for automatically counting the number of crop plants according to claim 6, characterized in that: The expression of classification loss is: Where M is the number of all the suggested points obtained by the neural network model for processing the sub-image, N is the number of suggested points classified as crops, is the predicted value of the confidence score of the i-th suggestion point, λ1 is the weighting factor of the negative suggestion point, and γ is the adjustment factor; c s (i) is the true value of the confidence score of the i-th proposed point, and the proposed point is the candidate point of the crop in the sub-image.
8. The method for automatically counting the number of crop plants according to claim 7, characterized in that: The expression of positioning loss is: Among them, p i Mark the position of the i-th dot; is the position of the i-th recommended point; ∥·∥ 2 is the Euclidean distance, and N is the number of proposed points classified as crops.
9. The method for automatically counting the number of crop plants according to claim 2, characterized in that: The adjusted Euclidean distance d between the predicted position points of two adjacent crops adj for; Among them, x1, y1 and x2, y2 are the coordinates of the two crop prediction position points, k is the adjustment constant, ρ is the density adjustment factor, n is the actual number of crop plants in the sub-image, H is the height of the image, and W is the width of the image.
10. The method for automatically counting the number of crop plants according to claim 2, characterized in that: Step S2.5 includes: S2.51, input the validation set into the plant automatic counting model to obtain the predicted number of plants, and calculate the mean square error between the predicted number of plants and the actual number of plants marked; S2.52, using mean square error as the performance evaluation method for the automatic plant counting model, and obtaining multiple automatic plant counting models by adjusting model hyperparameters and training strategies; S2.53, using the test set to obtain the mean square error of multiple plant automatic counting models; S2.
54. Select the automatic plant counting model with the smallest mean square error as the final optimized automatic plant counting model.