A method for intelligent high-throughput measurement of rapeseed inflorescence density in the field and its application

Abstract

This invention discloses an intelligent, high-throughput method for measuring rapeseed inflorescence density in field fields and its application. The method includes: using a drone paired with an RGB camera to collect rapeseed field images at intervals throughout the entire flowering period, creating a rapeseed inflorescence dataset; employing a flower cluster (inflorescence) detection and counting strategy, using the number of detection boxes for each rapeseed flower cluster as the flower cluster count output; designing a YOLOv5 network model detection method incorporating the CBAM attention mechanism, taking into account the small target characteristics of rapeseed flower clusters; and establishing a rapeseed inflorescence count calculation model using the field dataset, followed by model testing and optimization. This invention enables rapid, accurate, and high-throughput measurement of the total number of inflorescences per unit area during the entire flowering period of rapeseed in field fields. The acquired rapeseed inflorescence density data can be used to assess the growth vigor of rapeseed in field fields and estimate rapeseed yield levels, helping to understand the growth and development status of crops such as rapeseed, and providing an important new tool for their intelligent and efficient variety breeding.

Description

Technical Field

[0001] This invention belongs to the field of rapeseed genetics and breeding and rapeseed farmland production. Specifically, it relates to an intelligent high-throughput method for calculating rapeseed inflorescence density in the field. It uses UAV remote sensing images and artificial intelligence strategies to calculate the inflorescence density of rapeseed in the flowering period, and uses the inflorescence density data to predict rapeseed growth and yield per unit area. It also relates to its application in estimating rapeseed yield and assessing rapeseed growth / biomass, which can significantly improve the efficiency of breeding superior rapeseed varieties. Background Technology

[0002] Rapeseed yield is composed of the number of siliques per unit area, the number of seeds per silique, and the weight of 1000 seeds. Increasing the number of siliques per unit area is an important way to improve rapeseed yield. Rapeseed siliques develop from floral organs and are located on the main inflorescence and branch inflorescences; each flower has the potential to develop into a silique. Agronomic studies on major rapeseed varieties show that among yield-related traits, the number of branches and the length of the main inflorescence have a significant impact on rapeseed yield. Therefore, appropriately increasing the number of branches to improve the number of siliques per unit area has become a major breeding objective for rapeseed. Flowering is an important stage after rapeseed transitions from vegetative to reproductive growth, and the flowering period typically lasts about 30 days. The degree of flowering not only reflects the seedling growth but also the branching status. Investigating the degree of flowering in rapeseed is of great significance for understanding the overall growth of rapeseed and predicting yield. During the rapeseed flowering period, the flowers are densely packed (up to thousands per square meter), and the flowering process progresses rapidly, making it difficult to accurately survey the flowering characteristics of rapeseed through manual field counting. Furthermore, manual field surveys during the flowering period can severely negatively impact the later growth of rapeseed. Effective methods for surveying and estimating rapeseed flowering information in large-scale fields have long been lacking. The application of remote sensing technology in agriculture has become a trend. It can acquire multi-scale crop canopy information at different temporal and spatial scales without disrupting the structure of large areas of crops, and can be effectively and widely applied to precision agriculture, yield prediction, etc. However, the fixed-period image acquisition and low spatial resolution limit the development of satellite remote sensing technology in precision agriculture. In recent years, unmanned aerial vehicle (UAV) technology has developed rapidly, supporting images of different resolutions as needed and capable of acquiring field data at any time.

[0003] Currently, the counting problem in the plant field is considered an object counting problem in computer science. Some researchers treat it as a density estimation problem, but these related methods discard the object's location information, and the uninterpretability of the count location limits the counting performance of rapeseed clusters. Some researchers treat this problem as an object detection task. Object detection is a fundamental task in computer vision. Detection algorithms are generally divided into two categories: one-stage networks and two-stage networks. Two-stage networks, represented by Faster-RCNN, are a traditional object detection network suitable for detecting various plants and plant organs. Their characteristic is that the first stage trains to generate object candidate boxes, and the second stage trains to complete object detection. The advantage of two-stage networks is high accuracy, but they suffer from slow speed and poor real-time performance. Single-stage networks, represented by the YOLO series, treat detection as a regression process. The input image directly yields the object's location and category information, resulting in fast detection speed and strong real-time performance, but lower accuracy than two-stage networks. YOLOv5, as a classic single-stage deep recognition end-to-end network model, is the latest version of the YOLO series. This model can significantly improve detection efficiency while maintaining the detection accuracy of existing models, making it one of the best choices for high-throughput detection. However, rapeseed flower clusters are small targets, and YOLOv5 has the defect of not being sensitive enough to small target detection, which makes it easy to miss detections and reduces the ability to detect and count rapeseed flower clusters. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide an intelligent high-throughput method for measuring the density of rapeseed inflorescences in the field. The method combines UAV RGB imagery with a deep learning strategy to measure the density of rapeseed inflorescences in the field. The method is trained to take into account the characteristics of small and dense rapeseed flowers, which has a positive effect on the accuracy of counting. It can realize the rapid, accurate and high-throughput measurement of the total number of inflorescences per unit area during the entire flowering period of rapeseed in the field.

[0005] Another objective of this invention is to provide an application of the aforementioned method in estimating rapeseed yield and assessing rapeseed growth / biomass. By automatically detecting and counting rapeseed inflorescences in the field, it helps to understand the growth and development status of crops such as rapeseed, helps to analyze the influencing factors of rapeseed grain yield and assists in predicting rapeseed grain yield per unit area, and can significantly improve the efficiency of breeding superior rapeseed varieties.

[0006] To achieve the above objectives, the present invention employs the following technical measures: an intelligent high-throughput method for measuring the density of rapeseed inflorescences in the field, comprising the following steps:

[0007] Step 1: Use an airborne RGB camera on a drone to capture images of rapeseed in the field during its flowering period to obtain inflorescence image data. Specifically, this includes:

[0008] 1) Rapeseed includes major types of rapeseed crops such as Brassica napus, Chinese cabbage rapeseed, and mustard rapeseed, as well as three ecological types of rapeseed suitable for planting in various production areas: semi-winter type, spring type, and winter type.

[0009] 2) The flowering period refers to the period from when the main inflorescence of the rapeseed plant has several flower buds and petals have unfolded after the plant bolts until the end of the flowering period (when almost all the petals have fallen off).

[0010] 3) The number of inflorescences counted is usually based on randomly sampled areas in plots (breeding experiments) or fields, and the rapeseed field flowering status image data is obtained by taking pictures at intervals throughout the flowering period.

[0011] 4) Drones refer to devices that can be remotely controlled to fly horizontally at a fixed suitable height (usually 5-50 meters above the ground) above rapeseed fields, and are equipped with RGB cameras or multispectral cameras, etc.

[0012] 5) Based on the characteristics of rapeseed inflorescences (approximately 2-5 cm in diameter), and in combination with the performance parameters of the drone's onboard camera, such as pixel count, the drone's flight performance, and weather conditions, appropriate drone flight and shooting plans should be flexibly set.

[0013] 6) When photographing the target area of ​​the rapeseed field, a reasonable image overlap rate should be set according to the requirements of the corresponding image stitching software / program in the later stage (e.g., Photoscan software, image overlap rate is 75%).

[0014] 7) Establish a large-scale dataset of rapeseed inflorescences from drones, containing tens of thousands of rapeseed inflorescence samples and their annotation information;

[0015] Step 2: A deep learning-based rapeseed inflorescence counting strategy, specifically including:

[0016] 1) The problem of counting the number of inflorescences is identified as an object detection problem, and a deep learning-based object detection strategy is used to count the inflorescences.

[0017] It can not only distinguish the detection effect from the visualized results, but also accurately locate the false detections / false detections, so as to optimize the network model in a targeted manner.

[0018] 2) To extract and count the features of rapeseed inflorescences / floral clusters, we selected a convolutional neural network that combines UAV RGB imagery with an attention mechanism.

[0019] Compared with traditional satellite remote sensing imagery and multispectral imaging, UAV RGB imagery is convenient and quick to acquire, is not limited by weather or fixed acquisition cycles, and has the advantage of adjustable resolution as needed.

[0020] 3) Using a YOLOv5 network model that incorporates the CBAM attention mechanism (with an initial learning rate of 0.01 and a loss function of 0.02 after 400 epochs), the model takes one rapeseed flower image as input and outputs a rapeseed inflorescence detection box and an inflorescence count. The final detection accuracy can reach over 90%.

[0021] This invention compares classic detection networks (e.g., Fast-RCNN, CenterNet, YOLO series) and density estimation networks (TasselNet series). The detection evaluation metrics F1-core, Recall, and mAP of this invention are the highest, while the counting metrics RMSE and R are the lowest. 2 The results demonstrate the effectiveness of the network model of this invention in detection counting.

[0022] 4) Train and optimize the network model, use a large amount of image data to measure the running rate, and test its computing speed and throughput.

[0023] This invention, running on Ubuntu 20.04 with 128 GB of RAM and one NVIDIA GTX 3090Ti graphics card, achieved a runtime of 44.8 frames per second, enabling automatic and rapid detection and counting of rapeseed inflorescences using drone RGB images.

[0024] Optionally, in step 2, the evaluation metrics of formulas (1)-(4) are used to evaluate the detection performance of the network model:

[0025] (1)

[0026] (2)

[0027] (3)

[0028] (4)

[0029] In the formula: Precision represents the precision of the network model's detection evaluation metric, which is the proportion of correctly identified positive samples to the total number of positive samples classified by the classifier; Recall represents the recall, which is the proportion of correctly identified positive samples to the actual number of positive samples; F1-score is the harmonic mean of precision and recall, combining the results of precision and recall, with a value ranging from 0 to 1; mAP is used to measure the performance of the object detection algorithm; TP indicates that positive samples were classified as positive samples by the model, FP indicates that negative samples were classified as positive samples by the model, TN indicates that negative samples were classified as negative samples by the model, and FN indicates that positive samples were classified as negative samples by the model; C refers to the type of detection; N represents the number of images in the entire test set; P(k) represents the precision of the k-th recognized image; R(k) represents the change in recall from k-1 to k images.

[0030] The network model counting performance is evaluated using the evaluation metrics of formulas (5)-(6):

[0031] (5)

[0032] RMSE (6)

[0033] In the formula: The evaluation index for model counting This represents the predicted value from the actual value network model. This represents the fitted value of the linear model. This represents the average value; This indicates the correlation between the network model's predicted values ​​and the actual values; RMSE indicates the degree of deviation between the actual values ​​and the network model's predicted values.

[0034] As shown above, the rapeseed inflorescence counting strategy based on deep learning can obtain the total number of inflorescences in the photographed area in a high-throughput, automated, and accurate manner based on rapeseed field image data.

[0035] Accordingly, this invention also claims the application of rapeseed inflorescence density data obtained by the aforementioned method in estimating rapeseed yield and assessing rapeseed growth / biomass, including the following steps:

[0036] 1) First, use the established model to obtain the inflorescence density data of the rapeseed plot, including the inflorescence density during the entire flowering period and at each time point;

[0037] 2) Obtain yield data of rapeseed plots by yield measurement, and obtain important yield-related traits such as the number of branches and total biomass data at harvest by variety testing. Rapeseed growth level can be evaluated by setting up control scores during the seedling stage.

[0038] 3) Using the two sets of data mentioned above, correlation analysis was conducted with software such as Excel. The high correlation results indicate that the obtained data can be used to assess rapeseed growth and biomass, and to predict yield per unit area.

[0039] Compared with existing technologies, the beneficial effects of this invention are as follows: The rise of UAV camera technology and the maturity of deep learning technology in computer vision have made it possible to quickly, non-destructively, and with high throughput investigate the vigorous flowering of rapeseed. Rapeseed petals are mostly bright yellow, forming a striking contrast with the green of leaves and other rapeseed tissues and organs, making them highly recognizable. Rapeseed flowers grow on the main inflorescence and branch inflorescences, and UAV-mounted cameras can accurately identify inflorescences in the flowering stage. Using the flowering inflorescence as the identification and statistical object, inflorescence images are collected at multiple intervals throughout the flowering period for analysis and calculation to obtain single inflorescence density data, and the total inflorescence density is accumulated throughout the entire flowering and development period, thereby enabling accurate estimation of the vigorous flowering of rapeseed. The rapeseed flowering process connects the vegetative growth and silique development of rapeseed, and is closely related to rapeseed yield. Investigating inflorescence density helps to analyze the influencing factors of rapeseed grain yield and assists in predicting rapeseed grain yield per unit area, which can significantly improve the efficiency of breeding superior rapeseed varieties. This study transfers deep learning to the field of rapeseed flowering images, an important oilseed crop in agriculture, to achieve automatic detection and counting of rapeseed inflorescences in the field. This helps to understand the growth and development of rapeseed and other crops, and provides an important new tool for their intelligent and efficient breeding. Attached Figure Description

[0040] Figure 1 Images of rapeseed at different flowering stages, taken by drone;

[0041] Figure 2 Create a flowchart for the rapeseed flower cluster (inflorescence) dataset;

[0042] Figure 3 The diagrams show the network architectures of YOLOv5 and CBAM, where (a) is the original architecture of YOLOv5 and (b) is the architecture of CBAM.

[0043] Figure 4 The diagram shows the improved YOLOv5 network after adding CBAM to layer C3;

[0044] Figure 5 This is a loss iteration graph on the rapeseed flower cluster (inflorescence) dataset;

[0045] Figure 6 A comparison chart of detection results for YOLOv5 and YOLOv5-CBAM;

[0046] Figure 7 A graph showing the application of rapeseed flower cluster (inflorescence) counting results in rapeseed growth;

[0047] Figure 8 A diagram illustrating the application of rapeseed flower cluster (inflorescence) counting results in the breeding of rapeseed varieties in the field;

[0048] Figure 9 This figure illustrates the application of rapeseed flower cluster (inflorescence) count results in yield correlation analysis. Detailed Implementation

[0049] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0050] This invention uses YOLOv5 as the detection network and adds a CBAM attention mechanism module to its backbone network to suppress background information, allowing the network to focus more on the feature information of rapeseed flower clusters, thereby enhancing the feature extraction capability of YOLOv5, enabling it to extract the target features of rapeseed flower clusters more fully, reducing missed detections, and improving detection and counting capabilities.

[0051] Example 1:

[0052] A method for intelligent high-throughput measurement of rapeseed inflorescence density in field fields includes:

[0053] I. Rapeseed Material Situation and Image Data Acquisition Methods

[0054] 1.1 Data Acquisition

[0055] Rapeseed cultivation in China is mainly divided into winter rapeseed and spring rapeseed. Winter rapeseed is primarily concentrated in the Yangtze River basin and areas south of it. This study selected the Yangluo Base of the Oil Crops Research Institute of the Chinese Academy of Agricultural Sciences (30°42′N, 114°30′E), located in Xinzhou District, Wuhan City, Hubei Province, as the research area. The area has an altitude of approximately 24 m and a subtropical monsoon climate. The experimental field was divided into two main fields, A and B, comprising 165 plots, each planted with more than 40 rapeseed varieties. The plot sizes were 8 m² (2 m × 4 m) and 6 m² (2 m × 3 m). Image data on the rapeseed flowering period were collected primarily from February to May 2021, specifically on February 19, February 26, March 3, March 14, and March 22, 2021.

[0056] The drone used for data acquisition was a DJI Phantom 4 Pro V2.0, capturing images with a size of 5,472×3,648 pixels. An automatic aerial photography mode was employed, with forward and lateral overlap rates set at 75%, flight altitudes of 10 m and 15 m, and photos taken at equal intervals. The flight speed was 1.9 m / s, and data acquisition of the study area was completed within 2 hours. Figure 1 As shown, the images of rapeseed flowering periods in the A field area are presented in five different time periods, arranged from left to right. Figure 1 As shown, with the advancement of the rapeseed growth cycle, the number of rapeseed flowers gradually increases during the flowering period, the rapeseed grows more vigorously, and the flower clusters become more dense. With the arrival of the flowering and pod-forming stages, the rapeseed flowers gradually wither, the number of pods increases, and the number of flower clusters slowly decreases to none.

[0057] 1.2 Construction of Rapeseed Flowering Period Dataset

[0058] To obtain training samples for the network model, drones captured rapeseed images from field A at five different times between flowering and withering, and from field B at four different times between flowering and withering. The training set data was taken from the mid-flowering stage of rapeseed on March 3. The original image data from this period was preprocessed, and the dataset was divided into training, testing, and validation sets in a 7:2:1 ratio.

[0059] Specific steps are as follows: Figure 2 As shown. First, the drone aerial images of the study area were stitched together using Photoscan software to obtain orthophotos of the field. Then, the orthophotos of the field were cropped according to the actual ground size using Photoshop software to extract the rapeseed images of each plot. Next, the rapeseed images of each plot were input into LabelImg software, and all rapeseed flower clusters in each plot were labeled with green boxes and labeled with the word "rape". After labeling each plot, it was saved as an XML file, which stored the plot name, the label of each flower cluster and its actual frame location information. A total of about 70,000 labels were labeled for all plots. The labels and plot images were placed in two separate folders, labels and images, with one-to-one correspondence between the names, to obtain the original sample dataset.

[0060] To expand the training dataset, this study uses data augmentation during the network training phase. Operations such as rotation, scaling, translation, cropping, and perspective transformation are applied to the labeled original images to increase the diversity of the training dataset, thus obtaining the cropped training dataset required for the final network model training.

[0061] II. Deep Learning-Based Rapeseed Flower Cluster (Inflorescence) Counting Network

[0062] 2.1 YOLOv5 Detection Network Incorporating Attention Mechanism

[0063] The YOLOv5 network structure consists of three parts: the backbone network, the neck network, and the head network. The specific structure is as follows: Figure 3As shown in (a), the YOLOv5 model is prone to losing information about small targets during convolutional sampling. Since rapeseed flower clusters are small targets, YOLOv5 is prone to both false positives and false negatives. Therefore, to enhance YOLOv5's ability to extract small target features, this invention introduces a CBAM attention module, the specific structure of which is shown below. Figure 3 As shown in (b), it is embedded after the C3 module in layer 7 of the YOLOv5 backbone network to improve target saliency and enhance detection accuracy. Figure 4 The diagram shows a simplified structure of the YOLOv5-CBAM network. First, an image of rapeseed with dimensions of 640*640*3 is input. The backbone network generates three scale feature maps F1, F2, and F3 to detect targets of different sizes, with dimensions of 80*80*256, 40*40*512, and 20*20*1024, respectively. The 40*40*512 feature map F2 is then input into the CBAM attention structure. CBAM contains two independent sub-modules: Channel Attention (CAM) and Spatial Attention (SAM), which perform attention in the channel and spatial dimensions, respectively.

[0064] In the CBAM structure, feature map F2 first passes through the CAM (Channel Attention Module). Global max pooling and global average pooling are then performed on F2's width and height, respectively, resulting in two 1×1×512 feature maps. These are then fed into a neural network (MLP) and activated using the ReLU activation function. Next, the MLP output feature maps are summed and activated using the Sigmoid activation function to generate the channel attention feature map. Finally, the channel attention feature map and the input feature map F2 are multiplied to generate the final feature map. Then The input is first processed by the SAM (Spatial Attention Module). We perform channel-based global max pooling and global average pooling to obtain two 40×40×1 feature maps. Then, we concatenate the two feature maps using a concat operation and convolve them with a 7×7 kernel to reduce the number of channels to 1 (40×40×1). After passing the sigmoid activation function, we generate a spatial attention feature map. Finally, we combine the spatial attention feature map with the input feature map of this module. After multiplication, the final feature map is obtained with the scale unchanged size of 40*40*512. .

[0065] Feature map generated by CBAM structure The high-level features are further extracted through the backbone network to generate a 20*20*1024 feature map. Then, it is input into the Neck network for feature fusion. Feature map In the FPN network structure layer of the Neck section, a feature map A2 with a size of 20*20*1024 is generated through convolution operations. Feature map A2 is then upsampled to generate... Feature maps with the same scale are concatenated in the FPN network through a Concat operation to achieve the fusion of mid-level and high-level feature information, generating a fused feature map A1. A1 is then concatenated with feature map F1 through a Concat operation to achieve the fusion of low-level and mid-to-high-level feature information, resulting in feature map P1. Thus, the operation of passing high-level semantic feature information to low-level localization feature information and fusing them is completed in the FPN structure. Then, feature map P1 is divided into two branches. One branch is directly input into the Head layer for convolution and sigmoid activation to generate an 80*80*255 feature map to detect small targets. The other branch is input into the PAN structure, undergoes convolution and downsampling, and is concatenated with feature map A2 to fuse low-level and mid-level feature information, generating feature map P2. P2 is input into the Head layer to generate a 40*40*255 feature map to detect medium-sized targets. Simultaneously, it undergoes convolution and downsampling and is concatenated with A1 to fuse low-level, mid-level, and high-level feature information, resulting in feature map P3. This P3 is then input into the Head layer to generate a 20*20*255 feature map to detect large targets. Thus, the PAN structure completes the transfer of low-level localization feature information to high-level semantic feature information and performs feature fusion. Feature maps P1, P2, and P3 detect small, medium, and large targets respectively and count rapeseed flower clusters. At this point, YOLOv5-CBAM completes the detection and counting tasks for the rapeseed image.

[0066] 2.2 Evaluation Indicators

[0067] The deep learning environment used in this study was Ubuntu 20.04 with 128 GB of RAM and one NVIDIA GTX3090Ti graphics card. To test the stability of the model, the sample set was divided into training, testing, and validation sets in a 7:2:1 ratio, and the evaluation metrics were calculated after n training iterations. The evaluation metrics used in formulas (1)-(4) were used to evaluate the detection performance of the network model.

[0068] (1)

[0069] (2)

[0070] (3)

[0071] (4)

[0072] Among them, the network model's detection evaluation metrics include: Precision (the proportion of correctly identified positive samples out of all positive samples classified by the classifier); Recall (the proportion of correctly identified positive samples out of the actual number of positive samples); F1-score (the harmonic mean of precision and recall, combining the results of precision and recall, with values ​​ranging from 0 to 1, where 1 represents the best output and 0 represents the worst output); mAP (mAP) is used to measure the performance of the object detection algorithm; TP (True Positive) indicates that positive samples were classified as positive by the model, FP (False Positive) indicates that negative samples were classified as positive by the model, TN (True Negetive) indicates that negative samples were classified as negative by the model, and FN (False Negetive) indicates that positive samples were classified as negative by the model. C refers to the type of detection. In this invention, there is only one type of rapeseed flower cluster, so C is 1 here; N represents the number of images in the entire test set; P(k) represents the precision of the k-th recognized image; R(k) represents the recall change value from k-1 to k images.

[0073] The evaluation metrics of formulas (5)-(6) are used to evaluate the counting performance of the network model.

[0074] (5)

[0075] RMSE (6)

[0076] Among the evaluation metrics for model counting are... This represents the predicted value from the actual value network model. This represents the fitted value of the linear model. The coefficient of determination represents the average value. ), which is the goodness of fit of the regression model, represents the correlation between the network model's predicted values ​​and the actual values; the root mean square error (RMSE) represents the degree of deviation between the actual values ​​and the network model's predicted values.

[0077] 2.3 Network Training

[0078] To verify the effectiveness of this network model, the implementation example of this invention uses a dataset prepared on March 3rd. The augmented rapeseed dataset is divided into training, testing, and validation sets according to the classic deep learning dataset partitioning principle of 7:2:1. The iterative training and validation set loss results after adding the CBAM attention mechanism to the YOLOv5s network structure are shown below. Figure 5 As shown. By Figure 5 As can be seen, the training set loss decreases gradually with the increase of training epochs. After about 400 epochs, the loss is close to complete convergence. However, the validation set loss changes significantly in the early stage, with the loss rising and falling rapidly until it stabilizes and approaches convergence after 350 epochs.

[0079] 2.4 Ablation Experiment and Test Results Presentation

[0080] To verify the effectiveness of the CBAM attention mechanism in improving network detection, experiments were conducted on this dataset using four YOLOv5 network structures and the network model with the CBAM attention mechanism. The results are shown in Table 1. Table 1 shows that the detection method with the added attention mechanism improves both the precision and mAP (maximum accuracy) of rapeseed inflorescence detection compared to the original method. Specifically, YOLOv5x+CBAM achieved the highest coefficient of determination (COP) of 0.966 and the lowest root mean square error (RMSE) of 52.1%. This indicates that the rapeseed inflorescence density data obtained using this strategy can be used to assess the growth of rapeseed in the field and estimate its yield.

[0081] Table 1 Comparison of Evaluation Indicators

[0082]

[0083] The comparison chart of YOLOv5 detection before and after adding the attention mechanism is shown below. Figure 6 As shown in the figure, the purple and red rectangles represent the detection results at the same location of the original network and the network with the added attention mechanism, respectively. Figure 6 As can be seen, the original network detected several overlapping rapeseed flower clusters as one, missing the target. However, after adding the attention mechanism, the network model distinguished the overlapping rapeseed flower clusters, effectively detecting and counting them, thus reducing the missed detections.

[0084] Table 2 Comparison of different network evaluation metrics

[0085]

[0086] This invention compares the proposed method with the classic Faster-RCNN, adding a lightweight attention module structure to the YOLOv5 backbone network. This sacrifices a small amount of detection time but significantly improves the model's accuracy. Experimental results are shown in Figure 2, with quantitative results obtained using default parameter settings. The results show that the proposed method achieves an F1-Score of 88.7% and an mAP of 93.6%. Although CenterNet achieves the highest precision, its recall is relatively low, resulting in a low overall F1-Score. This phenomenon indicates that CenterNet, compared to the method used in this invention, suffers from a large number of missed detections. The proposed method improves the detection capability of dense, occluded target flower clusters, mitigating the missed detection of abundant rapeseed flower clusters and improving the network's accuracy in detecting and counting rapeseed flower clusters.

[0087] Example 2: An example of the application of rapeseed inflorescence density data obtained by an intelligent high-throughput method for measuring rapeseed inflorescence density in field applications for estimating rapeseed yield and assessing rapeseed growth / biomass.

[0088] 1. Application of rapeseed inflorescence density data in assessing rapeseed growth in a plot.

[0089] This invention is in Figure 7 (a) uses dashed lines of different colors to fit the changes in the number of rapeseed inflorescences in different plots during the flowering period. It can be found from the figure that the flowering time of each plot is inconsistent. Most plots flowered around February 18, 2021, and withered after March 20. The peak flowering time was around mid to late March of that year. Figure 7 (b) shows the temperature variation curves for the highest and lowest temperatures during the flowering period, representing the suitable temperatures for rapeseed growth. This invention randomly selected a small plot from field A. The total number of rapeseed inflorescences in the plot was calculated using the proposed method at six different time points. Figure 7 As shown in (c), the number of rapeseed inflorescences at six different stages were 53, 146, 387, 805, 605, and 0, respectively. The experiment demonstrates that the deep learning-based counting method proposed in this invention can provide accurate counting results, facilitating the quantitative analysis of growth.

[0090] 2. Examples of the application of inflorescence counting in field rapeseed

[0091] This invention utilizes drone imagery from Field A dated March 3, 2021, to conduct inflorescence counting tests. The experiment demonstrates that this method can quickly and quantitatively obtain the number of rapeseed inflorescences in different plots. The counting results are as follows: Figure 8As shown, the numbers marked on the red box and its 30 enlarged sub-images represent the corresponding count results. The results indicate that the automatic counting method is consistent with the observed rapeseed growth trend. The automatic counting method quantifies the inflorescence variation of different varieties during flowering, providing breeders with reliable phenotypic data on inflorescence count.

[0092] 3. Examples of the application of inflorescence counting results in yield correlation analysis

[0093] This invention utilizes representative plots, including two plots of different sizes, to perform regression analysis on the total number of rapeseed inflorescences and the sum and maximum seed yield of each plot over five periods. The total number of rapeseed inflorescences in each plot is used as the independent variable for the network model's predicted value, and the sum of seed yield in each plot is used as the dependent variable for the actual value. The least squares method is used to calculate a fitted value that continuously approximates the actual value, thereby establishing a linear regression model between the number of inflorescences and yield in each plot. Figure 9 As shown, the coefficient of determination is calculated using formula (5). The value of , which ranges from 0 to 1, is based on The magnitude of the value indicates the strength of the correlation; a larger value indicates a stronger correlation. The results demonstrate a strong correlation between the inflorescence count results and the yield of the corresponding plots. The value reached 0.4418, indicating a positive correlation between the number of inflorescences and the final yield.

[0094] Those skilled in the art can easily employ known similar strategies and methods, such as using drones of other brands or models, image stitching and cropping software with similar functions, or other deep learning-based algorithms with similar functions to calculate the inflorescence density of rapeseed crops, all of which fall within the scope of protection of this patent.

Claims

1. A method for intelligently measuring high-flux inflorescence density of oilseed rape in the field, characterized in that, Includes the following steps: Step 1: Use an airborne RGB camera on a drone to capture images of rapeseed in the field during the flowering period to obtain inflorescence image data. The flowering period refers to the period from when the main inflorescence of the rapeseed plant has several flower buds and petals unfolded after the plant bolts until the end of the flowering period. The number of inflorescences is counted based on randomly sampled areas in the plot or field. Throughout the flowering period, images of the rapeseed field flowering status are captured at intervals. Step 2: A deep learning-based rapeseed inflorescence counting strategy, including: 1) The problem of counting the number of inflorescences is identified as an object detection problem, and a deep learning-based object detection strategy is used to count the inflorescences; 2) To extract and count the features of rapeseed inflorescences / floral clusters, we selected a convolutional neural network that combines UAV RGB imagery with an attention mechanism. 3) A YOLOv5 network model combining CBAM attention mechanism is used to input a rapeseed flower image and output the rapeseed inflorescence detection box and the inflorescence count result. Feature map generated by CBAM structure The high-level features are further extracted through the backbone network to generate a 20*20*1024 feature map. Then, it is input into the Neck network for feature fusion; feature map In the FPN network structure layer of the Neck section, a feature map A2 with a size of 20*20*1024 is generated through convolution operations. Feature map A2 is then upsampled to generate... Feature maps of the same scale are concatenated in the FPN network through a concat operation to fuse mid- and high-level feature information, generating a fused feature map A1. A1 then undergoes convolutional upsampling and is concatenated with feature map F1 to fuse low- and mid-to-high-level feature information, resulting in feature map P1. This completes the process of passing high-level semantic feature information to low-level localization feature information and fusing them within the FPN structure. Feature map P1 is then divided into two branches: one branch is directly input into the Head layer for convolution and sigmoid activation to generate an 80*80*255 feature map for small target detection; the other branch is input into the PAN structure, undergoes convolution and downsampling, and is concatenated with feature map A2 through a concat operation. The ncat concatenation operation fuses low-level and mid-level feature information to generate feature map P2. P2 is input to the Head layer to generate a 40*40*255 feature map to detect medium-sized targets. On the other hand, it continues to perform convolutional downsampling operations and concat concatenates with A1 to fuse low-level, mid-level, and high-level feature information to obtain feature map P3. P3 is then input to the Head layer to generate a 20*20*255 feature map to detect large targets. Thus, in the PAN structure, the low-level localization feature information is transferred to the high-level semantic feature information and fused. The feature maps P1, P2, and P3 respectively detect small, medium, and large targets and count rapeseed flower clusters. At this point, YOLOv5-CBAM completes the detection and counting tasks of the rapeseed image. 4) Train and optimize the network model, use a large amount of image data to measure the running rate, and test its computing speed and throughput.

2. The method for intelligent high-throughput measurement of rapeseed inflorescence density in the field according to claim 1, characterized in that, In step 1, rapeseed includes Brassica napus, Chinese cabbage rapeseed, and mustard rapeseed, as well as three ecological types of rapeseed suitable for planting in various production areas: semi-winter type, spring type, and winter type.

3. The method for intelligent high-throughput measurement of rapeseed inflorescence density in the field according to claim 1, characterized in that, In step 1, the image overlap rate of the target area of ​​the rapeseed field should be set according to the requirements of the corresponding image stitching software / program in the later stage.

4. The method for intelligent high-throughput measurement of rapeseed inflorescence density in the field according to claim 1, characterized in that, In step 1, a large-scale dataset of rapeseed inflorescences from drones is established, containing tens of thousands of rapeseed inflorescence samples and rapeseed inflorescence annotation information.

5. The method for intelligent high-throughput measurement of rapeseed inflorescence density in the field according to claim 1, characterized in that, In step 2, the evaluation metrics of formulas (1)-(4) are used to evaluate the detection performance of the network model: （1）， （2）， （3）， （4）， In the formula: Precision represents the precision of the network model detection evaluation index, that is, the proportion of correctly identified positive samples to the total number of positive samples classified by the classifier; Recall represents the recall, that is, the proportion of correctly identified positive samples to the actual number of positive samples; F1-score is the harmonic mean of precision and recall, which combines the results of precision and recall, and its value ranges from 0 to 1; mAP is used to measure the performance of the object detection algorithm; TP indicates that positive samples are classified as positive samples by the model, FP indicates that negative samples are classified as positive samples by the model, TN indicates that negative samples are classified as negative samples by the model, and FN indicates that positive samples are classified as negative samples by the model; C refers to the type of detection; N represents the number of images in the entire test set; P(k) represents the precision of the k-th identified image; R(k) represents the change in recall from k-1 to k images. The evaluation metrics of formulas (5)-(6) are used to assess the counting performance of the network model: （5）， （6）， In the formula: The evaluation index for model counting This represents the predicted value from the actual value network model. This represents the fitted value of the linear model. This represents the average value; This indicates the correlation between the network model's predicted values ​​and the actual values; RMSE indicates the degree of deviation between the actual values ​​and the network model's predicted values.

6. An application of rapeseed inflorescence density data obtained by the method according to any one of claims 1-5 in estimating rapeseed yield and assessing rapeseed growth / biomass, comprising the following steps: 1) Obtain inflorescence density data of rapeseed plots using the established model, including inflorescence density during the entire flowering period and at various time points; 2) Obtain yield data of rapeseed plots through yield measurement, and obtain important traits related to branch number and yield, as well as total biomass data at harvest through variety testing. Rapeseed growth level can be assessed by setting up control scores during the seedling stage. 3) Using the two sets of data mentioned above, a correlation analysis was conducted with Excel software. The high correlation results indicate that the obtained data can be used to assess rapeseed growth and biomass, and to predict yield per unit area.

Images