Method for identifying and counting alfalfa seed pods with different maturity degrees based on RHE-DETR

Abstract

The invention discloses a method for identifying and counting alfalfa seed pods with different maturity degrees based on RHE-DETR, and belongs to the technical field of target detection. The method comprises the following steps: acquiring alfalfa seed pod images under different illumination conditions and shielding conditions in a field; labeling software is used for labeling the collected images, and an alfalfa seed pod data set is constructed and divided; the model is improved, and a lightweight target detection model RHE-DETR for a small target is obtained; inputting the constructed alfalfa seed pod data set into an RHE-DETR target detection model for training to obtain a weight file of the model after the alfalfa seed pod data set is trained; and deploying a weight file trained by the RHE-DETR target detection model to an edge computing platform to detect a field alfalfa seed pod image to obtain a real-time tracing frame, confidence and counting result of the alfalfa seed pod. According to the method, the conditions of error detection and missing detection in a complex field environment are reduced, the accuracy of identifying and counting the maturity of the alfalfa seed pods is improved, and meanwhile, the identifying and counting time is also shortened.

Description

Technical Field

[0001] The invention belongs to the technical field of target detection, and in particular relates to a method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR. Background Art

[0002] Alfalfa is a perennial flowering plant, and its seed maturity period is an important growth stage in the growth cycle. Only when the seeds are fully mature, they have the highest germination rate and growth potential, which can ensure the normal growth and reproduction of the next generation and provide strong support for the sustainable development of animal husbandry. Therefore, in the process of alfalfa planting and management, it is very important to accurately judge the maturity of seeds. At present, the judgment of the maturity of alfalfa seeds mostly relies on manual field investigation, which is affected by the complex environment of the field. Such a method will waste human resources and time costs, and is also easily affected by the uneven maturity of seeds in the field, resulting in inaccurate judgment, which in turn affects the yield and commercial value of alfalfa seeds.

[0003] Nowadays, target detection technology has been widely used in the agricultural field. However, due to the complex field environment, the overlap between alfalfa pods, and the small target characteristics of alfalfa pods, the existing target detection algorithms have increased the difficulty of identifying and counting the maturity of alfalfa pods.

[0004] Therefore, in view of the complex field environment and the small size of alfalfa seed pods, how to improve the accuracy of alfalfa seed pod maturity identification and counting is a difficult problem to be solved in this field. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR, which improves the accuracy of identifying and counting the maturity of alfalfa seed pods.

[0006] The present invention is achieved by adopting the following technical solutions:

[0007] A method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR, the method comprising the following steps:

[0008] Collect images of alfalfa seed pods in the field under different lighting conditions and occlusion;

[0009] Use annotation software to annotate the collected images, construct and divide the alfalfa seed pod dataset;

[0010] The RT-DETR model is improved based on the RepVi T backbone network structure, HS-FPN multi-scale feature fusion network structure, InnerWI OU loss function and ELA module to obtain the lightweight target detection model RHE-DETR for small targets;

[0011] Input the constructed alfalfa seed pod dataset into the RHE-DETR target detection model for training, and obtain the weight file of the model after the alfalfa seed pod dataset training;

[0012] The trained weight file of the RHE-DETR target detection model is deployed on the edge computing platform to detect field alfalfa seed pod images, and the real-time delineation, confidence and counting results of alfalfa seed pods are obtained.

[0013] Furthermore, the method for constructing and dividing the alfalfa seed pod dataset includes: performing preprocessing operations such as sharpening the collected images and enhancing the image contour details, annotating the images using LabelImg software, and expanding the dataset; dividing the images into a training set, a validation set, and a test set in a ratio of 8:1:1 to complete the construction of the dataset.

[0014] Furthermore, expanding the data set includes: using the Mixup method for data enhancement, the core formula of which is as follows:

[0015]

[0016]

[0017] Among them, x i and x j are two images randomly selected from the training set, y i and j is the corresponding one-hot label, and λ∈(0,1) is the linear combination coefficient, which is randomly generated each time it is executed.

[0018] Further improvements to the lightweight target detection model RHE-DETR for small targets include:

[0019] The lightweight RepViT network structure is used as the backbone, which combines the RepViTBlock and C3 modules to improve the model performance;

[0020] In Neck, a multi-level feature fusion pyramid HS-FPN is used to process multi-scale feature fusion, and the channel attention module CA in the feature selection module is replaced with an efficient local attention module ELA;

[0021] The loss function is replaced with an auxiliary bounding box-based bounding box regression loss InnerWIoU to improve the generalization ability of the model.

[0022] Furthermore, in Neck, multi-level feature fusion pyramid HS-FPN is used to process multi-scale feature fusion, including:

[0023] The selected low-level features and high-level features are combined through the selective feature fusion mechanism. The fusion process of feature selection is as follows:

[0024] f att =BL(T-Conv(f high ))

[0025] f out =f low *ELA(f att )+f att .

[0026] Furthermore, the ELA adopts a novel encoding method to generate an accurate position attention map. The output process of ELA is as follows:

[0027] y h =σ(G n (F h (z h )))

[0028] y w =σ(G n (F w (z w )))

[0029] Y = x c ×y h ×y w

[0030] Among them, σ represents the nonlinear activation function, G n Used to process enhanced location information, F h and F w represents one-dimensional convolution, z h and z w represents the output at height h and width w, respectively, x c Indicates the size of the input feature, the horizontal and vertical position attention are denoted by y h and w express.

[0031] Furthermore, a scale factor ratio is introduced into the Inner-WIoU, which can control the scale of the auxiliary bounding box. By using auxiliary bounding boxes of different scales for different data sets and detectors, the limitations of the existing methods in generalization ability can be overcome. The definition of the Inner-WIoU is as follows:

[0032]

[0033]

[0034]

[0035]

[0036]

[0037] union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter

[0038]

[0039] The groundtruth (GT) box and the anchor box are represented as B gt and B, the center point of the GT box and the center point of the inner GT box are indicates that (x c ,y c ) represents the center point of the Anchor box and the inner Anchor box, and the width and height of the GT box are w gt and h gt Indicates that the width and height of the Anchor box are represented by w and h, and the variable "ratio" corresponds to the scaling factor, usually in the range of [0.5, 1.5].

[0040] Furthermore, the RHE-DETR target detection model selects precision, recall and mean average precision (mAP) as evaluation indicators of model performance;

[0041] The Precision indicates the proportion of the correct number of alfalfa pods detected by the model:

[0042]

[0043] The recall represents the ratio of the number of alfalfa pods detected by the model to the actual number of alfalfa pods:

[0044]

[0045] The mAP represents the average precision of the model:

[0046]

[0047] Among them, TP represents the number of correctly detected targets, FN represents the number of incorrectly detected targets, FP represents the number of undetected targets, N is the number of categories, AP i is the average precision of the second category. When N=1, mAP=AP.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The invention proposes a method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR, which reduces the misdetection and missed detection in complex field environments, improves the accuracy of identifying and counting the maturity of alfalfa seed pods, and also reduces the time for identification and counting. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of the present invention.

[0051] Figure 2 This is a diagram of the improved HS-FPN network structure in the present invention.

[0052] Figure 3 It is a structural diagram of the ELA attention module in the present invention.

[0053] Figure 4 It is a schematic diagram of the principle of Inner-WI oU definition in the present invention.

[0054] Figure 5 It is a schematic diagram of outputting the counting result of a certain frame in the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the examples of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] like Figure 1 As shown, Figure 1 The present invention proposes a method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR, comprising the following steps:

[0057] S1. Collect images of alfalfa seed pods in the field under different lighting conditions and occlusion conditions, as follows:

[0058] The DJI M30 drone was used for data collection. The drone's flight altitude and speed were 7m and 5m / s respectively. The wind force generated by the drone when hovering was between 1.6-3.3m / s. The images were taken in a vertical orthophoto manner. A total of 12,000 images of alfalfa seed pod clusters were collected with an image resolution of 4000 pixels × 3000 pixels, including images under different lighting conditions and occlusion conditions such as front light, back light, with occlusion and without occlusion, which can make the model better adapt to various environmental conditions and scenes, and improve the robustness and adaptability of the model.

[0059] S2. Use annotation software to annotate the collected images and construct and divide the alfalfa seed pod dataset as follows:

[0060] Use openCV-python to sharpen the collected images, enhance the image contour details and other preprocessing operations, use LabelImg software to annotate the images, and expand the data set; divide the images into training set, validation set and test set in a ratio of 8:1:1 to complete the construction of the data set.

[0061] Furthermore, the data set expansion includes: using a Mixup method to perform data enhancement;

[0062] Mixup is a simple but powerful data enhancement method. Its core idea is to linearly combine two random samples in a certain proportion to generate new training samples. This method can effectively expand the distribution of training data and improve the generalization ability and robustness of the model. Its core formula is as follows:

[0063]

[0064]

[0065] Among them, x i and x j are two images randomly selected from the training set, y i and j is the corresponding one-hot label, and λ∈(0,1) is the linear combination coefficient, which is randomly generated each time it is executed.

[0066] S3. Based on the RepViT backbone network structure, HS-FPN multi-scale feature fusion network structure, InnerWIoU loss function and ELA module, the RT-DETR model is improved to obtain the lightweight target detection model RHE-DETR for small targets, as follows:

[0067] (1) The Backbone of RT-DETR is replaced with a lightweight RepViT network structure. This structure combines the RepViTBlock and C3 modules to improve model performance. This structure is an advanced visual Transformer architecture that combines modular design with local feature extraction capabilities. It enhances the effect of image content modeling through self-attention mechanism and multi-scale feature processing. At the same time, it adopts a pre-training strategy to improve performance and efficiency in various visual tasks. Although RepVi T is a lightweight model, it extracts comprehensive feature information of feature maps and can effectively capture alfalfa seed pod features.

[0068] (2) In Neck, a multi-level feature fusion pyramid HS-FPN is used to process multi-scale feature fusion. The selected low-level features and high-level features are combined through the selective feature fusion (SFF) mechanism to improve the detection performance of alfalfa seed pod small targets. Figure 2 As shown, its basic principle includes two key parts: feature selection module and feature fusion module;

[0069] The selective feature fusion module is a key component in the HS-FPN network. Its main function is to fuse feature maps of different scales. SFF uses high-level features as weights to filter important information in low-scale features. In this process, high-level features are resized through transposed convolution and bilinear interpolation operations to match the size of low-scale features. Then, high-level features are used as attention weights to filter out useful information in low-scale features. This fusion method can effectively combine the semantic information of high-level features and the detailed information of low-scale features, thereby improving the performance of the model in dealing with multi-scale problems. The fusion process of feature selection is as follows:

[0070] f att =BL(T-Conv(f high ))

[0071] f out =f low *ELA(f att )+f att .

[0072] (3) The channel attention module CA in the HS-FPN feature selection module is replaced with an efficient local attention module ELA. The structure of the attention module ELA is as follows: Figure 3 As shown in the figure, it enhances the ability to capture details by emphasizing important local areas in the image while maintaining computational efficiency, thereby improving the accuracy of the model in identifying and counting alfalfa pods. The output process of ELA is as follows:

[0073] y h =σ(G n (F h (z h )))

[0074] y w =σ(G n (F w (z w )))

[0075] Y = x c ×y h ×y w

[0076] Among them, σ represents the nonlinear activation function, G nUsed to process enhanced location information, F h and F w represents one-dimensional convolution, z h and z w represents the output at height h and width w, respectively, x c Indicates the size of the input feature, the horizontal and vertical position attention are denoted by y h and w express.

[0077] (4) The loss function is replaced with an auxiliary bounding box regression loss I nner-WIoU to improve the generalization ability of the model; Figure 4 As shown in Figure 1, in Inner-WIoU, a scale factor ratio is introduced to control the scale of the auxiliary bounding box. By using auxiliary bounding boxes of different scales for different data sets and detectors, the limitations of existing methods in generalization ability can be overcome. The definition of Inner-WIoU is as follows:

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter

[0084]

[0085] The ground truth (GT) box and the anchor box are represented as B gt and B, the center point of the GT box and the center point of the inner GT box are indicates that (x c ,y c ) represents the center point of the Anchor box and the inner Anchor box, and the width and height of the GT box are w gt and h gt Indicates that the width and height of the Anchor box are represented by w and h, and the variable "ratio" corresponds to the scaling factor, usually in the range of [0.5, 1.5].

[0086] S4. Input the constructed alfalfa seed pod dataset into the RHE-DETR target detection model for training, and obtain the weight file of the model after the alfalfa seed pod dataset is trained.

[0087] In terms of model evaluation, the RHE-DETR target detection model selects precision, recall and mean average precision (mAP) as evaluation indicators of model performance.

[0088] The Precision indicates the proportion of the correct number of alfalfa pods detected by the model:

[0089]

[0090] The recall represents the ratio of the number of alfalfa pods detected by the model to the actual number of alfalfa pods:

[0091]

[0092] The mAP represents the average precision of the model:

[0093]

[0094] Among them, TP represents the number of correctly detected targets, FN represents the number of incorrectly detected targets, FP represents the number of undetected targets, N is the number of categories, AP i is the average precision of the second category. When N=1, mAP=AP.

[0095] S5. Deploy the trained RHE-DETR target detection model weight file to the edge computing platform to detect alfalfa seed pods in the field and obtain the real-time delineation, confidence and counting results of alfalfa seed pods.

[0096] The edge computing equipment, camera, screen, power supply and other equipment are installed on the mobile platform. The edge computing platform TW-T506 equipped with the trained RHE-DETR target detection model can detect and count alfalfa seed pod images in real time. The counting result of a certain frame is as follows: Figure 5 shown.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR, characterized in that: The method comprises the following steps: Collect images of alfalfa seed pods in the field under different lighting conditions and occlusion; Use annotation software to annotate the collected images, construct and divide the alfalfa seed pod dataset; The RT-DETR model is improved based on the RepViT backbone network structure, the HS-FPN multi-scale feature fusion network structure, the InnerWIoU loss function and the ELA module to obtain the lightweight target detection model RHE-DETR for small targets. Input the constructed alfalfa seed pod dataset into the RHE-DETR target detection model for training, and obtain the weight file of the model after the alfalfa seed pod dataset training; The trained weight file of the RHE-DETR target detection model is deployed on the edge computing platform to detect field alfalfa seed pod images, and the real-time delineation, confidence and counting results of alfalfa seed pods are obtained.

2. The method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR according to claim 1, characterized in that: The method for constructing and dividing the alfalfa seed pod dataset includes: sharpening the collected images, enhancing the image contour details and other preprocessing operations, using Label Img software to annotate the images, and expanding the dataset; dividing the images into training set, validation set and test set in a ratio of 8:1:1 to complete the construction of the dataset.

3. The method for identifying and counting alfalfa pods of different maturity based on RHE-DETR according to claim 2, characterized in that: Expanding the data set includes using the Mixup method to enhance the data. The core formula is as follows: Among them, x i and x j are two images randomly selected from the training set, y i and j is the corresponding one-hot label, and λ∈(0,1) is the linear combination coefficient, which is randomly generated each time it is executed.

4. The method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR according to claim 1, characterized in that: Improvements to the lightweight target detection model RHE-DETR for small targets include: The lightweight RepViT network structure is used as the backbone, which combines the RepViTBlock and C3 modules to improve the model performance; In Neck, a multi-level feature fusion pyramid HS-FPN is used to process multi-scale feature fusion, and the channel attention module CA in the feature selection module is replaced with an efficient local attention module ELA; The loss function is replaced with an auxiliary bounding box-based bounding box regression loss InnerWIoU to improve the generalization ability of the model.

5. The method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR according to claim 4, characterized in that: Using multi-level feature fusion pyramid HS-FPN in Neck to process multi-scale feature fusion includes: The selected low-level features and high-level features are combined through the selective feature fusion mechanism. The fusion process of feature selection is as follows: f att =BL(T-Conv(f high )) f out =f low *ELA (f att )+f att .

6. The method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR according to claim 4, characterized in that: The ELA adopts a novel encoding method to generate accurate position attention maps. The output process of ELA is as follows: y h =σ(G n (F h (z h ))) y w =σ(G n (F w (z w ))) Y=x c ×y h ×y w Among them, σ represents the nonlinear activation function, G n Used to process enhanced location information, F h and F w represents one-dimensional convolution, z h and z w represents the output at height h and width w, respectively, x c Indicates the size of the input feature, the horizontal and vertical position attention are denoted by y h and w express.

7. The method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR according to claim 4, characterized in that: The Inner-WIoU introduces a scale factor ratio, which can control the scale of the auxiliary bounding box. By using auxiliary bounding boxes of different scales for different data sets and detectors, the limitations of existing methods in generalization ability can be overcome. The definition of Inner-WIoU is as follows: union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter The ground truth (GT) box and the anchor box are represented as B gt and B, the center point of the GT box and the center point of the inner GT box are indicates that (x c ,y c ) represents the center point of the Anchor box and the inner Anchor box, and the width and height of the GT box are w gt and h gt Indicates that the width and height of the Anchor box are represented by w and h, and the variable "ratio" corresponds to the scaling factor, usually in the range of [0.5,1.5].

8. The method for identifying and counting alfalfa seed pods of different maturity based on RHE-DETR according to claim 1, characterized in that: The RHE-DETR target detection model selects precision, recall and mean average precision (mAP) as evaluation indicators of model performance; The Precision indicates the proportion of the correct number of alfalfa pods detected by the model: The recall represents the ratio of the number of alfalfa pods detected by the model to the actual number of alfalfa pods: The mAP represents the average precision of the model: Among them, TP represents the number of correctly detected targets, FN represents the number of incorrectly detected targets, FP represents the number of undetected targets, N is the number of categories, AP i is the average precision of the second category. When N=1, mAP=AP.

Images