A blueberry counting and yield estimation method and system for complex field environment
By improving the YOLOv8n detection and classification model and the OC-SORT tracking algorithm, and combining appearance re-identification and camera motion compensation, the accuracy and robustness issues of blueberry detection and classification in complex field environments were solved, achieving high-precision blueberry counting and yield prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUIZHOU UNIV
- Filing Date
- 2026-04-22
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies lack accuracy and robustness in blueberry detection and classification under complex field conditions, especially when blueberry fruits are densely packed, severely shaded, subject to large changes in light intensity, or when there is interference from camera movement, leading to inaccurate counting results.
An improved YOLOv8n detection and classification model is adopted, combined with the OC-SORT tracking algorithm that integrates appearance re-identification and camera motion compensation. Fine-grained maturity classification is performed by fusing color, texture and shape features, and a dual-region counting method is designed to ensure the unique identification and accurate counting of each blueberry target.
It significantly improves the accuracy and robustness of blueberry detection and classification, reduces the frequency of ID switching, ensures the accuracy and reliability of counting results, and achieves high-precision yield prediction.
Smart Images
Figure CN122116149A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of agricultural computer vision technology, and in particular relates to a method and system for blueberry counting and yield prediction in complex field environments. Background Technology
[0002] As a high-value crop, accurate yield forecasting of blueberries is crucial for optimizing field management, planning harvest schedules, allocating labor, and developing market strategies. Traditional yield forecasting relies mainly on manual sampling and experience-based judgment. This method is not only time-consuming, labor-intensive, and costly, but its results are also susceptible to subjectivity and sample randomness, making it difficult to meet the accuracy requirements of modern precision agriculture.
[0003] With the development of artificial intelligence technology, various techniques have been introduced into the agricultural field, providing new solutions for automated fruit counting and classification. The mainstream approach typically employs object detection algorithms, such as the YOLO (You Only Look Once) series, to locate the fruit, combined with multi-object tracking (MOT) algorithms (such as DeepSORT or ByteTrack) to track each fruit in a video sequence, thereby achieving counting. Classification and recognition are mainly performed using convolutional neural networks (CNNs) to recognize images. However, directly applying these general techniques to the complex blueberry field environment faces four major challenges, significantly reducing the performance and reliability of existing methods: 1. Limitations of Detection and Classification: Blueberry berries are characterized by their small size and dense growth, often forming clusters with significant mutual shading between berries and between berries and foliage. Standard detection models are prone to missed detections and false positives in this complex environment. Furthermore, yield prediction requires not only quantity but also an understanding of fruit maturity distribution. Blueberries exhibit significant differences in color, texture, and shape at different growth stages (e.g., flowering, green fruit, veraison, and ripening), and variable field lighting conditions severely interfere with classification accuracy. Existing methods typically treat detection and classification as separate steps, resulting in inefficiency and difficulty in addressing the challenges posed by varying light conditions.
[0004] 2. ID Switching Issues During Tracking: In dynamic video counting, assigning and maintaining a unique ID for each fruit is crucial for accurate counting. However, in the dense blueberry growing environment, frequent occlusion, similar fruit appearances, and camera movement caused by the movement or shaking of the inspection equipment can severely interfere with the tracking algorithm. This can easily lead the tracker to incorrectly switch the ID of the same fruit between different frames, or mistakenly identify different fruits as the same.
[0005] 3. Camera motion interference: Field data acquisition typically involves moving shooting platforms, which introduces global camera motion. Traditional tracking algorithms are mostly based on the assumption of a static camera, and their built-in motion prediction models (such as Kalman filters) cannot effectively distinguish between the target's own motion and the camera's background motion, causing the predicted trajectory to deviate significantly from the actual position, thus exacerbating matching failures and ID switching problems.
[0006] 4. Vulnerability of Counting Logic: ID switching is a direct cause of counting errors (double counting or missed counting). Many existing counting methods, such as simply counting the largest ID number or defining a single counting line, are highly sensitive to ID switching and have poor robustness. Once ID switching occurs, these simple counting logics will generate huge cumulative errors, rendering the final production forecast unreliable.
[0007] Therefore, there is an urgent need for a comprehensive technical solution that can systematically and collaboratively address the above problems in order to achieve high-precision and robust prediction of blueberry yield in complex field environments. Summary of the Invention
[0008] The purpose of this invention is to provide a method and system for blueberry counting and yield prediction in complex field environments, thereby solving the problems existing in the prior art.
[0009] To achieve the above objectives, this invention provides a method for blueberry counting and yield prediction in complex field environments, comprising the following steps: S1. Collect videos of blueberries in the field as a dataset, label the blueberry targets in the dataset, and perform data augmentation on the dataset; S2. Construct an improved YOLOv8n detection and classification model that integrates fine-grained maturity classification. Input the blueberry video into the improved YOLOv8n detection and classification model, and obtain the detection results containing blueberry target location information and maturity level after frame-by-frame processing. S3. Construct an improved OC-SORT tracking algorithm that integrates appearance re-identification mechanism and camera motion compensation. Use the detection results as input to the improved OC-SORT tracking algorithm to obtain the tracking trajectory of blueberry targets at different maturity levels. S4. Based on the dual-region counting method, the tracking trajectory is statistically analyzed to obtain the counting results of blueberries at each maturity level.
[0010] Preferably, the process of constructing the dataset includes: a) Video data acquisition: Portable high-definition camera equipment was used to record videos along the blueberry planting rows at a stable moving speed to obtain high-resolution raw video sequences; b) Video frame cutting and annotation: Image frames are extracted from the high-resolution original video sequence at preset frame intervals, and bounding boxes are annotated for each visible blueberry fruit in the frame according to its maturity level. c) High-resolution image slicing: To avoid loss of small target feature information due to direct image scaling, the labeled high-resolution image frame and its corresponding label file are divided into multiple smaller image blocks suitable for the model input size, and it is ensured that the label boxes of any individual blueberry targets are not cut off during the slicing process. d) Dataset partitioning: All processed image patches and their annotation information are randomly divided into training set, validation set and test set according to a preset ratio.
[0011] Preferably, the construction of the improved YOLOv8n detection and classification model includes: The Mamba-C2f module is integrated into the backbone network of the YOLOv8n model to enhance feature extraction capabilities, and three parallel classification heads—color, texture, and shape—are added to its detection head. Specifically, the color classification head converts input image patches from the RGB color space to the HSV color space and classifies them based on the H (hue) channel histogram of the HSV color space; the texture classification head classifies based on the descriptor of the Gray-Level Co-occurrence Matrix (GLCM); and the shape classification head classifies based on the geometric descriptors of the fruit's outline (such as roundness and aspect ratio).
[0012] Preferably, the improved OC-SORT tracking algorithm, which integrates appearance re-identification (Re-ID) and camera motion compensation (CMC), specifically includes: An appearance re-identification mechanism is used to extract a depth feature vector as the appearance identifier for each detected blueberry target. When a target reappears after being lost due to occlusion, the correct ID of the target is recovered by comparing the appearance feature vectors of the newly detected target with those of the historical trajectory. An exponential moving average (EMA) method is then used to smoothly update the appearance features of the trajectory; the calculation formula is as follows: ; In the formula, For the first The trajectory at the th Smoothed appearance feature vector at frame time. For the first The trajectory at the th Historical appearance feature vector at frame time, For the first The trajectory at the th The original appearance features of the detection box matched at frame time. It is a smoothing factor; By using the camera motion compensation mechanism, an affine transformation matrix is estimated by analyzing the global pixel displacement between consecutive frames, and the affine transformation matrix is used to correct the position deviation of the tracking trajectory prediction caused by camera movement or jitter.
[0013] Preferably, the process of using the detection results as input to the improved OC-SORT tracking algorithm to obtain the tracking trajectory of blueberry targets at different maturity levels is as follows: a) State prediction and correction: Using the affine transformation matrix estimated by camera motion compensation, the state of all existing tracking trajectories predicted by the Kalman filter is corrected. b) Construction of the comprehensive cost matrix: A comprehensive cost matrix is constructed for the detection box and the corrected tracking trajectory of the current frame. The cost value of this comprehensive cost matrix integrates the following three metrics: i. Appearance similarity based on appearance re-identification; ii. Consistency of motion based on observation-centric momentum; iii. Spatial overlap based on intersection-union ratio; c) Association matching: The Hungarian algorithm is used to solve the comprehensive cost matrix to achieve the optimal matching between the detection box and the tracking trajectory; d) State recovery and update: For trajectories that were lost in previous frames due to occlusion but were rematched in the current frame, an observation-centered re-update is performed, and their Kalman filter parameters are smoothly corrected by generating virtual trajectories; for trajectories that failed to be associated in the first round of matching, an observation-centered recovery is performed, and a secondary association is attempted with the remaining unmatched detection boxes; e) Track Management: Based on the matching results, create new tracks, update existing tracks, or terminate tracks that have not been matched for a long time.
[0014] Preferably, the Kalman filter employs the following state equation for the prediction and update steps after considering camera motion correction: Prediction steps: ; ; Update steps: ; ; ; in, Indicates based on the first Frame information for the first The predicted prior state estimate of the frame. This represents the rotation and scaling matrix from frame k-1 to frame k. Indicates the first The posterior state estimate after frame update. This represents the translation matrix from frame k-1 to frame k. Indicates the first The prior state covariance matrix of the frame. Indicates the first The posterior state covariance matrix after frame update, superscript Representation matrix transpose Indicates the first The covariance matrix of the process noise of the frame. Indicates the first The observation matrix of the frame, Indicates the first The covariance matrix of the observation noise of the frame. Indicates the first Kalman gain of the frame Indicates the first The posterior state estimate of the frame after observation and update. Indicates the first Frame observations, Indicates the first The posterior state covariance matrix of the frame after observation and update. Represents the identity matrix.
[0015] Preferably, the dual-region counting method includes: In the video frame, a confidence judgment area and a counting area are set, where the confidence judgment area is denoted as the first area and the counting area is denoted as the second area; When a blueberry target's tracking trajectory first enters the first area, the system records its unique ID and stores it in a temporary ID cache list; When the tracking trajectory continues to move and enters the second area, the system verifies again whether its ID exists in the temporary ID cache list. If it exists, the system increments the counter of the corresponding category according to the current maturity level of the tracking trajectory and removes the ID from the cache list to ensure that each target is counted only once.
[0016] A blueberry counting and yield prediction system for complex field environments includes: The data processing module is used to collect, label, and enhance blueberry video data in the field, and to build the dataset required for training. The detection and classification module has a built-in improved YOLOv8n detection and classification model, which is used to receive video frames and output the location and fine-grained maturity level of the blueberry target. The tracking module incorporates an improved OC-SORT tracking algorithm for associating and maintaining the IDs of blueberry targets across frames; The counting and output module has a built-in dual-region counting method for accurately counting the tracking trajectory and finally outputting the blueberry quantity statistics for each maturity level.
[0017] Therefore, the present invention employs the above-mentioned method and system for blueberry counting and yield prediction in complex field environments, which has the following beneficial effects: 1. Integrated Detection and Classification: Enhancing Both Accuracy and Efficiency: By directly integrating multi-head fine-grained maturity classification functionality into the YOLOv8n detection model, a one-step simultaneous processing method is achieved for blueberry target location detection and maturity level determination. This design not only improves processing efficiency but also significantly enhances the robustness of the classification algorithm under complex lighting conditions by fusing multi-dimensional features such as color, texture, and shape, laying a solid foundation for accurate yield grading and prediction.
[0018] 2. Deep Fusion of Appearance and Motion Information to Effectively Suppress ID Switching: The OC-SORT tracking algorithm has been significantly improved by integrating appearance re-identification (Re-ID) and camera motion compensation (CMC). The Re-ID mechanism provides the algorithm with powerful appearance discrimination capabilities, enabling it to accurately identify and recover the correct ID of the target even after prolonged occlusion; the CMC mechanism actively eliminates interference caused by camera movement, ensuring the accuracy of motion prediction. The combination of these two mechanisms greatly reduces the frequency of ID switching, significantly improving the continuity and stability of tracking.
[0019] 3. Robust dual-region counting logic ensures the accuracy of counting results: A dual-region counting method is designed, which adds a reliable verification step to the counting process through a "confirm first, then count" mechanism. This design can effectively filter out false trajectories caused by brief false detections or tracking instability, and ensure that each real target is counted exactly once in the entire video sequence, thereby fundamentally guaranteeing the high accuracy and reliability of the final yield prediction results.
[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0021] Figure 1 This is a schematic diagram of the overall process of a blueberry counting and yield prediction method for complex field environments proposed in this invention. Figure 2 This is a schematic diagram of the data acquisition and processing flow in this invention; Figure 3 This is a schematic diagram of the network structure of the improved YOLOv8n detection and classification model in this invention; Figure 4This is a flowchart illustrating the improved OC-SORT tracking algorithm and dual-region counting method in this invention. Detailed Implementation
[0022] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0023] Please see Figure 1 A method for blueberry counting and yield prediction in complex field environments includes the following steps: S1: Data acquisition and processing; Reference Figure 2 This step involves collecting videos of blueberries in the field as a dataset, extracting frames, labeling, and augmenting data on the blueberry targets in the dataset, and constructing a high-quality, diverse blueberry dataset to support the training of subsequent models.
[0024] a) Video Data Acquisition: Portable high-definition video equipment was used to record videos along the blueberry planting rows at a stable speed of approximately 0.5 meters per second. The recording covered different weather conditions (sunny, cloudy), lighting environments (front lighting, backlighting), and different times of day to ensure data diversity. The video resolution was set to high resolution, such as 2500×2160 pixels, with a frame rate of 30 FPS.
[0025] b) Video frame segmentation and annotation: To reduce data redundancy and ensure the continuity of information between frames, one frame is extracted from every 10 frames of the recorded video. Then, using a professional online annotation tool (such as T-Rex), each visible blueberry in the image is divided into four maturity levels—flowering stage, green fruit stage, color-changing stage, and ripe stage—based on its biological characteristics (as shown in Table 1), and precise bounding box annotations are performed.
[0026] Table 1. Blueberry Maturity Grading Standards
[0027] c) High-resolution image tiling: Considering that the original image resolution is much larger than the model input size (e.g., 640×640), direct scaling would cause severe loss of feature information for the small target, blueberries. Therefore, this embodiment adopts an image tiling strategy. After annotation, each high-resolution image and its corresponding annotation file are divided into multiple 640×640 pixel small image blocks. During the segmentation process, it is ensured that the algorithm does not cut off any individual blueberry annotation boxes, thus fully preserving the detailed features of the small target and effectively expanding the scale of the dataset.
[0028] d) Dataset partitioning: All processed small image patches and their annotation information are randomly divided into training set, validation set and test set in a ratio of 8:1:1, which are used for model parameter learning, hyperparameter tuning and performance monitoring, and final model performance evaluation, respectively.
[0029] S2: Construct and apply an improved YOLOv8n detection and classification model; This step aims to build a model that can simultaneously perform blueberry detection and fine-grained maturity classification, and apply it to video frame processing.
[0030] a) Model Construction: Refer to Figure 3 This embodiment uses YOLOv8n as its basic architecture. To enhance the model's ability to extract features from small targets in complex backgrounds, a Mamba-C2f module is integrated at a key location in the backbone network to enhance the extraction of gradient flow information from image features in complex scenes. To achieve integrated detection and classification, in addition to the original detection head responsible for bounding box regression and target confidence prediction, three independent classification heads are added in parallel in the model's detection head: a color classification head, a texture classification head, and a shape classification head. These three classification heads share feature maps from the neck network and are each responsible for judging the ripeness of blueberries from different dimensions.
[0031] b) Model Training: The improved YOLOv8n detection and classification model was trained using the training set constructed in step S1. During training, the model's loss function consisted of four parts: bounding box regression loss, target confidence loss, and classification loss for the three maturity classification heads. Through end-to-end training, the model learned the ability to simultaneously locate blueberries and accurately determine their maturity level. The training results of YOLOv8n are shown in Table 2.
[0032] Table 2. Training results of the improved YOLOv8n
[0033] c) Model Application: The blueberry video to be processed is input frame by frame into the trained detection and classification model. For each frame, the model outputs a list of results, where each element contains the bounding box coordinates of a blueberry, its target confidence score, and its maturity level.
[0034] S3: Constructing and applying the improved OC-SORT tracking algorithm; Reference Figure 4 This step aims to maintain the uniqueness of detected blueberries by building and applying an improved OC-SORT tracking algorithm to perform cross-frame tracking. Specifically, it includes: a) Model Construction: This embodiment is based on an improved version of the advanced OC-SORT tracking algorithm. First, a lightweight convolutional neural network (CNN) is integrated as an appearance re-identification (Re-ID) feature extractor. Second, a camera motion compensation (CMC) module is introduced, which estimates an affine transformation matrix through inter-frame feature point matching (e.g., using the ORB algorithm).
[0035] b) Tracking process: For each frame of the video sequence: i. State Prediction and Correction: The Kalman filter is used to predict the state of the next frame for all existing trajectories. Then, the affine transformation matrix calculated by the CMC module is used to correct the predicted state, compensating for errors caused by camera motion. The Kalman filter prediction formula is as follows: ; ; in, Indicates based on the first Frame information for the first The predicted prior state estimate of the frame. This represents the rotation and scaling matrix from frame k-1 to frame k. Indicates the first The posterior state estimate after frame update. This represents the translation matrix from frame k-1 to frame k. Indicates the first The prior state covariance matrix of the frame. Indicates the first The posterior state covariance matrix after frame update, superscript Represents the transpose of a matrix Indicates the first The covariance matrix of the frame's process noise; In one embodiment of the present invention, the affine transformation matrix is as follows: ; ; in, The affine transformation matrix from frame k-1 to frame k consists of scaling, rotation, and translation matrices.
[0036] ii. Data Association: Construct a comprehensive cost matrix for the first round of Hungarian matching. The comprehensive cost matrix integrates appearance similarity (calculated by cosine distance of Re-ID features) and motion-corrected spatial location information (IoU).
[0037] iii. State Recovery and Update: For successfully matched trajectories, update their Kalman filter state and EMA-smoothed appearance features. For trajectories temporarily lost due to occlusion or other reasons but successfully re-identified by Re-ID in subsequent frames, trigger the ORU mechanism to smoothly correct their accumulated state errors using virtual trajectories. For unmatched trajectories and detections, attempt a second round of matching based on IoU, i.e., the OCR mechanism. The Kalman filter update formula is as follows: ; ; ; in, Indicates the first The observation matrix of the frame, Indicates the first The covariance matrix of the observation noise of the frame. Indicates the first Kalman gain of the frame Indicates the first The posterior state estimate of the frame after observation and update. Indicates the first Frame observations, Indicates the first The posterior state covariance matrix of the frame after observation and update. Represents the identity matrix.
[0038] iv. Track Management: Based on the matching results, create new tracks, update existing tracks, or terminate tracks that have not been matched for a long time.
[0039] S4: Dynamic counting and production forecasting are achieved through a dual-region counting algorithm.
[0040] Reference Figure 4 This step utilizes robust counting logic and a dual-region counting algorithm to statistically analyze the tracking results and derive the final production estimate. Specifically, it includes: a) Counting area setting: Two adjacent areas are virtually set in the video frame: a confidence judgment area (first area) located upstream of the frame and a counting area (second area) located downstream of the frame.
[0041] b) Counting execution: i. When the center point of a blueberry target's tracking trajectory (e.g., ID k) first enters the first region, the system records that ID in a temporary ID cache list.
[0042] ii. When the trajectory continues to move and its center point first enters the second region, the system checks whether its ID k exists in the cache list.
[0043] iii. If present, increment the counter for the corresponding maturity category based on the current maturity level of the trajectory (provided by the detection classification model). Then, immediately remove ID k from the cache list.
[0044] c) Output Results: This process ensures that each blueberry target that stably tracks and completely traverses the counting area is counted precisely only once, regardless of whether its ID has changed. After video processing is complete, the system outputs the total number of blueberries at each maturity level as the basis for yield estimation.
[0045] To evaluate the performance of the detection model, the mean accuracy (mAP) is used as the core metric. Specifically, calculations are performed within a range of Intersection over Union (IoU) thresholds of 0.5 (mAP@0.5) and from 0.5 to 0.95 (mAP@0.5:0.95) to comprehensively measure the model's performance under different positioning accuracy requirements. The calculation formula is as follows: ; ; ; ; In the formula, For accuracy, it represents the proportion of truly correct positive predictions out of all positive predictions output by the detector; The number of blueberry targets correctly detected; The number of false positives (non-blueberry targets) or duplicate detections; Recall rate represents the proportion of all real blueberry targets that are correctly detected. The actual target number of blueberries that were missed; Average precision represents the average precision across different recall levels. Let precision be a function of recall, representing the sum of the numbers at which the recall rate is 1. The corresponding precision value; integration interval This indicates that the recall rate is integrated over the entire range from 0 to 1. The mean of the average precision represents the arithmetic mean of the average precision across all categories. The total number of target categories in this invention These correspond to four blueberry maturity levels: flowering period, green fruit period, color-changing period, and ripening period. This is a category index, with a value range of [value range missing]. arrive ; For the first Average precision values for each category; Indicates all The average precision of each category is summed.
[0046] To evaluate the effectiveness of the tracking algorithm and counting method, the coefficient of determination (R²) was used. 2 R is used to assess the consistency between predicted counts and actual field counts using the mean absolute error (MAE). 2 The closer the value is to 1, the stronger the correlation between the model prediction and the actual situation; MAE, on the other hand, intuitively reflects the magnitude of the average counting error.
[0047] As shown in Table 3, the improved YOLOv8n detection and classification model in this invention achieved a mAP@0.5 of 95.2% and a mAP@0.5:0.95 of 66.1% on the test set for the Blueberry category, which is significantly better than mainstream lightweight detectors such as YOLOv5n, YOLOv8n, and YOLOv10s. This demonstrates that the improved design of this invention for small targets like blueberries is effective.
[0048] Table 3 Performance Comparison of Different Detection Algorithms (Languo)
[0049] As shown in Table 4, in the final counting task, the method of this invention predicted a total of 3385 Blueberry categories, which is very close to the actual total of 3250 Blueberry categories. Its R... 2 The value reached 0.93, and the MAE was only 8.7. The experimental results clearly demonstrate the significant superiority of the method of this invention in effectively suppressing ID switching and achieving accurate dynamic counting. 2 The formula for calculating MAE is as follows: ; ; in This represents the true value of the i-th sample. This represents the predicted value of the i-th sample. This represents the arithmetic mean of all true values, where n represents the total sample size.
[0050] Table 4 Performance evaluation of blueberry counting (R) 2 and MAE)
[0051] In summary, through the detailed steps of the above embodiments, the present invention collaboratively solves the problems of blueberry detection, classification, tracking and counting in complex field environments, and achieves high-precision dynamic counting and yield prediction.
[0052] Please see Figure 2 A blueberry counting and yield prediction system for complex field environments includes: The data processing module is used to collect, label, and enhance blueberry video data in the field, and to build the dataset required for training. The detection and classification module has a built-in improved YOLOv8n detection and classification model, which is used to receive video frames and output the location and fine-grained maturity level of the blueberry target. The tracking module incorporates an improved OC-SORT tracking algorithm for associating and maintaining the IDs of blueberry targets across frames; The counting and output module has a built-in dual-region counting method for accurately counting the tracking trajectory and finally outputting the blueberry quantity statistics for each maturity level.
[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for blueberry counting and yield prediction in complex field environments, characterized in that, Includes the following steps: S1. Collect videos of blueberries in the field as a dataset, label the blueberry targets in the dataset, and perform data augmentation on the dataset; S2. Construct an improved YOLOv8n detection and classification model that integrates fine-grained maturity classification. Input the blueberry video into the improved YOLOv8n detection and classification model, and obtain the detection results containing blueberry target location information and maturity level after frame-by-frame processing. S3. Construct an improved OC-SORT tracking algorithm that integrates appearance re-identification mechanism and camera motion compensation. Use the detection results as input to the improved OC-SORT tracking algorithm to obtain the tracking trajectory of blueberry targets at different maturity levels. S4. Based on the dual-region counting method, the tracking trajectory is statistically analyzed to obtain the counting results of blueberries at each maturity level.
2. The method for blueberry counting and yield prediction in complex field environments according to claim 1, characterized in that, The process of constructing the dataset includes: a) Video data acquisition: Portable high-definition camera equipment was used to record videos along the blueberry planting rows at a stable moving speed to obtain high-resolution raw video sequences; b) Video frame cutting and annotation: Image frames are extracted from the high-resolution original video sequence at preset frame intervals, and bounding boxes are annotated for each visible blueberry fruit in the frame according to its maturity level. c) High-resolution image tiling: The labeled high-resolution image frame and its corresponding annotation file are divided into multiple image tilings suitable for the model input size, ensuring that the annotation boxes of any individual blueberry targets are not cut off during the tiling process; d) Dataset partitioning: All processed image patches and their annotation information are randomly divided into training set, validation set and test set according to a preset ratio.
3. The method for blueberry counting and yield prediction in complex field environments according to claim 2, characterized in that, The construction of the improved YOLOv8n detection and classification model includes: The Mamba-C2f module is integrated into the backbone network of the YOLOv8n model, and three parallel classification heads for color, texture, and shape are added to the detection head of the YOLOv8n model.
4. The method for blueberry counting and yield prediction in complex field environments according to claim 3, characterized in that, The specific details of constructing the improved OC-SORT tracking algorithm, which integrates appearance re-identification mechanism and camera motion compensation, are as follows: An appearance re-identification mechanism is used to extract a depth feature vector as the appearance identifier for each detected blueberry target. When a target reappears after being lost due to occlusion, the correct ID of the target is recovered by comparing the appearance feature vectors of the newly detected target with those of the historical trajectory. An exponential moving average method is then used to smoothly update the appearance features of the trajectory; the calculation formula is as follows: ； In the formula, For the first The trajectory at the th Smoothed appearance feature vector at frame time. For the first The trajectory at the th Historical appearance feature vector at frame time, For the first The trajectory at the th The original appearance features of the detection box matched at frame time. It is a smoothing factor; By using the camera motion compensation mechanism, an affine transformation matrix is estimated by analyzing the global pixel displacement between consecutive frames, and the affine transformation matrix is used to correct the position deviation of the tracking trajectory prediction caused by camera movement or jitter.
5. The method for blueberry counting and yield prediction in complex field environments according to claim 4, characterized in that, The color classification head is configured to: convert the input image patch from the RGB color space to the HSV color space and extract the color histogram of the hue channel as the classification basis; the texture classification head is configured to calculate the gray-level co-occurrence matrix of the image patch and extract the texture descriptor as the classification basis; the shape classification head is configured to extract the fruit outline through image segmentation and calculate the shape descriptor as the classification basis.
6. The method for blueberry counting and yield prediction in complex field environments according to claim 5, characterized in that, The process of using the detection results as input to the improved OC-SORT tracking algorithm to obtain the tracking trajectory of blueberry targets at different maturity levels is as follows: a) State prediction and correction: Using the affine transformation matrix estimated by camera motion compensation, the state of all existing tracking trajectories predicted by the Kalman filter is corrected. b) Construction of the comprehensive cost matrix: A comprehensive cost matrix is constructed for the detection box and the corrected tracking trajectory of the current frame. The cost value of this comprehensive cost matrix integrates the following three metrics: i. Appearance similarity based on appearance re-identification; ii. Consistency of motion based on observation-centric momentum; iii. Spatial overlap based on intersection-union ratio; c) Association matching: The Hungarian algorithm is used to solve the comprehensive cost matrix to achieve the optimal matching between the detection box and the tracking trajectory; d) State recovery and update: For trajectories that were lost in previous frames due to occlusion but were rematched in the current frame, an observation-centered re-update is performed, and their Kalman filter parameters are smoothly corrected by generating virtual trajectories; for trajectories that failed to be associated in the first round of matching, an observation-centered recovery is performed, and a secondary association is attempted with the remaining unmatched detection boxes; e) Track Management: Based on the matching results, create new tracks, update existing tracks, or terminate tracks that have not been matched for a long time.
7. The method for blueberry counting and yield prediction in complex field environments according to claim 6, characterized in that, The Kalman filter employs the following state equation for its prediction and update steps after considering camera motion correction: Prediction steps: ； ； Update steps: ； ； ； in, Indicates based on the first Frame information for the first The predicted prior state estimate of the frame. This represents the rotation and scaling matrix from frame k-1 to frame k. Indicates the first The posterior state estimate after frame update. This represents the translation matrix from frame k-1 to frame k. Indicates the first The prior state covariance matrix of the frame. Indicates the first The posterior state covariance matrix after frame update, superscript To represent the transpose of a matrix, Indicates the first The covariance matrix of the process noise of the frame. Indicates the first The observation matrix of the frame, Indicates the first The covariance matrix of the observation noise of the frame. Indicates the first Kalman gain of the frame Indicates the first The posterior state estimate of the frame after observation and update. Indicates the first Frame observations, Indicates the first The posterior state covariance matrix of the frame after observation and update. Represents the identity matrix.
8. The method for blueberry counting and yield prediction in complex field environments according to claim 7, characterized in that, Two-region counting methods include: In the video frame, a confidence judgment area and a counting area are set, where the confidence judgment area is denoted as the first area and the counting area is denoted as the second area; When a blueberry target's tracking trajectory first enters the first area, the system records its unique ID and stores it in a temporary ID cache list; When the tracking trajectory continues to move and enters the second area, the system verifies again whether its ID exists in the temporary ID cache list. If it exists, the system increments the counter of the corresponding category according to the current maturity level of the tracking trajectory and removes the ID from the cache list to ensure that each target is counted only once.
9. A blueberry counting and yield prediction system for complex field environments, characterized in that, A method for blueberry counting and yield prediction in complex field environments as described in any one of claims 1-8, comprising: The data processing module is used to collect, label, and enhance blueberry video data in the field, and to build the dataset required for training. The detection and classification module has a built-in improved YOLOv8n detection and classification model, which is used to receive video frames and output the location and fine-grained maturity level of the blueberry target. The tracking module incorporates an improved OC-SORT tracking algorithm for associating and maintaining the IDs of blueberry targets across frames; The counting and output module has a built-in dual-region counting method for accurately counting the tracking trajectory and finally outputting the blueberry quantity statistics for each maturity level.