Blueberry maturity detection method and system

Abstract

The invention discloses a blueberry maturity detection method and system, and relates to the technical field of image processing methods. The method comprises the following steps: acquiring blueberry images by using an unmanned aerial vehicle; preprocessing the collected blueberry images, and constructing a model training set, a verification set and a test set; an improved yolov8n model is built; using the model training set to train the improved yolov8n model, and using the verification set and the test machine to verify and test the improved yolov8n model to complete optimization of the improved yolov8n model; and S5, carrying out blueberry maturity detection by using the optimized and improved yolov8n model. The method and the system have the advantages of high blueberry maturity detection precision, high robustness and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a blueberry maturity detection method and system. BACKGROUND

[0002] The maturity of blueberry fruit is directly related to its picking time, eating quality, storage life and market pricing. Therefore, accurate detection of fruit maturity is crucial for planting management and supply chain efficiency. In the traditional blueberry production process, the judgment of fruit maturity mainly relies on manual visual inspection. This method has the defects of strong subjectivity, non-uniform standard, and being easily affected by environment and personnel experience. Especially in large-scale plantations, manual detection is not only inefficient and costly, but also difficult to achieve full coverage, often leading to early or late picking of fruit, affecting the final economic benefit. According to statistics, the labor cost of blueberry picking has accounted for 30% to 50% of the total production cost, and the labor shortage problem has further exacerbated this contradiction.

[0003] In recent years, with the development of smart agriculture technology, some researches have tried to introduce image recognition and target detection technology to assist the automatic recognition of fruit maturity. However, the existing technology still has many shortcomings in application. On the one hand, fruits are easily blocked by natural backgrounds such as leaves and branches, leading to misjudgment or missed detection of traditional detection models in complex scenes; on the other hand, the color difference of fruits of different maturity is sometimes not obvious, especially under uneven lighting or fruit surface reflection conditions, the accuracy of model recognition is greatly reduced. In addition, some methods only rely on color features for judgment, ignoring the size change characteristics of fruits during the ripening process, and cannot fully utilize multi-modal information to improve classification accuracy.

[0004] The existing fruit detection methods generally lack the ability to recognize multi-scale fruits, making it difficult to simultaneously consider the detection effect of small and large fruits, and the generalization ability is weak, with poor stability under different planting areas and different imaging conditions. At the same time, the existing methods are mostly static image processing, which cannot realize dynamic tracking and management of the fruit ripening process, limiting the scientific decision-making ability of fruit farmers on picking time. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a blueberry maturity detection method with high detection accuracy and high robustness.

[0006] To solve the above technical problems, the technical solution adopted by the present application is: a blueberry maturity detection method, comprising the following steps:

[0007] S1, using a drone to collect blueberry images;

[0008] S2, pre-process the collected blueberry image, and construct a model training set, a validation set and a test set;

[0009] S3, build an improved yolov8n model;

[0010] S4, train the improved yolov8n model using the model training set, and verify and test the improved yolov8n model using the validation set and the test set, and complete the optimization of the improved yolov8n model;

[0011] S5, use the optimized improved yolov8n model to detect the maturity of blueberries.

[0012] The application also discloses a blueberry maturity detection system, comprising:

[0013] An image acquisition module is used to acquire blueberry images using a drone;

[0014] An image pre-processing module is used to pre-process the collected blueberry image, and construct a model training set, a validation set and a test set;

[0015] A model construction module is used to build an improved yolov8n model;

[0016] A model training and optimization module is used to train the improved yolov8n model using the model training set, and verify and test the improved yolov8n model using the validation set and the test set, and complete the optimization of the improved yolov8n model;

[0017] A detection module is used to use the optimized improved yolov8n model to detect the maturity of blueberries.

[0018] The technical scheme has the advantages that the method detects the maturity of blueberry fruits by using the improved yolov8n model, and has the advantages of high detection precision and high robustness. BRIEF DESCRIPTION OF DRAWINGS

[0019] The application will be further described in detail below with reference to the drawings and specific embodiments.

[0020] Figure 1 is the main flowchart of the method described in the embodiments of the application;

[0021] Figure 2a is the perspective (pre-pan perspective) of the drone collecting blueberries in the method described in the embodiments of the application;

[0022] Figure 2b is the perspective (pre-pan perspective) of the drone collecting blueberries in the method described in the embodiments of the application;

[0023] Figure 3a is the principle diagram of the improved yolov8n model in the embodiment of the application (wherein part A is the model Backbone, part B is the model Neck, and part C is the model Head);

[0024] Figure 3b is the principle diagram of the EFCSP module in the improved yolov8n model in the embodiment of the application;

[0025] Figure 3c is the principle diagram of the FasterEMA module in the improved yolov8n model in the embodiment of the application;

[0026] Figure 4 is the principle diagram of the lightweight detection head in the method described in the embodiment of the application;

[0027] Figure 5a is the principle diagram of the deep separable convolution feature extraction in the method described in the embodiment of the application (standard convolution);

[0028] Figure 5b is the principle diagram of the deep separable convolution feature extraction in the method described in the embodiment of the application (deep convolution);

[0029] Figure 5c is the principle diagram of the deep separable convolution feature extraction in the method described in the embodiment of the application (point convolution);

[0030] Figure 6 is the detection flowchart of the method described in the embodiment of the application;

[0031] Figure 7 is the principle diagram of the system described in the embodiment of the application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0033] In the following description, many specific details are set forth in order to provide a thorough understanding of the application. However, the application can be practiced without the specific details, which are not described in the description, and it can be apparent that the application can be implemented in other different ways, and those skilled in the art can make similar generalizations without departing from the spirit and scope of the application, so the application is not limited to the specific embodiments disclosed below.

[0034] In general, as Figure 1As shown, the embodiment of the present application discloses a blueberry maturity detection method, which comprises the following steps:

[0035] S1, using a drone to collect blueberry images;

[0036] S2, preprocessing the collected blueberry images, constructing a model training set, a validation set and a test set;

[0037] S3, building an improved yolov8n model;

[0038] S4, training the improved yolov8n model using the model training set, and verifying and testing the improved yolov8n model using the validation set and the test set, and optimizing the improved yolov8n model;

[0039] S5, using the optimized improved yolov8n model to detect the maturity of blueberries.

[0040] The above steps will be described in detail below in combination with specific contents

[0041] Step S1, using a drone to collect data.

[0042] The blueberry images used in this application are collected in a blueberry picking garden in Shimen Town, Tongxiang City, Jiaxing City, Zhejiang Province, China. The experimental field (30.66N, 120.43E) has relatively good conditions, sufficient light, and blueberries are guarded for 24 hours. The red solid line area in the figure is selected as the research object, and the distance between blueberry plants is about 1.5 meters. The height of each row of blueberry plants varies from 1.5 meters to 2.0 meters.

[0043] This collection uses DJI Phantom 4Pro (DJI Lingling 4Pro) drone, and the RGB camera CMOS size is 1 inch, the effective pixel is 20 million, and the resolution of a single image is 5472*3078. Since the experimental field where the blueberries are cultivated is covered with a bird net above the blueberries, the flight height is set between 4-6 meters, and the distance between the drone camera and the blueberry plants is about 1-1.5 meters. The image is collected by translating the drone, as shown in the figure. Figures 2a-2b Select three rows of adjacent fruit trees with a length of about 50 meters in the blueberry garden, and collect data for four weeks from April 18, 2024 to June 18, 2024. The average temperature of the four weeks is about 25℃, the sampling frequency is once a week, 150 pictures are collected each time, and 20% of the samples are processed.

[0044] Step S2, data preprocessing, constructing model training set, validation set and test set

[0045] In this application, the original image of blueberry fruit is labeled with a label YOLO format data set and a.txt type label file is generated according to the label format. The fully mature blueberry is labeled as "blueberry", the semi-mature blueberry is labeled as "purple berry", and the unripe blueberry is labeled as "green berry". When labeling, the bounding box used is the smallest rectangle that can completely enclose the entire blueberry fruit, which is done to minimize unnecessary pixels in the background.

[0046] The 120 images collected from the four surrounding data with a resolution of 5472*3078 are cropped along the length and width of 640*640 resolution to adapt to the input of the baseline model YOLOV8 of this experiment, and a total of 3614 images are obtained, including 2585 training sets, 661 validation sets and 368 test sets. The accuracy of all subsequent experiments is the performance of the model on the test set.

[0047] Step S3, building an improved yolov8n model for blueberry maturity detection.

[0048] As shown in Figure 3a , the improved yolov8n model includes a model Backbone, a model Neck and a model Head; the model Backbone is connected with the model Head through the model Neck, and the model Backbone includes a plurality of Conv modules, a plurality of EFCSP modules (as shown in Figure 3b ), an SCDown module and an SPPF module; the EFCSP module includes a Conv module, a Split module and a plurality of high-efficiency hybrid attention modules FasterEMA connected in series.

[0049] 1) High-efficiency hybrid attention module FasterEMA:

[0050] The application provides a high-efficiency hybrid attention module FasterEMA based on feature enhancement, and the FasterEMA module is as shown in Figure 3c , which aims to improve the feature modeling capability for complex background and small targets in the target detection task. The module includes the following steps:

[0051] First, the input feature map x is received, and it is judged whether the input channel number inc is consistent with the internally set channel number dim; if the two are not consistent, a group of 1*1 convolution operations (adjust_channel module) is used to adjust the channels of the input feature, so that the channel number of the input feature is consistent with the internally set channel number, to ensure the consistency of the subsequent processing steps.

[0052] Secondly, the channel-adjusted feature map is reserved as a residual branch (shortcut) and input into a local spatial feature mixing module (spatial_mixing). The local spatial feature mixing module is constructed based on a Partial_conv3 structure,

[0053] The channel dimension is divided into two parts by a preset division parameter n_div, one part has dim_conv3 channels, and the other part has dim_untouched. For the channel region corresponding to dim_conv3, a convolution operation (partial_conv3) with a convolution kernel size of 3×3, a stride of 1, and padding of 1 is used for local feature extraction to enhance the local spatial perception ability.

[0054] The calculation amount of the general standard convolution kernel and the calculation amount of PConv (partial_conv3) are shown in formulas (1) and (2), respectively.

[0055] The calculation amount of the general standard convolution kernel Conv is:

[0056] h×w×k 2 ×c 2 (1)

[0057] where k is the convolution kernel size, c is the input and output channel number, w is the width, and h is the height.

[0058] The calculation amount of PConv is:

[0059] h×w×k 2 ×c p 2 (2)

[0060] As shown in formulas (1) and (2), PConv only applies a regular Conv to part of the input channels for spatial feature extraction, and the remaining channels remain unchanged. For continuous or regular memory access, the first or last continuous c p channel is taken as the representative of the entire feature map for calculation. Since only a few continuous c p channels are involved in the calculation, c p is less than the total number of channels c, so the calculation amount of PConv will be times that of Conv.

[0061] The dim_untouched part remains unchanged to retain the information of the original features. Further, the Partial_conv3 module supports two forward propagation modes: when set to the split_cat mode, the input feature map is first split along the channel dimension into a convolution processing part and an untouched part, and after local convolution and preservation operations are performed respectively, the feature concatenation (Concatenation) along the channel dimension is performed, thereby realizing the effect of local enhancement and overall feature fusion; when set to the slicing mode, the local convolution operation is directly applied to the previous part of the channel of the original feature map, and the remaining part of the channel feature is kept unchanged, which is suitable for inference acceleration requirements. Through the above Partial_conv3 design, the local spatial feature mixing module can effectively improve the local modeling capability of the feature map while ensuring the calculation efficiency, and enhance the perception ability of the model to the target fine-grained features. Then, the local spatial mixed feature map is input into a multi-layer perceptron module (MLP module). The MLP module is composed of two layers of 1x1 convolution, the first layer increases the channel number of the feature map to dim multiplied by mlp_ratio, and the second layer restores it to dim. Through this process, the cross-channel information interaction and feature expression ability are enhanced. If the drop_path parameter is set, the MLP module output further introduces a random depth (DropPath) operation to improve the training stability and generalization ability of the network.

[0062] Further, for the spatial context information extraction requirement of the feature map, the local spatial mixed feature map is input into a channel enhancement module (EMA module). The EMA module is constructed based on a grouping mechanism, and by setting a grouping factor (groups), the input feature map is divided into multiple subgroups along the channel dimension, and each subgroup independently models attention to improve the expression ability and calculation efficiency of local features.

[0063] Specifically, first, the input feature map is grouped and reshaped according to the number of groups, and each subgroup is operated respectively; for each subgroup, adaptive average pooling operations are performed along the horizontal (pool_h) and vertical directions (pool_w) to extract information in a single spatial dimension. Then, the horizontal and vertical direction pooled features are concatenated, and feature fusion is performed through a convolution kernel size of 1x1 (conv1x1) convolution operation, and then separated into horizontal and vertical features along the channel direction, and Sigmoid activation is applied respectively as the attention weight of the spatial dimension. Next, the original grouped features and the above-mentioned spatial dimension attention weight are multiplied element by element, and a group normalization (GroupNorm) operation is applied to enhance the stability of the feature distribution. At the same time, in order to supplement the local fine-grained features, a convolution kernel size of 3x3 (conv3x3) convolution operation is used to further process the grouped features, and global features are extracted through adaptive average pooling.

[0064] In order to strengthen the interaction between features, the features extracted by the above two paths are normalized by Softmax, and a weight matrix is calculated. Through matrix multiplication (MatMul), the weight information is fused with the feature map, and the final weight map is generated. Finally, according to the weight map, the grouped feature map is weighted element by element, and restored to the original feature map size, thereby completing the spatial context enhancement of the input feature map.

[0065] Finally, the features after attention enhancement processing are element-wise added to the initial residual branch (shortcut) to form the final output result of the module. In order to improve the stability of the model during the initial training, when the layer_scale_init_value parameter is set to greater than zero, a learnable scaling factor (layer_scale parameter) is introduced to apply channel-wise scaling modulation to the main branch features before fusion.

[0066] Through the above steps, this module effectively combines local spatial mixing, multi-layer perception enhancement and efficient attention mechanism, while ensuring inference efficiency, significantly improving the feature modeling and expression ability of the model in complex background, small target detection and other tasks. The FasterEMA module can be flexibly stacked and applied to the bottleneck structure in the backbone network structure CSP to meet the feature extraction requirements of different depths and scales, further improving the overall blueberry detection performance. The structure after application is as shown in Figure 3c .

[0067] 2) PIoU:

[0068] In the target detection task, when the target is partially occluded by other objects, the prediction box is easily disturbed by the occluded area, leading to a decrease in the regression accuracy of the bounding box, and thus affecting the overall detection performance. In view of the problem that the detection accuracy is reduced due to the occlusion of fruits by leaves in the blueberry cultivation scene, an improved bounding box regression loss function is proposed, which introduces an occlusion penalty term based on the existing CompleteIoU (CIoU) as shown in formulas (3) to (9).

[0069] The method dynamically analyzes the IoU value of the prediction box and the real box, and when the IoU value is higher than a preset threshold (for example, 0.7), an occlusion penalty term (Overlap Penalty) is applied. By introducing the occlusion penalty term, the model can adaptively perceive the degree of occlusion of the target and make targeted corrections to the prediction box. Further, to avoid excessive punishment leading to unstable training, the IoU residual is calculated and a weight factor (for example, 0.1) is introduced to balance the occlusion penalty and the detection accuracy.

[0070] The occlusion penalty mechanism proposed in the application can effectively guide the model to focus on the target area with less prominent features due to occlusion. Specifically, by introducing a penalty on high IoU samples, the model is prompted to more accurately fit the boundary details of the occluded area, thereby enhancing the ability to distinguish targets from backgrounds and reducing the interference of complex backgrounds (such as leaf texture and color) on the detection results.

[0071] Especially in the blueberry fruit detection scene, after the fruit is occluded by the leaves, only about 30% or less of the visible area is usually visible. The conventional IoU loss function is prone to ignore such difficult-to-detect targets, leading to missed detection. By introducing the occlusion penalty term, the model can extract more semantic features from the limited visible area, improve the prediction ability of partially occluded targets, and effectively reduce the missed detection rate. In addition, the introduction of the occlusion penalty term also enhances the model's attention to small targets and local features, further improving the fitting effect of the boundary details in the occluded area.

[0072] The method described in the application combines the geometric optimization characteristics of CIoU, and can guide the prediction box to more accurately align the real box position by geometric constraints such as center point distance and aspect ratio, and adjust the prediction box shape according to the visible area, thereby improving the overall bounding box regression performance.

[0073] In summary, by introducing the IoU-based occlusion penalty term based on CIoU, the detection ability of the model for occluded targets can be effectively enhanced during fruit detection, and the boundary fitting accuracy and overall detection robustness in the occluded scene can be improved. For ease of expression, the improved bounding box regression loss function proposed in the application is named PIoU (PenaltyIoU).

[0074]

[0075]

[0076]

[0077]

[0078] The overlap penalty is defined by the following formula:

[0079] overlap_penalty=max(0,IoU-γ)×w (7)

[0080] The PIoU with overlap penalty is defined by the following formula:

[0081]

[0082] The LossCIoU is defined by the following formula:

[0083]

[0084] Wherein, CIoU is a boundary box regression loss function adopted by an original model, alpha is a shape consistency term, which measures the aspect ratio difference between a predicted box and a real box; v is a normalized index measuring the angle difference between the predicted aspect ratio and the real aspect ratio, is an index for normalizing the center point distance difference in target detection; overlap_penalty is an occlusion penalty term, gamma is an occlusion IoU threshold, and w is a penalty scaling coefficient; when the IoU exceeds the threshold gamma, the penalty term of PIoU takes effect, and in the gradient back propagation process, an additional constraint is applied to the loss function of the model, guiding the model to pay more attention to the boundary details and actual outline of the occluded fruit.

[0085] 3) NWD loss function:

[0086] In the small target detection task, the traditional IoU (Intersection over Union) measurement method cannot effectively provide gradient information when the predicted box and the real box have no overlap, which easily leads to gradient disappearance or gradient shock in the model training process, thereby affecting the model convergence effect. In order to solve the above problems, the present application adopts a new boundary box similarity measurement method, namely normalized wasserstein distance (Normalized Wasserstein Distance, NWD for short).

[0087] Wasserstein distance originates from the optimal transport theory, and is mainly used to measure the minimum transmission cost between two probability distributions. Unlike the IoU which only depends on the overlapping area of the bounding box, the Wasserstein distance can still reflect the difference between the center position and the scale of the predicted box and the real box in the case of no overlap, so as to provide continuous and smooth gradient information and ensure that the model can be gradually optimized in the process of bounding box prediction. Specifically, the present application models the bounding box as a two-dimensional Gaussian distribution, in which the center point of the box is taken as the mean value, and the width and height are taken as the parameters of the covariance matrix. Through this soft representation, the bounding box is mapped to a unified probability space, effectively reducing the similarity measurement fluctuation problem caused by the change of the scale. At the same time, by normalizing the Wasserstein distance, the similarity measurement of targets of different scales has good numerical stability. The normalized Wasserstein distance (NWD) can not only alleviate the problem of IoU dramatic change caused by small position offset in small target detection, but also can significantly improve the detection accuracy and robustness of small size targets (such as small blueberries, green berries with a diameter of less than 6*6 pixels) compared with the traditional IoU measurement method in actual detection tasks. For example, under the traditional IoU measurement, a small target of only 1 pixel offset can cause the IoU value to drop from 0.5 to 0.1, while the NWD can effectively tolerate the small position offset and reduce the occurrence of false detection and missed detection due to the introduction of continuous measurement of the center position and scale difference.

[0088] The normalized Wasserstein distance (NWD) proposed by the present application is defined as follows:

[0089] For a horizontal bounding box R=(c x ,c y ,w,h), where (c x ,c y ), w and h represent the center coordinates, width and height respectively. The equation of the inscribed ellipse can be expressed as:

[0090]

[0091] The probability density function of the two-dimensional Gaussian distribution is:

[0092]

[0093] Where x, μ and ∑ represent the coordinates (x, y) of the Gaussian distribution, the mean vector and the covariance matrix respectively.

[0094] When (x-μ) T Σ -1 (x-μ)=1(12)

[0095] The ellipse in equation (11) will be the density profile of a two-dimensional Gaussian distribution. Therefore, the horizontal bounding box R = (c x , c y , w, h) can be modeled as a two-dimensional Gaussian distribution N(μ,∑):

[0096]

[0097] For two two-dimensional Gaussian distributions μ1 = N(m1,∑1) and μ2 = N(m2,∑2), the second-order Wasserstein distance between μ1 and μ2 is defined as:

[0098]

[0099] Furthermore, for Gaussian distributions Na and Nb modeled from bounding boxes A = (cx a , cy a , w a , h a ) and B = (cx b , cy b , w b , h b ), equation (6) can be further simplified as:

[0100]

[0101] With its exponential form normalization, a new metric, called Normalized Wasserstein Distance (NWD), is obtained:

[0102]

[0103] In practical application scenarios, the data set processed by the present application not only contains a large number of small size immature blueberries (Young Blueberries), but also contains a large number of large size mature blueberries (Mature Blueberries). In view of the characteristics of different size targets, the traditional IoU type loss function has obvious advantages in processing large size targets, which can further assist the optimization and convergence of the model by punishing the center point offset and the inconsistent aspect ratio of the bounding box. In view of this, the present application proposes a bounding box regression optimization method for different target scales. Specifically, for the predicted box and the real box of each sample, the optimized weighted IoU loss (Optimized PIoU Loss) and the normalized Wasserstein distance loss (NWD Loss) are calculated respectively, and the two losses are weighted and fused according to the preset proportion, thereby forming the final bounding box regression loss. This method can meet the detection needs of large size targets and small size targets, and realize the overall improvement of detection accuracy and robustness. Specifically, for each sample, first, the optimized PIoU loss is calculated based on the predicted box and the real box. This loss function retains the basic characteristics of the traditional IoU, and further strengthens the positioning accuracy of large size targets by introducing a center point offset penalty term and an aspect ratio penalty term. At the same time, based on the aforementioned normalized Wasserstein distance (NWD) method, the smooth loss value of the small size target which is more sensitive to displacement and scale change is calculated. Subsequently, the final bounding box regression loss (Loss_bbox) is obtained by the following weighted fusion method:

[0104] Loss_bbox=α×LossPIoU+(1-α)×LossNWD (17)

[0105] Wherein, LossPIoU represents the optimized weighted IoU loss, LossNWD represents the normalized Wasserstein distance loss, and a is a fusion weight factor, the value range of which is [0, 1], which is used to dynamically balance the contribution proportion of the two losses in the total loss. By adjusting the value of a, the detection performance can be flexibly optimized according to the proportion of small targets and large targets in the actual task. For example, when the proportion of small targets in the data set is high, the value of a can be appropriately reduced to enhance the influence of the NWD loss; when the proportion of large targets is high, the value of a can be appropriately increased to exert the advantage of the optimized PIoU in large target detection. The bounding box regression loss fusion mechanism proposed by the present application can effectively improve the performance of the model in the multi-scale target detection task, so that the detection algorithm can adapt to the detection needs of large size and small size blueberries, improve the overall detection accuracy and robustness, and is especially suitable for complex application scenarios such as agricultural unmanned aerial vehicle blueberry fruit automatic detection.

[0106] 4) Lightweight detection head:

[0107] The application proposes a lightweight detection head structure (Lightweight Attention-Decoupled Head, LADH for short), as shown in Figure 4 . The structure is based on the traditional decoupled dual-branch design, and in view of the high detection difficulty problem of the regression task, a depthwise separable convolution (DSConv) module is introduced in the regression branch, thereby effectively reducing the parameter quantity and calculation cost of the model while ensuring the detection accuracy. Specifically, the depthwise separable convolution DSConv decomposes the standard convolution operation (as shown in Figure 5a ) into two sub-operations: depthwise convolution and pointwise convolution. Among them, the depthwise convolution (as shown in Figure 5b ) respectively applies a convolution kernel to each input channel, focusing on extracting local spatial features within a single channel, and avoiding interference between channel features. This independent processing method can greatly reduce the computational complexity while maintaining the feature expression ability. The pointwise convolution (as shown in Figure 5c ) uses a 1×1 size convolution kernel to linearly weight and combine all input channel features at each spatial position, realizing effective fusion of different channel features. Specifically, the output feature of each spatial position is obtained by weighted sum of all channel features in the input feature map, thereby realizing cross-channel information interaction and integration. In the detection head structure proposed in the application, the depthwise convolution first independently extracts and fuses the local features in each channel to learn the fine-grained feature relationship within the channel; then the pointwise convolution strengthens the information interaction and integration between different channels through the cross-channel weighting combination mechanism. Through this design, the network can effectively extract valuable information from each channel and improve the feature expression ability. Compared with the traditional standard convolution, the depthwise separable convolution can greatly reduce the model parameter quantity and calculation quantity while maintaining or even improving the detection performance of the model. Therefore, the LADH detection head structure proposed in the application can realize efficient and accurate blueberry target detection tasks in resource-constrained environments such as agricultural unmanned aerial vehicle platforms and edge computing devices.

[0108] Assuming that the number of channels of the input feature map is C in , the number of output channels is C out , the size of the convolution kernel is k×k, the height and width of the feature map are H and W, respectively, the parameter quantity and calculation quantity of the standard convolution are shown by the following formula:

[0109] P standard =C in ×C out ×k2 (17)

[0110] O standard =C in ×C out ×k 2 ×H×W (18)

[0111] wherein, P standard is the parameter quantity of the standard convolution, O standard is the calculation quantity of the standard convolution;

[0112] The parameter quantity and the calculation quantity of the depth separable convolution are shown by the following formulas:

[0113] P depthwise =C in ×k 2 (19)

[0114] P pointwise =C in ×C out (20)

[0115] P separable =P depthwise +P pointwise =C in ×k 2 +C in ×C out

[0116] P depthwise is the parameter quantity of the depth convolution, P pointwise is the parameter quantity of the point convolution,

[0117] P separable is the parameter quantity of the depth separable convolution as a whole.

[0118] O depthwise =C in ×k 2 ×H×W (21)

[0119] O pointwise =C in ×C out ×H×W (22)

[0120] O separable =O depthwise +O pointwise =C in ×k 2 ×H×W+C in ×C out ×H×W (23)

[0121] O depthwise is the calculation quantity of the depth convolution, Opointwise is the computation of pointwise convolution, O separable is the computation of depthwise separable convolution.

[0122] By comparison, the overall parameter quantity and computation quantity are reduced by times, and this multiple will continue to expand with the number of our output channels.

[0123] 5) Evaluation Metrics

[0124] Various indicators are used to comprehensively evaluate the detection performance of the model, including mAP (mean average precision), single-class average precision (AP), precision, recall, model parameter quantity, computational complexity, and detection speed. In the evaluation process, we set the IoU threshold to 0.5. The formulas for calculating AP, mAP, precision, and recall correspond to formulas (24)-(27), respectively:

[0125]

[0126]

[0127]

[0128]

[0129] where TP represents the number of blueberries correctly detected by the model, FP represents the number of blueberries incorrectly detected by the model, and FN represents the number of blueberries not detected by the network. The precision index measures the proportion of correctly detected positive samples among all detected samples. The recall measures the proportion of truly positive samples that are correctly detected. In each plant disease detection scenario, we can draw the precision-recall (P-R) curve according to these two indicators. The average precision (AP) is the area under the P-R curve, and a value close to 1 indicates better model performance. To balance accuracy and recall, we also use the F1 score, which represents the weighted average of accuracy and recall. mAP is the average of AP in multiple categories, making it the most commonly used evaluation indicator in object detection. In the formula, Q represents the number of categories in the dataset. The overall processing flow is shown in Figure 6 .

[0130] The method detects blueberry fruits through an improved yolov8n model, the FasterEMA module is introduced into the improved yolov8n model, the feature extraction capability is effectively improved, the model can still accurately separate the fruit and the background area in the face of complex background, fruit occlusion and similar color, and the overall detection precision is improved. The module fuses PConv light weight convolution and multi-scale attention mechanism without information loss, which not only improves the multi-scale feature expression capability, but also significantly reduces the calculation overhead of the model, and lays a foundation for edge deployment.

[0131] In view of the problem that the fruits are often occluded by leaves in the blueberry scene, the PIoU bounding box regression mechanism based on the occlusion penalty term is adopted, the fitting capability of the model to the boundary of the partially occluded fruit is enhanced, and the missed detection and false detection are effectively reduced. Compared with the traditional IoU or CIoU loss, the method can guide the model to adaptively optimize the boundary prediction result in the high overlap scene. In order to consider the detection performance of blueberry fruits of different sizes (such as small size greenberry and large size blueberry), the loss function weighting strategy fusing PIoU and NWD (Normalized Wasserstein Distance) is designed, the former is better for large target positioning, and the latter can tolerate small target position deviation, and the dynamic balance of the two significantly improves the robustness and stability of multi-scale detection.

[0132] In addition, in order to improve the real-time performance and deployment efficiency of the method, the light weight detection head LADH structure proposed by the application introduces the depth separable convolution technology while keeping the advantages of separating classification and regression tasks, and significantly reduces the parameter quantity and calculation complexity. The calculation amount of the optimized model is 4.1 GFLOPs, and the detection speed of the quantized model deployed on the Jetson AGX Orin platform is as high as 134.23 FPS, which meets the real-time detection requirements in the agricultural unmanned aerial vehicle scene.

[0133] Corresponding to the method, as Figure 7 shown, the embodiment of the application also discloses a blueberry maturity detection system, comprising:

[0134] The image acquisition module 101 is used for acquiring blueberry images using a unmanned aerial vehicle;

[0135] The image preprocessing module 102 is used for preprocessing the acquired blueberry images, and constructing a model training set, a verification set and a test set;

[0136] The model construction module 103 is used for building an improved yolov8n model;

[0137] The model training and optimization module 104 is configured to train the improved yolov8n model using a model training set, and verify and test the improved yolov8n model using a verification set and a test machine, and complete optimization of the improved yolov8n model.

[0138] The detection module 105 is configured to use the optimized improved yolov8n model to detect the maturity of blueberries.

[0139] It should be noted that the specific implementation steps of each module of the detection system described in the embodiments of the present application can refer to the foregoing detection method, which will not be described here. The method and system have the advantages of high detection accuracy and high robustness for blueberry maturity detection.

Claims

1. A method of detecting blueberry ripeness, characterized by The method comprises the following steps: S1, collecting blueberry images using a drone; S2, preprocessing the collected blueberry images, constructing a model training set, a validation set and a test set; S3, building an improved yolov8n model; S4, training the improved yolov8n model using the model training set, and verifying and testing the improved yolov8n model using the validation set and the test set, to complete optimization of the improved yolov8n model; S5, using the optimized improved yolov8n model to detect the maturity of blueberries.

2. The blueberry ripeness detection method of claim 1, wherein, The improved yolov8n model in step S3 comprises a model Backbone, a model Neck and a model Head; the model Backbone is connected with the model Head through the model Neck, and the model Backbone comprises a plurality of first Conv modules, a plurality of EFCSP modules, an SCDown module and an SPPF module; the EFCSP module comprises a second Conv module, a Split module and a plurality of series-connected high-efficiency mixed attention modules FasterEMA.

3. A blueberry ripeness detection method as claimed in claim 2, characterized in that, The processing method of the high-efficiency mixed attention module FasterEMA comprises the following steps: First, an input feature map x is received, and it is judged whether the input channel number inc is consistent with the internally set channel number dim; if not, a group of 1×1 convolution operations is used to adjust the channels of the input feature, so that the channel number of the input feature is consistent with the internally set channel number; Second, the feature map after channel adjustment is reserved as a residual branch shortcut, and is input into a local spatial feature mixing module spatial_mixing; The channel dimension is divided into two parts by a preset division parameter n_div, one part has a channel number dim_conv3, and the other part has a channel number dim_untouched; for the channel region corresponding to the channel number dim_conv3, a convolution operation partial_conv3 with a convolution kernel size of 3×3, a stride of 1 and a padding of 1 is used for local feature extraction; the channel number dim_untouched part remains unchanged; The convolution operation Partial_conv3 module supports two forward propagation modes: when set to split_cat mode, the input feature map is first split into a convolution processing part and an untreated part along the channel dimension, local convolution and reservation operations are performed respectively, and then feature splicing is performed along the channel dimension, so as to realize the effect of local enhancement and overall feature fusion; When set to slicing mode, the local convolution operation is directly applied to the previous part of the channel of the original feature map, and the remaining part of the channel feature remains unchanged. Then, the locally spatially mixed feature map is input into a multi-layer perception module MLP; the MLP module includes two layers of 1x1 convolution, the first layer increases the number of channels of the feature map to dim multiplied by mlp_ratio, and the second layer restores it to dim; through this process, cross-channel information interaction and feature expression capability enhancement are realized.

4. The blueberry maturity detection method of claim 3, wherein: The feature map after local spatial mixing is input into a channel enhancement module EMA, which is constructed based on a grouping mechanism. The input feature map is divided into multiple subgroups along the channel dimension by setting a grouping factor groups, and each subgroup independently models attention to improve the expression capability and computational efficiency of local features.

5. A blueberry ripeness detection method as claimed in claim 4, characterized in that, The processing method of the channel enhancement module EMA includes the following steps: The input feature map is grouped and reshaped according to the number of groups, and each subgroup is operated separately; for each subgroup, adaptive average pooling operations are performed along the horizontal pool_h and vertical pool_w directions; The horizontal and vertical direction pooled features are concatenated and fused through a convolution operation with a 1x1 convolution kernel, then separated into horizontal and vertical features along the channel direction, and Sigmoid activation is applied respectively as the attention weight of the spatial dimension; The original grouped features are multiplied element by element with the attention weight of the spatial dimension, and group normalization is applied to enhance the stability of the feature distribution; at the same time, to supplement the local fine-grained features, a 3x3 convolution operation is used to further process the grouped features, and global features are extracted through adaptive average pooling; The features extracted by the above two paths are normalized by Softmax respectively, and a weighting matrix is calculated; the weight information and the feature map are fused through matrix multiplication to generate the final weight map; the grouped feature map is weighted element by element according to the weight map, and restored to the original feature map size, completing the spatial context enhancement of the input feature map; The features after attention enhancement processing are element-wise added to the initial residual branch to form the final output result of the module.

6. The blueberry ripeness detection method of claim 2, wherein, The improved yolov8n model uses an improved bounding box regression loss function PIoU for model optimization, and the processing process is as follows: The overlap penalty is defined by the following formula: overlap_penalty=max(0,IoU-γ)×w; The PIoU with overlap penalty is defined by the following formula: LossCIoU is defined by the following formula: wherein CIoU is a bounding box regression loss function adopted by the original model, a is a shape consistency term, which measures the aspect ratio difference between the predicted box and the real box; v is a normalized index measuring the angle difference between the predicted aspect ratio and the real aspect ratio, is an index for normalizing the center point distance difference in target detection; overlap_penalty is an occlusion penalty term, γ is an occlusion IoU threshold, and w is a penalty scaling coefficient; when the IoU exceeds the threshold γ, the penalty term of PIoU takes effect, and in the process of gradient back propagation, an additional constraint is imposed on the loss function of the model, guiding the model to pay more attention to the boundary details and actual outline of the occluded fruits.

7. The blueberry maturity detection method of claim 6, wherein: For each sample, the PIoU Loss and the normalized Wasserstein distance loss NWDLoss are calculated respectively, and the two losses are weighted and fused according to a predetermined proportion to form the final bounding box regression loss Loss_bbox: Loss_bbox=α×LossPIoU+(1-α)×LossNWD; Wherein, LossPIoU represents the optimized weighted IoU loss, LossNWD represents the normalized Wasserstein distance loss, and a is a fusion weight factor, the value range of which is [0, 1], and is used for dynamically balancing the contribution proportion of the two losses in the total loss.

8. The blueberry ripeness detection method of claim 2, wherein: The model Head adopts a lightweight detection head structure, and a depth separable convolution DSConv module is introduced in the regression branch for a regression task. The depth separable convolution DSConv decomposes a standard convolution operation into two sub-operations: depth convolution and point convolution. The depth convolution is applied to each input channel independently, focusing on extracting local spatial features in a single channel. The point convolution uses a 1x1 convolution kernel to linearly weight and combine all input channel features at each spatial position, effectively fusing different channel features. The depth convolution first extracts and fuses local features in each channel independently, learning the fine-grained feature relationship within the channel. Then, the point convolution strengthens the information interaction and integration between different channels through a cross-channel weighting combination mechanism.

9. The blueberry ripeness detection method of claim 2, wherein: The processing method of the DSConv module includes the following steps: Let the channel number of the input feature map be C in , the output channel number be C out , the convolution kernel size be k x k, and the height and width of the feature map be H and W, respectively. The parameter quantity and the calculation quantity of the standard convolution are shown in the following formula: P standard = C in x C out x k 2 ; O standard =C in xC out xk 2 xH xW; where P standard is the parameter quantity of the standard convolution, O standard is the computation quantity of the standard convolution; The parameter quantity and the calculation quantity of the depth separable convolution are shown in the following formula: P depthwise = C in x k 2 ; P pointwise = C in x C out ; where P depthwise is the parameter amount of the depth convolution, P pointwise is the parameter amount of the point convolution, and P separable is the parameter amount of the depth separable convolution as a whole. P separable = P depthwise + P pointwise = C in x k 2 + C in x C out ; O depthwise = C in x k 2 x H x W; O pointwise =C in xC out xH xW; O separable = O depthwise + O pointwise = C in x k 2 x H x W + C in x C out x HxW; where O depthwise is the computation of the depthwise convolution, O poimtwise is the computation of the pointwise convolution, and O separable is the computation of the depthwise separable convolution as a whole.

10. A blueberry ripeness detection system characterized by It comprises: An image acquisition module is configured to acquire images of blueberries using a UAV; An image preprocessing module is configured to preprocess the acquired images of blueberries, and construct a model training set, a validation set and a test set; A model construction module is configured to build an improved yolov8n model; A model training and optimization module is configured to train the improved yolov8n model using the model training set, and validate and test the improved yolov8n model using the validation set and the test set, thereby optimizing the improved yolov8n model; A detection module is configured to detect the maturity of blueberries using the optimized improved yolov8n model.

Images