A multi-target fruit recognition method under complex background

Abstract

This paper provides a method for multi-target fruit recognition in complex backgrounds. It uses GhostNet as the backbone network to achieve network lightweighting. A DBM module, consisting of depthwise separable convolution and Mish activation, replaces the CBL module in Neck to further reduce model memory usage. An efficient channel attention mechanism (ECA) is introduced at key locations in the network to suppress interference from complex backgrounds. Furthermore, a soft DIoU-NMS algorithm is used to enhance the recognition of overlapping or occluded fruit. This method can accurately and rapidly identify multi-target fruit in complex growing environments. It can be applied to harvesting robots, optimize their machine vision systems, and promote high-quality and efficient agricultural development.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method for identifying multi-target fruits under complex backgrounds. Background Art

[0002] Since the 1980s, robotics technology has gradually matured and been applied to agricultural production, greatly improving the efficiency of harvesting agricultural products. However, in complex growing environments, lush branches and foliage on fruit trees severely obscure the fruit, making the operation of agricultural harvesting robots more difficult, thus affecting harvesting efficiency. Agricultural harvesting robots still face bottlenecks such as low precision and slow speed in sensing target fruit. Therefore, improving the accuracy and speed of target fruit recognition in complex growing environments is of great practical significance for increasing efficiency and production in the fruit industry.

[0003] Given the importance of target perception in harvesting robots, researchers both domestically and internationally have explored fruit recognition methods based on cutting-edge technologies such as spectral analysis, image processing, and machine learning. Spectral analysis, which can capture rich information beyond the visible light range, has been used by many researchers to achieve fruit recognition. However, due to the high cost of hyperspectral acquisition equipment, methods for identifying target fruits through spectral analysis have not been widely adopted. In recent years, researchers have gradually shifted their focus to capturing target fruit images under natural lighting conditions using low-cost visible light cameras. As a result, a number of methods have emerged that use image processing techniques to identify multiple target fruits. However, these methods require manual setting of fixed thresholds and rely on human experience. They are particularly sensitive to cluttered natural light, resulting in unstable and robust recognition results that are difficult to meet the requirements of harvesting robots operating in complex scenarios.

[0004] Compared to the aforementioned technologies, machine learning has lower equipment costs and can generate adaptive thresholds through learning. Therefore, machine learning has become a key technology for agricultural harvesting robots to achieve target recognition. Because crop fruits have distinct characteristics such as color and shape, fruit recognition can be achieved through shallow machine learning combined with image processing technology. Commonly used shallow machine learning methods include support vector machines (SVM) and naive Bayes. The above scheme can achieve multi-target fruit recognition in structured environments, but it relies on tedious manual feature engineering[] and is easily affected by complex environmental factors such as illumination changes, overlap, and occlusion. It has poor robustness and generalization, making it difficult to meet the actual work requirements of harvesting robots.

[0005] In recent years, deep learning methods have gradually replaced shallow machine learning methods and become more effective tools for precision agriculture. Deep convolutional neural networks (CNNs) can achieve "end-to-end" feature extraction. Currently, agricultural harvesting robots face the following difficulties in recognizing fruit: (1) The natural lighting environment is complex, and the image quality under different lighting angles varies greatly; (2) Under natural conditions, the growth states of fruits are diverse, randomly distributed, and overlap or are obscured by branches and leaves; (3) The convolutional neural network recognition model based on deep learning takes up a lot of memory, is difficult to load onto mobile terminals, and has poor portability. Summary of the Invention

[0006] To address the challenges of existing technologies, this paper proposes a method for multi-target fruit recognition in complex backgrounds, primarily for use in harvesting robots. GhostNet is used as the backbone network to achieve network lightweighting. A DBM module, consisting of depthwise separable convolutions and Mish activation functions, replaces the CBL module in Neck to further reduce model memory usage. An efficient channel attention mechanism (ECA) is introduced at key locations in the network to mitigate interference from complex backgrounds. Furthermore, a softDIoU_NMS algorithm is employed to enhance the recognition of overlapping or occluded fruit.

[0007] The technical solution adopted by the present invention is as follows: a method for multi-target fruit recognition under complex background, comprising the following steps:

[0008] Step 1: A lightweight HPL-YOLOv4 network model is established based on the YOLOv4 network structure as the target recognition basic model, where the lightweight feature extraction network GhostNet is used as the backbone network;

[0009] Step 2: Introduce depthwise separable convolution and Mish activation function into the neck network;

[0010] Step 3: Add the ECA attention mechanism to the junction of the backbone network and the neck, upsampling, and downsampling in the HPL-YOLOv4 network model.

[0011] Step 4: In the prediction output part of the HPL-YOLOv4 network model, flexible DIoU_NMS is introduced to optimize the screening of prediction boxes;

[0012] Step 5: Collect and create a data set, and use the data set to complete the training of the HPL-YOLOv4 network model;

[0013] Step 6: Use the trained HPL-YOLOv4 network model to complete the recognition of multiple target fruits in complex backgrounds.

[0014] Furthermore, in step 1, the three feature maps of different sizes (13×13, 26×26, and 52×52) in the GhostNet network are combined with the traditional YOLOv4 for multi-scale feature splicing.

[0015] Furthermore, in step 2, the convolution operation Conv in the original CBL module of the neck network is replaced by the depthwise separable convolution DWConv, and the nonlinear activation function Mish is used to replace the original Leaky ReLU activation function.

[0016] Furthermore, the flexible DIoU_NMS optimization process is as follows: the prediction box with the highest score is set as the reference box, and then the DIoU is calculated with the remaining prediction boxes in the current target respectively, and the prediction boxes with DIoU less than the set threshold are retained; for the prediction boxes with DIoU greater than the set threshold, the confidence score of the box is continuously reduced, and some boxes with higher scores are regarded as correct detection boxes in subsequent calculations.

[0017] Furthermore, the flexible DIoU_NMS is expressed as follows:

[0018]

[0019] Among them, S i is the confidence score of the current prediction box; M is the prediction box with the highest confidence among all prediction boxes; b i N is all the compared prediction boxes in the current target; t is the set threshold, generally 0.5; σ is the penalty coefficient;

[0020] E is the prediction box with the highest current confidence score, F is one of the remaining prediction boxes, b is the center of box F, and b gt is the center of the frame E, ρ 2 (b,b gt ) represents the square of the distance d between the centers of box E and box F 2 , c represents the diagonal length of the minimum closure area of ​​box E and box F.

[0021] Furthermore, in step 5, the collected original data consists of fruit images at different time periods, different lighting conditions and different angles. The image types are consistent with the actual conditions of fruit growth in a natural environment, including front lighting, back lighting, overlap and occlusion. At the same time, the original data is expanded.

[0022] Furthermore, after data expansion, the fruits in the images were manually labeled using the LabelImg tool, and the labeled files were stored in the PASCAL VOC dataset format; the dataset was randomly divided into a training set, a test set, and a validation set in a ratio of 7:2:1.

[0023] Furthermore, during training, the backbone network part loaded with pre-trained weights is first frozen, the second half of the network is trained, and then the backbone network part is unfrozen to train all layers.

[0024] Compared with the existing technology, the beneficial effects of adopting the above technical solution are: the present invention can accurately and quickly identify multiple target fruits in complex growth environments, can be applied to picking robots, and optimize the machine vision system of the picking robots, and can promote high-quality and efficient development of agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of the multi-target fruit recognition method under complex background proposed by the present invention.

[0026] Figure 2 This is the HPL-YOLOv4 network model structure diagram proposed in this invention.

[0027] Figure 3 Schematic diagram of DIoU calculation in one embodiment of the present invention.

[0028] Figure 4 Schematic diagram of fruit images collected in one embodiment of the present invention.

[0029] Figure 5 A schematic diagram comparing the training loss curves of the YOLOv3, YOLOv4, and HPL-YOLOv4 models. DETAILED DESCRIPTION

[0030] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar modules or modules with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. On the contrary, the embodiments of the present application include all changes, modifications, and equivalents that fall within the spirit and scope of the appended claims.

[0031] like Figure 1 As shown, this embodiment proposes a multi-target fruit recognition method under complex background, including the following steps:

[0032] Step 1: A lightweight HPL-YOLOv4 network model is established based on the YOLOv4 network structure as the target recognition basic model, where the lightweight feature extraction network GhostNet is used as the backbone network;

[0033] Step 2: Introduce depthwise separable convolution and Mish activation function into the neck network;

[0034] Step 3: Add the ECA attention mechanism to the junction of the backbone network and the neck, upsampling, and downsampling in the HPL-YOLOv4 network model.

[0035] Step 4: In the prediction output part of the HPL-YOLOv4 network model, flexible DIoU_NMS is introduced to optimize the screening of prediction boxes;

[0036] Step 5: Collect and create a data set, and use the data set to complete the training of the HPL-YOLOv4 network model;

[0037] Step 6: Use the trained HPL-YOLOv4 network model to complete the recognition of multiple target fruits in complex backgrounds.

[0038] Specifically, YOLOv4's backbone feature extraction network is CSPDarknet53, which consists of a CBM module (Conv, BN, Mish) and five CSPX modules (where X is 1, 2, 8, 8, and 4, respectively). The CSPX module consists of a CBM module and a Resunit residual component. CSPDarknet53 introduces the Mish activation function to replace the Leaky ReLU activation function. The Mish function has the characteristics of being unbounded, lower bounded, non-monotonic, infinite-order continuous, and smooth, which helps regularize the model and stabilize the network gradient flow.

[0039] YOLOv4's neck network is a feature enhancement module, primarily composed of SPP (Spatial Pyramid Pooling) and PANet. SPP employs maximum pooling of 1×1, 5×5, 9×9, and 13×13 to perform multi-scale fusion on the 13×13×1024 feature layer obtained by CSPDarknet53. PANet transfers and fuses high-level feature information through upsampling and conveys the positioning information of low-level features through downsampling. This allows for parameter aggregation of the backbone feature layer and the detection layer, accelerating the fusion of features at different scales.

[0040] YOLOv4's Yolo Head uses these features for prediction. Traditional YOLOv4 first generates multiple prediction boxes for the target during the prediction phase. It then sorts all the prediction boxes by confidence score, from highest to lowest. It then calculates the intersection over union (IoU) between the box with the highest score and the remaining boxes. Predictions exceeding a set threshold are removed, and this process repeats until all prediction boxes have been traversed. This process is known as non-maximum suppression (NMS).

[0041] However, the YOLOv4 network has high accuracy but a large number of parameters, requiring high storage space and making it difficult to migrate to the mobile terminal of the harvesting robot. To build a lightweight object detection model suitable for harvesting robots, and to reduce the number of parameters in the YOLOv4 network, the HPL-YOLOv4 (High Precision and Lightweight YOLOv4) network model was constructed.

[0042] like Figure 2 As shown in the figure, the HPL-YOLOv4 network model is based on the YOLOv4 network structure and uses the lightweight feature extraction network GhostNet as the backbone network. Figure 2 As can be seen in the figure, the three feature maps of different sizes (13×13, 26×26, and 52×52) in the GhostNet network are combined with the traditional YOLOv4 for multi-scale feature splicing, so that the YOLOv4 network not only preserves the global features in the backbone feature extraction stage, but also reduces the overall network computation.

[0043] Among them, GhostNet is a lightweight feature extraction network built by Ghost bottleneck, in which Ghostbottleneck is composed of a stack of Ghost modules. Compared with basic convolution, Ghost module can extract more features with fewer parameters. For the input feature layer, the Ghost module uses conventional convolution operations to generate part of the real feature layer, and then performs a linear transformation on each channel of the real feature layer to obtain the Ghost feature layer. Finally, the real feature layer and the Ghost feature layer are concat-concatenated to obtain the complete output feature layer. Assuming that the input feature map is h×w×c, the output feature map is h'×w'×n, the convolution kernel size is k×k, and the input feature layer is divided into s parts, the computational complexity of standard convolution is as follows:

[0044] h'×w'×n×k×k×c (1)

[0045] The calculation amount of Ghost module is:

[0046]

[0047] It can be seen from formula (2) that the Ghost module is equivalent to splitting the multiplication operation of the standard convolution into two multiplication additions. Compared with the standard convolution, the model compression rate is about s, which greatly reduces the model calculation amount.

[0048] Furthermore, a large number of CBL modules (composed of standard convolution operation Conv, batch normalization BN and Leaky relu activation function) are used in the Neck (neck network) of the original YOLOv4 network structure, which is one of the key reasons for the huge number of YOLOv4 parameters. In the HPL-YOLOv4 network model proposed in this embodiment, depthwise separable convolution (DWConv) is introduced to replace the standard convolution operation Conv to further reduce the model size. Depthwise separable convolution is composed of channel-by-channel convolution (DW) combined with point-by-point convolution (PW). Compared with the standard convolution operation, the depthwise separable convolution has lower parameters; at the same time, the nonlinear activation function Mish is used to replace the original Leaky relu activation function. The smoothness of Mish enables feature information to penetrate deep into the neural network, which can promote information transmission, thereby improving the stability and accuracy of the network. The improved module is named DBM. The DBM module structure is as follows: Figure 2 As shown in the partial figure.

[0049] In order to prevent non-detection objects from interfering with the detection of target fruits, and to solve the problem of decreased detection accuracy of occlusion and overlapping targets caused by modifying the backbone network, an efficient channel attention mechanism ECA is introduced at the key position of Neck to further enhance the ability of feature representation to distinguish targets. If the ECA module is used for all DBM layers, it will increase the amount of invalid calculation. Therefore, in this embodiment, the ECA attention mechanism is added at the splicing point of the backbone network and Neck, upsampling and downsampling in the model, such as Figure 2 shown.

[0050] Among them, the channel attention mechanism (ECA) focuses on the correlation between channels in deep neural networks and is an effective method for improving network performance. Common channel attention mechanisms include SENet and ECANet. The ECA attention mechanism is an improvement on SE. It is a local cross-channel interaction strategy without dimensionality reduction and a method for adaptively selecting the size of a one-dimensional convolution kernel. ECA uses a sigmoid function to generate the weight proportion of each feature channel and combines the original H×W×C input features with the channel weights. This assigns large weights to important features in the original features, achieving focused attention. Ineffective features are assigned small weights to achieve autonomous suppression. Ultimately, features with channel attention are obtained.

[0051] During the prediction phase, the existing NMS algorithm calculates the Intersection over Union (IoU) value between the detection box with the highest prediction score and all other detection boxes, and removes detection boxes whose IoU values ​​exceed the NMS threshold. However, in real-world fruit picking environments, multiple target fruits often overlap densely. Blindly removing prediction boxes with IoU values ​​greater than the set threshold may suppress prediction boxes belonging to different fruits, leading to missed detection of occluded and overlapping fruits. To address this issue, this embodiment introduces a soft DIoU_nms (soft DIoU_nms) algorithm in the prediction output of the HPL-YOLOv4 network model.

[0052] DIoU_nms can change the calculation method of IoU (Intersection over Union) in the NMS algorithm. The IoU calculation in NMS is as follows:

[0053]

[0054] Where E is the prediction box with the highest current confidence score, and F is one of the remaining prediction boxes.

[0055] The DIoU is calculated as follows:

[0056]

[0057] like Figure 3 As shown, the black box represents the box with the highest current confidence score, which is denoted as box E; the yellow box represents one of the remaining prediction boxes, which is denoted as box F; in formula (4), b is the center of box F, and b gt is the center of the frame E. 2 (b,b gt ) represents the square of the distance d between the centers of box E and box F 2 , c represents the diagonal length of the minimum enclosed area between boxes E and F. DIoU is used instead of IoU when filtering predicted regression boxes. This further considers the distance between the center points of the bounding boxes, improving the accuracy of the predicted regression boxes. When filtering other redundant bounding boxes, soft DIoU_nms does not directly delete all boxes larger than the threshold, but instead reduces their confidence. The specific method is as follows:

[0058]

[0059] Where S i is the confidence score of the current prediction box; M is the prediction box with the highest confidence among all prediction boxes; b i N is all the compared prediction boxes in the current target; t is the set threshold, generally 0.5; σ is the penalty coefficient.

[0060] Equation (5) shows that Soft DIoU_nms sets the highest-scoring prediction box as the reference box. It then calculates the DIoU with the remaining prediction boxes in the current target, retaining those with a DIoU less than the set threshold. For prediction boxes with a DIoU greater than the set threshold, its score is not directly set to 0, but its confidence score is continuously reduced. Some high-scoring boxes may be considered correct detection boxes in subsequent calculations. Therefore, using soft DIoU_nms can effectively improve the detection performance of occluded and overlapping fruit.

[0061] In this example, the proposed HPL-YOLOv4 network model is verified using Ehime No. 38 citrus as an example. Data collection is required for verification. The collected data consists of citrus images at different time periods, different lighting conditions, and different angles. The image types are consistent with the actual conditions of citrus growth in natural environments, including front lighting, back lighting, overlap, and occlusion. A total of 1,371 images with clear target outlines and textures were selected, and some of the data are presented as follows. Figure 4 shown.

[0062] Furthermore, image processing techniques such as rotation, mirroring, noise addition, and brightness modification were used to expand the original data to 12,336 images to enhance the generalization of the model and avoid overfitting. Citrus fruits in the images were manually annotated using the LabelImg tool, and the generated annotation files were stored in the PASCAL VOC dataset format. The dataset was randomly divided into training, test, and validation sets in a ratio of 7:2:1. In the test set, A represents samples with an average occlusion degree of less than 30%, i.e., light occlusion. Conversely, B represents samples with heavy occlusion. The dataset division is shown in Table 1.

[0063] Table 1. Distribution of dataset numbers

[0064]

[0065] The experimental environment used for verification is shown in Table 2:

[0066] Table 2 Experimental environment

[0067]

[0068] The constructed HPL-YOLOv4 network model was trained using transfer learning. To allocate more resources for training the second half of the network, the backbone network, loaded with pretrained weights, was first frozen. This second half of the network was then trained, and then the backbone network was unfrozen to train all layers. Freezing some training layers effectively prevented damage to the backbone weights and accelerated training efficiency.

[0069] In order to objectively measure the detection performance of the proposed network model for the target fruit, recall rate (Recall), accuracy (Precision), F1 value (F1-score), AP value, detection speed, total network parameters and model size are introduced for evaluation. The calculation of Recall, Precision, F1 and AP are shown in formulas (6) to (9).

[0070]

[0071]

[0072]

[0073]

[0074] Where TP (True Positive) is the number of true positive samples, FP (False Positive) is the number of false positive samples, and FN (False Negative) is the number of false negative samples. AP is the area enclosed by the PR (Precision-Recall) curve and the coordinate axis, ranging from 0 to 1. r is the integral variable of the product of recall and precision.

[0075] In unstructured orchards, it is inevitable that branches and leaves will block the fruits or the fruits will overlap with each other. Therefore, it is crucial to test the detection accuracy of the model for fruit targets with different occlusion levels. Taking the occlusion level as the control variable, we take the test sets A and B and A+B (see Table 1, Figure 4 ) as experimental data, Figure 4 (a) Figure 4 (b) Figure 4 (c) shows the detection results of the proposed HPL-YOLOv4 network model in three real-world scenarios: overlapping occlusion, frontal illumination, and backlighting. Table 3 shows that HPL-YOLOv4 significantly improves on all metrics compared to the YOLOv3 and YOLOv4 models, regardless of whether the model is subjected to light or heavy occlusion. Notably, even in environments with heavy occlusion and dense objects, HPL-YOLOv4 still achieves an AP value of 97.13% and an F1 score of 0.93.

[0076] Table 3 Comparison of detection results at different occlusion levels

[0077]

[0078] The loss curves of the HPL-YOLOv4 model, YOLOv3, and YOLOv4 on the training set are as follows: Figure 5As shown in the figure, the final loss value of YOLOv3 is approximately 13.31, while the loss of YOLOv4 is approximately 12.39. The final loss value of the HPL-YOLOv4 model is approximately 2.40, which is 80.63% lower than the final loss value of the original YOLOv4. The results show that the HPL-YOLOv4 network model proposed in this example has better accuracy and faster convergence speed during the training process.

[0079] We selected 200 images each of front-lit, side-lit, and back-lit images from the lightly occluded dataset A as experimental subjects to test the model's robustness under different lighting angles. The detection results are shown in Table 4. Compared with other detection models, the HPL-YOLOv4 model achieved the lowest variance and the highest performance in all performance indicators under front-lit, side-lit, and back-lit conditions, indicating that HPL-YOLOv4 is more robust to changes in lighting angle.

[0080] Table 4 Comparison of detection results of YOLOv3, YOLOv4 and HPL-YOLOv4 models at different illumination angles

[0081]

[0082] Classic object detection algorithms such as Faster-RCNN and SSD were compared with the proposed HPL-YOLOv4. The results from testing on the full test set are shown in Table 5. The FPS and detection speed of each model were obtained using a GPU on images with a resolution of 416×416. The comparison results show that HPL-YOLOv4 has stronger overall performance, capable of detecting target fruits in images in real time. Its small memory footprint makes it suitable for real-time performance and easy portability to mobile platforms.

[0083] Table 5 Comparison of results of different detection methods

[0084]

[0085] Note: The best results are shown in bold in the table.

[0086] Analysis of data from comparative experiments with various models shows that in unstructured orchards, the HPL-YOLOv4 network significantly outperforms classic object detection models such as Faster-RCNN, SSD, and YOLOv4. Especially in complex backgrounds with severe occlusion and high overlap, the HPL-YOLOv4 network achieves a 9.41% increase in AP and a 2.36% increase in Precision compared to the traditional YOLOv4 network. The model size is reduced by 5 / 6, and the detection speed reaches 44 frames per second. On a GPU, the inference speed for a single 416×416 image is 23ms. Therefore, HPL-YOLOv4 boasts higher detection accuracy, a lighter model size, and faster detection speed. HPL-YOLOv4 is suitable for deployment on embedded devices and mobile terminals, providing a solid theoretical foundation and technical support for the development of agricultural harvesting robots. It can be seen that the present invention focuses on the key technology of fruit recognition of fruit picking robots, proposes a fusion mechanism of deep network structure and basic functional modules under the condition of parameter reduction, clarifies the interference relationship of complex environmental factors on the performance of network models, and overcomes the difficult problems of low multi-target fruit recognition rate and poor transplantability in unstructured orchards, providing a feasible solution for the upgrading and transformation of agricultural machinery and equipment.

[0087] It should be noted that, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "setting" and "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances; the drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations.

[0088] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A multi-target fruit recognition method under complex background, characterized in that: The following steps are involved: Step 1: A lightweight HPL-YOLOv4 network model is established based on the YOLOv4 network structure as the target recognition basic model, where the lightweight feature extraction network GhostNet is used as the backbone network; Step 2: Introduce depthwise separable convolution and Mish activation function into the neck network; Step 3: Add the ECA attention mechanism to the junction of the backbone network and the neck, upsampling, and downsampling in the HPL-YOLOv4 network model. Step 4: In the prediction output part of the HPL-YOLOv4 network model, flexible DIoU_NMS is introduced to optimize the screening of prediction boxes; Step 5: Collect and create a data set, and use the data set to complete the training of the HPL-YOLOv4 network model; Step 6: Use the trained HPL-YOLOv4 network model to complete the recognition of multiple target fruits in complex backgrounds.

2. The multi-target fruit recognition method under complex background according to claim 1, characterized in that In step 1, three feature maps of different sizes (13×13, 26×26, and 52×52) in the GhostNet network are combined with traditional YOLOv4 for multi-scale feature splicing.

3. The multi-target fruit recognition method under complex background according to claim 1 or 2, characterized in that In step 2, the convolution operation Conv in the original CBL module of the neck network is replaced by the depthwise separable convolution DWConv, and the nonlinear activation function Mish is used to replace the original Leaky relu activation function.

4. The multi-target fruit recognition method under complex background according to claim 1, characterized in that The flexible DIoU_NMS optimization process is as follows: the prediction box with the highest score is set as the reference box, and then the DIoU is calculated with the remaining prediction boxes in the current target, and the prediction boxes with DIoU less than the set threshold are retained; for the prediction boxes with DIoU greater than the set threshold, the confidence score of the box is continuously reduced, and some boxes with higher scores are regarded as correct detection boxes in subsequent calculations.

5. The multi-target fruit recognition method under complex background according to claim 4, characterized in that Flexible DIoU_NMS is expressed as follows: Among them, S i is the confidence score of the current prediction box; M is the prediction box with the highest confidence among all prediction boxes; b i N is all the compared prediction boxes in the current target; t is the set threshold, generally 0.5; σ is the penalty coefficient; E is the prediction box with the highest current confidence score, F is one of the remaining prediction boxes, b is the center of box F, and b gt is the center of the frame E, ρ 2 (b,b gt ) represents the square of the distance d between the centers of box E and box F 2 , c represents the diagonal length of the minimum closure area of ​​box E and box F.

6. The multi-target fruit recognition method under complex background according to claim 1, characterized in that In step 5, the collected original data consists of fruit images at different time periods, different lighting conditions and different angles. The image types are consistent with the actual conditions of fruit growth in a natural environment, including front lighting, back lighting, overlap and occlusion. The original data is expanded at the same time.

7. The multi-target fruit recognition method under complex background according to claim 6, characterized in that After data expansion, the LabelImg tool was used to manually annotate the fruits in the images, and the annotated files were stored in the PASCAL VOC dataset format; the dataset was randomly divided into training set, test set, and validation set in a ratio of 7:2:

1.

8. The multi-target fruit recognition method under complex background according to claim 1, characterized in that During training, the backbone network part loaded with pre-trained weights is first frozen, the second half of the network is trained, and then the backbone network part is unfrozen to train all layers.

Images