A rapid and accurate detection method for apple hiding

Abstract

The application discloses a kind of quick and accurate detection methods of sheltered apples, the method is based on deep learning instance segmentation model, and single apple is segmented from RGB image, and during model training, by increasing model branch and artificial labeling, guide model to learn how to extract, fuse and comprehensively consider the features of the apple around target apple, improve the identification ability of model to sheltered apple.The application effectively reduces the decline of detection model on the detection performance of dense, mutually sheltered apples, and the performance decline on the public dataset is only 50% of the original, improves the overall performance and robustness of the algorithm, can make accurate segmentation detection on sheltered apple, and the algorithm runs fast, only needs a little extra calculation, can be easily integrated with other models.

Description

Technical Field

[0001] This invention belongs to the field of intelligent robot target recognition technology, specifically a rapid and accurate detection method for occluded apples. Background Technology

[0002] In intelligent apple harvesting equipment, most existing apple target perception methods are based on image data collected by image sensors. Some methods use traditional image algorithms to detect apples. For example, Chinese patent CN110766698B discloses a method for tracking and recognizing oscillating apples in a dynamic background. The key technologies are binary image segmentation and template matching, which can segment apples from a dynamic background. Chinese patent CN110648359B discloses a fruit target localization and recognition method and system, which uses a graph theory-based segmentation method to segment and recognize the target region of a circular fruit target's RGB target. Another approach uses deep learning-based object detection or instance segmentation models to detect apples. For example, Chinese patent CN115880477A discloses an apple detection and localization method and system based on deep convolutional neural networks, which uses the YOLOv4 model to train on image data containing apple information to obtain target prediction boxes. Chinese patent CN112711985B discloses a fruit recognition method, device and robot based on an improved SOLO network, which uses an improved SOLO network recognition model to recognize fruit trees.

[0003] However, existing methods suffer from significant performance degradation when dealing with crowded, densely packed occluded apples. Specifically, when predicting the location of a particular apple, they are easily interfered with by adjacent apples with similar characteristics such as color and texture, leading to incomplete segmentation of the current apple or the inclusion of other apples in the current segment, resulting in undersegmentation and oversegmentation. Existing patented methods typically use general improvement modules to enhance the overall accuracy of the model when handling this type of occluded fruit problem. For example, Chinese patent publication CN202210565489.1 discloses an occluded and overlapping fruit recognition method for harvesting robots that uses a Transformer module to improve the YOLOv5 model and enhance overall semantic recognition capabilities. These methods are not specifically designed for the characteristics of occluded fruit, and performance gaps still exist between occluded and unoccluded apples. Summary of the Invention

[0004] The purpose of this invention is to address the problems existing in the prior art by providing a fast and accurate detection method for occluded apples, which can still accurately and effectively segment individual apples from an image when dealing with dense, crowded, and mutually occluded apples, without adding too much extra computation.

[0005] The technical solution for achieving the objective of this invention is: a rapid and accurate detection method for occluded apples, the method comprising the following steps:

[0006] Step 1: Acquire a color image of an apple using an image sensor and perform preprocessing.

[0007] Step 2: Build and train the YOLACT-Crowded model;

[0008] Step 3: Using the YOLACT-Crowded model, calculate the preprocessed image to obtain the model output, and convert the model output into a prediction result; the prediction result is a mask image of N individual apples, where the pixel value consists of 0 and 1, respectively indicating whether a certain pixel is an apple;

[0009] Step 4: Perform an image AND operation between the prediction result and the input image of the YOLACT-Crowded model to obtain N images of individual apples, where each individual apple image contains only a specific apple region.

[0010] Furthermore, the YOLACT-Crowded model is derived using deep learning methods with multiple sets of data; the multiple sets of data include: color images of apples and annotation files containing apple pixel location information and interfering apple pixel location information.

[0011] Further, step 2 involves building and training the YOLACT-Crowded model, specifically including:

[0012] Step 2-1, Constructing the dataset: Collect a large number of color images of apples and filter them according to clarity and content; label the filtered images as the label file; divide the dataset into training set, validation set and test set according to a certain ratio; use the labeled training set, validation set and test set as the multiple sets of data for YOLACT-Crowded model training;

[0013] Step 2-2, construct the YOLACT-Crowded model: including a basic model, an interference feature extraction module, and an interference target prediction head. The basic model includes a backbone network, a neck network, and a main target prediction head.

[0014] Steps 2-3: Train the YOLACT-Crowded model: Input the dataset into the YOLACT-Crowded model to obtain the model output, and convert the model output into prediction results; calculate the loss function value based on the prediction results and the annotation file; adjust the internal parameters of the YOLACT-Crowded model according to the loss function value and the optimization method, wherein the optimization method is gradient descent.

[0015] Steps 2-4: Optimize the training process: Adjust the hyperparameters of the YOLACT-Crowded model to control the training process and obtain multiple sets of model internal parameter configurations; compare the effects of multiple sets of model internal parameter configurations, and select the optimal model internal parameter configuration to complete the training and obtain the final YOLACT-Crowded model.

[0016] Further, in step 2-2, the backbone network uses the input image of the YOLACT-Crowded model as input, and performs convolutional operations through convolutional layers to extract features at one or more different scales. The neck network is used to fuse feature maps at various scales. The interference feature extraction module includes one or more convolutional layers to extract interference target-related features from the feature maps fused by the neck network. The main target detection head is used to fuse the interference target-related features and the output feature map of the neck network to obtain the model output. The interference target prediction head is used to output the interference target mask and the real interference mask annotation to calculate the loss and guide the interference feature extraction module to learn.

[0017] Furthermore, the model output includes: a first output, a second output, a third output, and / or a fourth output; the first output is W×H×N. a The first output is a probability value indicating the probability that an apple exists at a certain location; the second output is W×H×N. a The group bounding box offsets represent multiple predicted bounding boxes as a scaling factor relative to the prior box size and a coordinate offset relative to the prior position; the third output is N. c Zhang Apple Mask Prototype; the fourth output is N a ×N c The N apple mask coefficients; a The total number of prior bounding boxes is given by W and H, which are the width and height of a certain feature map, respectively. The interference target prediction head is only used during training and outputs a first auxiliary output and a second auxiliary output using the relevant features of the interference target. The first auxiliary output is the prototype of the interference target mask, and the second auxiliary output is the interference target mask coefficient.

[0018] Furthermore, steps 2-3 convert the model output into prediction results, specifically including the following steps:

[0019] Step 3-1: Use the third output as a weight to perform weighted fusion on the fourth output to obtain N. a A single apple's predicted mask image;

[0020] Step 3-2: Convert the second output into N by combining it with the prior bounding box. a A dense predicted bounding box indicating the predicted location;

[0021] Step 3-3: If the model is in the training phase, the densely predicted location bounding box is directly used as the final predicted box; otherwise, based on the densely predicted location bounding box and the first output corresponding to the densely predicted bounding box, confidence threshold filtering and non-maximum suppression are performed to obtain the final predicted bounding box.

[0022] Steps 3-4: Crop the predicted mask image according to the final predicted bounding box, keeping only the pixel values ​​of the mask image inside each bounding box, and setting the mask image outside the bounding box to 0; if in the model training stage, continue to execute steps 3-4, otherwise proceed to the next step;

[0023] Step 3-5: Replace the third and fourth outputs in step 3-1 with the first and second auxiliary outputs of the interference target prediction head to obtain N. a A single apple-shaped interference mask image;

[0024] Steps 3-6: Enlarge the final predicted bounding box with a preset coefficient, crop the interference mask image, retain only the pixel values ​​of the mask image inside each enlarged bounding box, and set the mask image outside the bounding box to 0.

[0025] Furthermore, the loss function in steps 2-3 is the L-function, with the following form:

[0026] L = L base +βL crowd

[0027] Among them, L base The original loss for the basic model is as follows:

[0028] L base =L cls +L bbox +L mask

[0029] Among them, L cls For classification loss, L bbox For bounding box localization loss, L mask For predicting loss of the apple mask, L crowd The interference mask prediction loss is β, which is a set hyperparameter used to control the ratio of the original loss to the interference mask loss.

[0030] Furthermore, the calculation of the loss function includes the following steps:

[0031] Step 4-1: Based on the densely predicted location bounding boxes and the first output corresponding to the densely predicted bounding boxes, perform confidence filtering and positive and negative sample allocation to obtain the ground truth label corresponding to each dense prediction; the ground truth label includes the ground truth bounding box, the ground truth apple mask, and the ground truth interference mask.

[0032] Step 4-2: Calculate L based on the positive and negative sample allocation results. cls ;

[0033] Step 4-3: Based on the positive and negative sample allocation results, the dense predicted location bounding boxes, and the ground truth bounding boxes, calculate the L value of the positive samples. bbox And calculate the mean;

[0034] Step 4-4: Based on the positive and negative sample allocation results, the predicted mask image, and the real apple mask, calculate the positive sample L. mask Then, a weighted average is calculated using the area of ​​the corresponding bounding box as the weight.

[0035] Steps 4-5: Based on the positive and negative sample allocation results, the interference mask image, and the real interference mask, calculate the L of the predicted-sample pair for positive samples where the real interference mask is greater than a certain threshold. crowd Then, a weighted average is calculated using the area of ​​the corresponding bounding box as the weight.

[0036] Furthermore, the features related to the interference target and the output feature map of the neck network are fused, specifically including:

[0037] Step 5-1: Extract the feature map of the deepest layer of the neck network;

[0038] Step 5-2: Check whether the interference target feature map has the same width and height as the deepest feature map. If the width and height are the same, proceed to step 5-4; if the width and height are different, proceed to step 5-3.

[0039] Step 5-3: Upsample or downsample the feature map of the interfering target to the width and height of the deepest feature map, then proceed to step 5-4;

[0040] Step 5-4: Add the corresponding elements of the interference target feature map and the deepest feature map to obtain the fused feature map.

[0041] Compared with the prior art, the significant advantages of this invention are:

[0042] (1) Based on the characteristics of the single-stage instance segmentation model, the present invention designs an interference mask prediction branch to explicitly guide the model to learn how to remove the interference of dense and overlapping apples around a single apple, thereby improving the overall performance and robustness of the algorithm and enabling accurate segmentation and detection of occluded apples.

[0043] (2) The YOLACT-OCC designed in this invention has basically the same number of parameters as the basic model, requires less additional calculation, can run quickly, and can be easily integrated with other models.

[0044] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the overall process of the rapid and accurate detection method for occluded apples according to the present invention.

[0046] Figure 2 This is a schematic diagram of the YOLACT-OCC model structure in one embodiment.

[0047] Figure 3 This is a comparison image of the occluded apple segmentation effect in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0050] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0051] In one embodiment, combined Figure 1 This provides a rapid and accurate method for detecting occluded apples, specifically including:

[0052] Step 1: Acquire a color image of an apple using an image sensor and perform preprocessing. The preprocessing steps include: processing the color image using scaling and / or cropping and / or filling methods to make its image dimensions consistent with preset values.

[0053] Step 2: Build and train the YOLACT-Crowded model;

[0054] Here, the YOLACT-Crowded model is derived using deep learning methods with multiple sets of data; the multiple sets of data include: color images of apples and annotation files containing apple pixel location information and interfering apple pixel location information.

[0055] Combination Figure 2 The YOLACT-Crowed model is trained using deep learning methods on a large number of apple images and their corresponding annotations.

[0056] Step 2-1, Constructing the dataset: Collect a large number of color images of apples and filter them according to clarity and content; label the filtered images as the label file; divide the dataset into training set, validation set and test set according to a certain ratio; use the labeled training set, validation set and test set as the multiple sets of data for YOLACT-Crowded model training;

[0057] Step 2-2, construct the YOLACT-Crowded model: including a basic model, an interference feature extraction module, and an interference target prediction head. The basic model includes a backbone network, a neck network, and a main target prediction head.

[0058] Here, the backbone network takes the input image of the YOLACT-Crowded model as input, and performs convolutional operations through convolutional layers to extract features at one or more scales. The neck network is used to fuse feature maps at various scales. The interference feature extraction module includes one or more convolutional layers to extract interference target-related features from the feature maps fused by the neck network. The main target detection head is used to fuse the interference target-related features and the output feature map of the neck network to obtain the model output. The interference target prediction head is used to output the interference target mask and the real interference mask annotation to calculate the loss and guide the interference feature extraction module to learn. Preferably, the basic model is the YOLACT model; the interference feature extraction module is a single-layer convolutional layer with a kernel size of 3 and a kernel number of 256. c =32.

[0059] The model output includes: a first output, a second output, a third output, and / or a fourth output; the first output is W×H×N. aThe first output is a probability value indicating the probability that an apple exists at a certain location; the second output is W×H×N. a The group bounding box offsets represent multiple predicted bounding boxes as a scaling factor relative to the prior box size and a coordinate offset relative to the prior position; the third output is N. c Zhang Apple Mask Prototype; the fourth output is N a ×N c The N apple mask coefficients; a The total number of prior bounding boxes is given by W and H, which are the width and height of a certain feature map, respectively. The interference target prediction head is only used during training and outputs a first auxiliary output and a second auxiliary output using the relevant features of the interference target. The first auxiliary output is the prototype of the interference target mask, and the second auxiliary output is the interference target mask coefficient.

[0060] Steps 2-3: Train the YOLACT-Crowded model: Input the dataset into the YOLACT-Crowded model to obtain the model output, and convert the model output into prediction results; calculate the loss function value based on the prediction results and the annotation file; adjust the internal parameters of the YOLACT-Crowded model according to the loss function value and the optimization method, wherein the optimization method is gradient descent.

[0061] Here, converting the model output into a prediction result involves the following steps:

[0062] Step 3-1: Use the third output as a weight to perform weighted fusion on the fourth output to obtain N. a A single apple's predicted mask image;

[0063] Step 3-2: Convert the second output into N by combining it with the prior bounding box. a A dense predicted bounding box indicating the predicted location;

[0064] Step 3-3: If the model is in the training phase, the densely predicted location bounding box is directly used as the final predicted box; otherwise, based on the densely predicted location bounding box and the first output corresponding to the densely predicted bounding box, confidence threshold filtering and non-maximum suppression are performed to obtain the final predicted bounding box.

[0065] Steps 3-4: Crop the predicted mask image according to the final predicted bounding box, keeping only the pixel values ​​of the mask image inside each bounding box, and setting the mask image outside the bounding box to 0; if in the model training stage, continue to execute steps 3-4, otherwise proceed to the next step;

[0066] Step 3-5: Replace the third and fourth outputs in step 3-1 with the first and second auxiliary outputs of the interference target prediction head to obtain N.a A single apple-shaped interference mask image;

[0067] Steps 3-6: Enlarge the final predicted bounding box with a preset coefficient, crop the interference mask image, retain only the pixel values ​​of the mask image inside each enlarged bounding box, and set the mask image outside the bounding box to 0.

[0068] Here, the loss function is the L-function, which has the following form:

[0069] L = L base +βL crowd

[0070] Among them, L base The original loss for the basic model is as follows:

[0071] L base =L cls +L bbox +L mask

[0072] Among them, L cls For classification loss, L bbox For bounding box localization loss, L mask For predicting loss of the apple mask, L crowd The interference mask prediction loss is β, which is a set hyperparameter used to control the ratio of the original loss to the interference mask loss.

[0073] The calculation of the loss function L includes the following steps:

[0074] Step 4-1: Based on the densely predicted location bounding boxes and the first output corresponding to the densely predicted bounding boxes, perform confidence filtering and positive and negative sample allocation to obtain the ground truth label corresponding to each dense prediction; the ground truth label includes the ground truth bounding box, the ground truth apple mask, and the ground truth interference mask.

[0075] Step 4-2: Calculate L based on the positive and negative sample allocation results. cls ;

[0076] Step 4-3: Based on the positive and negative sample allocation results, the dense predicted location bounding boxes, and the ground truth bounding boxes, calculate the L value of the positive samples. bbox And calculate the mean;

[0077] Step 4-4: Based on the positive and negative sample allocation results, the predicted mask image, and the real apple mask, calculate the positive sample L. mask Then, a weighted average is calculated using the area of ​​the corresponding bounding box as the weight.

[0078] Steps 4-5: Based on the positive and negative sample allocation results, the interference mask image, and the real interference mask, calculate the L of the predicted-sample pair for positive samples where the real interference mask is greater than a certain threshold. crowd Then, a weighted average is calculated using the area of ​​the corresponding bounding box as the weight.

[0079] Here, the features related to the interference target and the output feature map of the neck network are fused. The specific process includes:

[0080] Step 5-1: Extract the feature map of the deepest layer of the neck network;

[0081] Step 5-2: Check whether the interference target feature map has the same width and height as the deepest feature map. If the width and height are the same, proceed to step 5-4; if the width and height are different, proceed to step 5-3.

[0082] Step 5-3: Upsample or downsample the feature map of the interfering target to the width and height of the deepest feature map, then proceed to step 5-4;

[0083] Step 5-4: Add the corresponding elements of the interference target feature map and the deepest feature map to obtain the fused feature map.

[0084] Steps 2-4: Optimize the training process: Adjust the hyperparameters of the YOLACT-Crowded model to control the training process and obtain multiple sets of model internal parameter configurations; compare the effects of multiple sets of model internal parameter configurations, and select the optimal model internal parameter configuration to complete the training and obtain the final YOLACT-Crowded model.

[0085] Step 3: Using the YOLACT-Crowded model, calculate the preprocessed image to obtain the model output, and convert the model output into a prediction result; the prediction result is a mask image of N individual apples, where the pixel value consists of 0 and 1, respectively indicating whether a certain pixel is an apple;

[0086] Step 4: Perform an image AND operation between the prediction result and the input image of the YOLACT-Crowded model to obtain N images of individual apples, where each individual apple image contains only a specific apple region.

[0087] As a specific example, the invention will be further verified and illustrated in one embodiment.

[0088] The method of this invention was trained and tested on an open-source apple segmentation dataset. The YOLACT-OCC model structure described above was trained on the training set of the dataset and tested on the cross-validation set. The experimental results are shown in Table 1.

[0089] Table 1 Comparison of the effects of the basic model and the present invention.

[0090]

[0091] Experimental results show that the basic model exhibits significant performance degradation when facing densely packed, occluded apple targets, with a maximum Dice difference of up to 3%. In contrast, the method of this invention shows a significantly reduced performance degradation when facing apple targets with dense apple interference, with a maximum Dice difference of only 1%. Visualization results are shown below. Figure 3 As shown.

[0092] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A rapid and accurate detection method for occluded apples, characterized in that, The method includes the following steps: Step 1: Acquire a color image of an apple using an image sensor and perform preprocessing. Step 2: Build and train the YOLACT-Crowded model; Step 3: Using the YOLACT-Crowded model, calculate the preprocessed image to obtain the model output, and convert the model output into a prediction result; the prediction result is a mask image of N individual apples, where the pixel value consists of 0 and 1, respectively indicating whether a certain pixel is an apple; Step 4: Perform an image AND operation between the prediction result and the input image of the YOLACT-Crowded model to obtain N images of individual apples, wherein each image of an individual apple contains only a certain apple region. The YOLACT-Crowded model is derived using deep learning methods with multiple sets of data; the multiple sets of data include: color images of apples and annotation files containing apple pixel location information and interfering apple pixel location information; Step 2 involves building and training the YOLACT-Crowded model, specifically including: Step 2-1, Constructing the dataset: Collect a large number of color images of apples and filter them according to clarity and content; label the filtered images as the label file; divide the dataset into training set, validation set and test set according to a certain ratio; use the labeled training set, validation set and test set as the multiple sets of data for YOLACT-Crowded model training; Step 2-2, construct the YOLACT-Crowded model: including a basic model, an interference feature extraction module, and an interference target prediction head. The basic model includes a backbone network, a neck network, and a main target prediction head. Steps 2-3: Train the YOLACT-Crowded model: Input the dataset into the YOLACT-Crowded model to obtain the model output, and convert the model output into prediction results; calculate the loss function value based on the prediction results and the annotation file; adjust the internal parameters of the YOLACT-Crowded model according to the loss function value and the optimization method, wherein the optimization method is gradient descent. Steps 2-4: Optimize the training process: Adjust the hyperparameters of the YOLACT-Crowded model to control the training process and obtain multiple sets of model internal parameter configurations; compare the effects of multiple sets of model internal parameter configurations, and select the optimal model internal parameter configuration to complete the training and obtain the final YOLACT-Crowded model.

2. The rapid and accurate detection method for occluded apples according to claim 1, characterized in that, In step 2-2, the backbone network takes the input image of the YOLACT-Crowded model as input and performs convolutional operations through convolutional layers to extract features at one or more scales. The neck network is used to fuse feature maps at various scales. The interference feature extraction module includes one or more convolutional layers to extract interference target-related features from the feature maps fused by the neck network. The main target detection head is used to fuse the interference target-related features and the output feature map of the neck network to obtain the model output. The interference target prediction head is used to output the interference target mask and the real interference mask annotation to calculate the loss and guide the interference feature extraction module to learn.

3. The rapid and accurate detection method for occluded apples according to claim 2, characterized in that, The model output includes: a first output, a second output, a third output, and / or a fourth output; the first output is... The first output is a probability value indicating the probability that an apple exists at a certain location; the second output is... The group bounding box offset is represented by a scaling factor relative to the prior box size and a coordinate offset relative to the prior position, indicating multiple predicted bounding boxes; the third output is... Zhang Apple Mask Prototype; the fourth output is Apple mask coefficients; the This represents the total number of prior bounding boxes. , These represent the width and height of a certain feature map, respectively. The interference target prediction head is only used during training and outputs a first auxiliary output and a second auxiliary output using the relevant features of the interference target. The first auxiliary output is the prototype of the interference target mask, and the second auxiliary output is the interference target mask coefficient.

4. The rapid and accurate detection method for occluded apples according to claim 3, characterized in that, Steps 2-3 convert the model output into prediction results, specifically including the following steps: Step 3-1: Use the third output as a weight to perform weighted fusion on the fourth output to obtain... A single apple's predicted mask image; Step 3-2: Convert the second output into a priori bounding box. A dense predicted bounding box indicating the predicted location; Step 3-3: If the model is in the training phase, the densely predicted location bounding box is directly used as the final predicted box; otherwise, based on the densely predicted location bounding box and the first output corresponding to the densely predicted bounding box, confidence threshold filtering and non-maximum suppression are performed to obtain the final predicted bounding box. Steps 3-4: Crop the predicted mask image according to the final predicted bounding box, keeping only the pixel values ​​of the mask image inside each bounding box, and setting the mask image outside the bounding box to 0; if in the model training stage, continue to execute steps 3-4, otherwise proceed to the next step; Step 3-5: Replace the third and fourth outputs in step 3-1 with the first and second auxiliary outputs of the interference target prediction head, to obtain... A single apple-shaped interference mask image; Steps 3-6: Enlarge the final predicted bounding box with a preset coefficient, crop the interference mask image, retain only the pixel values ​​of the mask image inside each enlarged bounding box, and set the mask image outside the bounding box to 0.

5. The rapid and accurate detection method for occluded apples according to claim 4, characterized in that, The loss function in step 2-3 is The function has the following form: in, The original loss for the basic model is as follows: in, For classifying losses, For bounding box localization loss, Predicting losses for Apple masking To prevent interference masking from predicting loss, The hyperparameter is set to control the ratio of the original loss to the interference mask loss.

6. The rapid and accurate detection method for occluded apples according to claim 5, characterized in that, The calculation of the loss function includes the following steps: Step 4-1: Based on the densely predicted location bounding boxes and the first output corresponding to the densely predicted bounding boxes, perform confidence filtering and positive and negative sample allocation to obtain the ground truth label corresponding to each dense prediction; the ground truth label includes the ground truth bounding box, the ground truth apple mask, and the ground truth interference mask. Step 4-2: Calculate based on the positive and negative sample allocation results. ; Step 4-3: Based on the positive and negative sample allocation results, the dense predicted location bounding boxes, and the ground truth bounding boxes, calculate the positive samples'... And calculate the mean; Step 4-4: Based on the positive and negative sample allocation results, the predicted mask image, and the real apple mask, calculate the positive samples. Then, a weighted average is calculated using the area of ​​the corresponding bounding box as the weight. Steps 4-5: Based on the positive and negative sample allocation results, the interference mask image, and the real interference mask, calculate the predicted-sample pairs for positive samples where the real interference mask is greater than a certain threshold. Then, a weighted average is calculated using the area of ​​the corresponding bounding box as the weight.

7. The rapid and accurate detection method for occluded apples according to claim 6, characterized in that, The features related to the interference target and the output feature map of the neck network are fused. The specific process includes: Step 5-1: Extract the feature map of the deepest layer of the neck network; Step 5-2: Check whether the interference target feature map has the same width and height as the deepest feature map. If the width and height are the same, proceed to step 5-4; if the width and height are different, proceed to step 5-3. Step 5-3: Upsample or downsample the feature map of the interfering target to the width and height of the deepest feature map, then proceed to step 5-4; Step 5-4: Add the corresponding elements of the interference target feature map and the deepest feature map to obtain the fused feature map.

8. The rapid and accurate detection method for occluded apples according to claim 7, characterized in that, The basic model is the YOLACT model; the interference feature extraction module is a single-layer convolutional layer with a kernel size of 3 and a kernel count of 256. .

Images