Field severely-shielded fruit detection method based on task-driven generative repair

Abstract

The invention relates to the field of field severely-shielded fruit detection, in particular to a field severely-shielded fruit detection method based on task-driven generative repair. According to the technical scheme, the method comprises the steps that an adversarial mask generation module based on uncertainty guidance generates an adversarial sample, the adversarial mask generation module is configured at the input end of a detection network, and for an input original field image, the information entropy of prediction probability distribution of each pixel in an image space is calculated; generating a task-driven adversarial mask based on the information entropy of each pixel prediction probability distribution, generating an adversarial sample by using the simulated leaf texture tensor and the adversarial mask, and outputting a repaired feature map through a feature-level generative repair module coupled to the backbone network; a multi-task joint adversarial optimization loss function is constructed, an alternate optimization strategy is adopted for training, and a detection network with optimal robustness is obtained; and the severely shielded fruits in the field are detected through the detection network with the optimal robustness. The method is suitable for detection of severely shielded fruits in the field.

Description

Technical Field

[0001] This invention relates to the field of detection of severely shaded fruits in the field, and specifically to a method for detecting severely shaded fruits in the field based on task-driven generative repair. Background Technology

[0002] In modern smart agriculture, vision-based automatic fruit recognition and localization technology is a prerequisite for automated harvesting, yield prediction, and precision plant protection by agricultural robots. Currently, single-stage object detection algorithms, represented by the YOLO (You Only Look Once) series, have become the mainstream solution for agricultural visual perception due to their high inference speed and excellent global perception capabilities. These convolutional neural network-based methods can achieve high-precision fruit recognition under experimental environments with uniform lighting and simple backgrounds.

[0003] However, such purely visual detection solutions have inherent physical limitations in real-world field scenarios: in the natural environment where fruit trees are lush and fruits grow in clusters, "leaf occlusion" and "fruit overlap" are extremely common. Especially under conditions of severe occlusion (occlusion rate exceeding 50%) or "same-color occlusion" (such as unripe fruits being obscured by green leaves), key features of the target (such as contours and textures) are severely lost, leading to a large number of missed and false detections in conventional detection models.

[0004] To address this issue, mainstream solutions in the industry typically employ data augmentation techniques (such as Mixup, Mosaic, and random erasure) to expand the training samples, attempting to improve the model's robustness to occlusion. Alternatively, attention mechanisms can be introduced to enhance the model's focus on visible regions.

[0005] Nevertheless, most existing data augmentation methods are based on "passive" interference with random pixels, lacking a realistic simulation of occlusion environments; and existing detection models are essentially "discriminative" models, lacking the ability to "generate" or "repair" missing features. Therefore, how to construct a detection method that can "actively simulate high-difficulty occlusion and intelligently repair missing features" has become a core technical challenge that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for detecting heavily shaded fruits in the field based on task-driven generative repair, which improves the target detection accuracy in unstructured field environments with heavy shading of branches and leaves and interference from backgrounds of the same color scheme.

[0007] This invention achieves the above objectives by adopting the following technical solution: This invention provides a method for detecting severely shaded fruits in the field based on task-driven generative repair, comprising:

[0008] S1. An adversarial mask generation module based on uncertainty generates adversarial samples;

[0009] S101, the adversarial mask generation module is configured at the input end of the detection network. For the input original field image, it calculates the information entropy of the predicted probability distribution of each pixel in the image space.

[0010] S102, Adversarial mask generated based on information entropy of the predicted probability distribution for each pixel;

[0011] S103. Generate adversarial samples using simulated leaf texture tensors and adversarial masks.

[0012] S2. Output the repaired feature map through the feature-level generative repair module coupled to the backbone network;

[0013] S3. Construct a multi-task joint adversarial optimization loss function, which includes target detection loss, feature-aware consistency loss and adversarial generation loss. The target detection loss is used to ensure that the repaired features are accurately identified, and the feature-aware consistency loss is used to constrain the features generated by the feature-level generative repair module to conform to the real physical distribution.

[0014] S4. Use an alternating optimization strategy for training to obtain the detection network with the best robustness;

[0015] S5. Detect severely shaded fruits in the field using a detection network with optimal robustness.

[0016] Furthermore, step S101 specifically includes:

[0017] If the input original field image is The category probability feature map output by the detection network is as follows For image space coordinates For each pixel, calculate the information entropy of the predicted probability distribution for that pixel. ;

[0018] ;

[0019] In the formula, This indicates the total number of target categories for the detection task. This indicates that the detection network predicts that the pixel belongs to the first... The probability value of the class. It represents a constant.

[0020] Furthermore, step S102 specifically includes:

[0021] Define a binarized adversarial mask The generating function, through a preset dynamic uncertainty threshold. Filter out high-confidence regions:

[0022] ;

[0023] In the formula, Indicates an indicator function, This indicates the target area of ​​the fruit defined by the Ground Truth bounding box, restricting occlusion to only occur on the fruit.

[0024] Furthermore, in step S103, generating adversarial examples specifically includes:

[0025] ;

[0026] In the formula, This represents the simulated leaf texture tensor. This indicates an adversarial example.

[0027] Furthermore, step S2 specifically includes:

[0028] If the damaged feature map input to the feature-level generative repair module is First, a multi-scale dilated residual convolution combined with feature concatenation is used to obtain the concatenated feature map, as follows:

[0029] ;

[0030] In the formula, This represents the spliced ​​feature map. : indicates the expansion rate Dilated convolution operation, , This indicates a feature concatenation operation performed along the channel dimension;

[0031] The repaired feature map is generated by constructing residual connections, as follows:

[0032] ;

[0033] In the formula, This represents the feature map after repair. express Convolutional layer This represents a non-linear activation function.

[0034] Furthermore, in step S3, the multi-task joint adversarial optimization loss function is performed. for:

[0035] ;

[0036] In the formula, , as well as These represent the hyperparameter weight coefficients of the three losses, used to balance the gradient contributions of each task;

[0037] Indicates the target detection loss. for:

[0038] ;

[0039] In the formula, This represents the CloU bounding box regression loss. This represents the category classification loss, BCEWithLogits. Indicates the distribution focus loss. Represents the predicted bounding box. This represents the predicted category distribution. The label represents the actual bounding box. Indicates the actual category label;

[0040] This represents the feature-aware consistency loss, calculated by the repaired features. Features corresponding to the unoccluded original image The L2 norm distance between them is obtained, and the calculation method is as follows:

[0041] ;

[0042] Indicates adversarial generation loss, Set to minimize the negative logarithm of the detection confidence:

[0043] ;

[0044] In the formula, Conf This represents the average confidence level of the detection network against the adversarial sample output.

[0045] Furthermore, step S4 specifically includes:

[0046] Phase 1:

[0047] Freeze the parameters of the adversarial mask generation module and input the adversarial sample. calculate +λcon Based on the calculated values, the parameters of the detection network and the feature-level generative instigation module are updated using gradient descent.

[0048] Phase Two:

[0049] Parameters of the frozen detection network and the feature-level generative instigation module are calculated. The parameters of the adversarial mask generation module are updated through gradient descent.

[0050] Through iterative cycles of the first and second stages, a Nash equilibrium is eventually reached, resulting in a detection network with optimal robustness.

[0051] The beneficial effects of this invention are as follows:

[0052] This invention abandons the practice of existing solutions (such as Random Erasing and Cutout) that "randomly" cover black blocks or noise on images. By introducing an adversarial mask generation module, it utilizes the real-time feedback (uncertainty entropy value) of the detection network to accurately lock the region that the model is "most confident" about and generate adversarial occlusion with realistic leaf texture.

[0053] This design transforms the training process from a static "data feeding" to a dynamic "attack and defense drill." The system can automatically identify and attack the model's weaknesses (i.e., samples that the model can easily identify based on local textures), forcing the model to abandon shortcuts and instead learn more robust global contours and contextual features. This significantly improves the model's generalization ability when faced with unknown and complex occlusion patterns in the field.

[0054] This invention addresses the inherent limitation of existing YOLO series models in their inability to perform effective reasoning when features are missing. It embeds a feature-level generative inpainting module into the backbone network. Utilizing a long-range context-aware field constructed through multi-scale dilated convolutions, it performs semantic reconstruction and completion of occluded regions at the feature map dimension.

[0055] A fruit physically obscured by more than 50% of its branches and leaves is often filtered out as background in traditional detection networks due to its weak feature response. This invention, however, utilizes the surrounding environment (such as the direction of branches and the edges of the fruit) to fill in the missing features, thereby reactivating the detection head's response to the target. This "feature self-healing" mechanism fundamentally solves the problem of missed detections caused by severe occlusion and fruit stacking, significantly improving the accuracy of field yield prediction and harvesting location.

[0056] This invention introduces a multi-task joint adversarial optimization strategy, constructing a hybrid game objective function that includes detection loss, adversarial generation loss, and feature-aware consistency loss. Through an alternating optimization strategy, the generator engages in a zero-sum game with either the repair or detection network.

[0057] This design avoids the awkward situation in traditional methods where "the restored image looks great, but the detection network can't recognize it." The Nash equilibrium achieved through adversarial training ensures that the restored features are specifically optimized for the detection task (i.e., strong semantic consistency). This provides an algorithmic guarantee with extremely high robustness and fault tolerance for agricultural robots operating in harsh conditions with large variations in lighting and severe occlusion. Attached Figure Description

[0058] Figure 1 This is a flowchart of a method for detecting severely shaded fruits in the field based on task-driven generative repair provided by the present invention.

[0059] Figure 2 This is a structural block diagram of an intelligent sensing system for severely shaded fruits in the field provided by the present invention;

[0060] Figure 3 This is a flowchart of the feature-level generative repair module provided by the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0062] The principles of this invention will be explained first.

[0063] This invention abandons the randomness and open-loop characteristics of traditional data augmentation methods and constructs a closed-loop feedback that integrates "adversarial occlusion generation - feature-level information reconstruction - highly robust target detection".

[0064] The system inputs of this invention include:

[0065] Input A (Raw Field Observation Data): RGB image data collected from unstructured agricultural environments. This data includes fruit targets under different light conditions and growth stages, and contains statistically diverse natural shading samples.

[0066] Input B (Inference State Feedback Signal): The uncertainty heatmap and classification entropy value calculated in real time by the detection model during forward propagation, used to characterize the current confidence distribution of the detection network in the local features of the image.

[0067] The core technical solution of this invention lies in using input B (inference state feedback signal) as an adversarial guidance variable to drive the occlusion generation and feature repair process, which is mainly achieved through the following methods:

[0068] An adversarial mask generation module based on uncertainty-guided generation generates an adversarial mask:

[0069] The adversarial mask generation module is configured at the input end to receive the feedback signal (input B) from the detection network. Its logic is to locate the fruit region with the highest confidence response of the current detection network through a gradient ascent strategy, and use a differentiable texture synthesis algorithm to generate an adversarial occlusion mask with real leaf texture features in this region, realizing the dynamic transformation from "easy-to-detect samples" to "difficult-to-detect samples".

[0070] The feature-level generative repair module coupled to the backbone network outputs the repaired feature map:

[0071] This feature-level generative inpainting module is coupled in parallel to the backbone feature extraction network of the detection network. It utilizes multi-scale dilated convolutions to construct a context-aware field, aiming to reconstruct and complete the semantic information of the occluded region along the feature map dimension, and then fuses the repaired feature tensor into the detection head to restore the integrity of the target features.

[0072] Multi-task joint combat optimization:

[0073] For end-to-end training of the system, a hybrid objective function was constructed, comprising object detection loss, adversarial generation loss, and feature-aware consistency loss. During training, the parameters of these modules were jointly optimized using the backpropagation algorithm, enabling the detection network to maintain high-precision object localization and classification performance through feature reconstruction mechanisms even under adversarial occlusion conditions.

[0074] The present invention will now be described in detail with reference to the accompanying drawings.

[0075] This invention provides a method for detecting severely shaded fruits in the field based on task-driven generative repair, such as... Figure 1 As shown, it specifically includes:

[0076] S1. An adversarial mask generation module based on uncertainty generates adversarial samples;

[0077] S101, the adversarial mask generation module is configured at the input end of the detection network. For the input original field image, it calculates the information entropy of the predicted probability distribution of each pixel in the image space.

[0078] If the input original field image is The category probability feature map output by the detection network is as follows For image space coordinates For each pixel, calculate the information entropy of the predicted probability distribution for that pixel. ;

[0079] ;

[0080] In the formula, This represents the total number of target categories for the detection task (e.g., fruit, leaves, background). ;

[0081] This indicates that the detection network predicts that the pixel belongs to the first... The probability value of the class, with a range of [0,1].

[0082] For a very small constant (e.g.) ), used to prevent errors in logarithmic operations. The numerical values ​​are unstable.

[0083] Physical meaning: The lower the value, the sharper the probability distribution, meaning the more confident the detection model is in classifying the region (usually corresponding to a clear, unobstructed core area of ​​the fruit). The higher the value, the more confused the detection model is.

[0084] S102, Adversarial mask generated based on information entropy of the predicted probability distribution for each pixel;

[0085] To attack the weak points of the model (i.e., the areas that the detection model considers the simplest), this invention defines a binarized adversarial mask. The generating function. Based on a preset dynamic uncertainty threshold. Filter out high-confidence regions:

[0086] ;

[0087] In the formula, This indicates an indicator function, which takes the value 1 when the condition within the parentheses is met, and 0 otherwise.

[0088] This represents the dynamic uncertainty threshold, which is adaptively adjusted during training based on the average entropy value of the current batch.

[0089] This indicates the target area of ​​the fruit defined by the Ground Truth annotation box, restricting occlusion to only occur on the fruit and avoiding background occlusion.

[0090] S103. Generate adversarial samples using simulated leaf texture tensors and adversarial masks.

[0091] To simulate a realistic field environment, this invention does not use pure black pixel blocks, but instead utilizes differentiable texture tensors. Generate adversarial examples

[0092] ;

[0093] This represents the Hadamard product, which is the element-wise multiplication of matrices. This represents the simulated leaf texture tensor. In this embodiment, the tensor is generated by the Berlin noise algorithm and matched with the green channel distribution of the current image through a color migration matrix to make it visually close to a real leaf.

[0094] Technical effect: generated It forcibly preserved the background and hard-to-detect areas, but precisely erased the originally clearest fruit details.

[0095] This necessitates that subsequent networks cannot rely on local features and must learn to utilize contextual information.

[0096] S2. Output the repaired feature map through the feature-level generative repair module coupled to the backbone network;

[0097] This feature-level generative inpainting module is coupled in parallel to the deep feature extraction layers of the YOLO detection backbone (such as before the SPPF layer). Its function is to perform semantic completion on the occluded regions along the feature map dimension.

[0098] Input definition: Let the damaged feature map of the input feature-level generative repair module be... ,in For the number of channels, The size of the feature map.

[0099] To capture both local textures (such as the apple peel) and global context (such as the connection between the apple and the tree branch) simultaneously, this invention employs a multi-scale dilated residual convolution structure.

[0100] First, a multi-scale dilated residual convolution combined with feature concatenation is used to obtain the concatenated feature map, as follows:

[0101] ;

[0102] In the formula, This represents the spliced ​​feature map. : indicates the expansion rate Dilated convolution operation, , Capture local detailed features, ,4 Exponentially expands the receptive field to perceive the movement of branches or leaves around the obscured fruit.

[0103] This indicates a feature concatenation operation performed along the channel dimension;

[0104] The repaired feature map is generated by constructing residual connections, as follows:

[0105] ;

[0106] In the formula, This represents the feature map after repair. express Convolutional layers are used to reduce the dimensionality of the stitched multi-scale features back to the original number of channels. This represents the nonlinear activation function SILU.

[0107] Physical meaning: The " in the formula The plus sign forms a residual connection. This means that the feature-level generative instigation module is only responsible for learning the incremental (residual) missing features, while preserving the original, unoccluded features. This ensures stable gradient propagation.

[0108] S3. Construct a multi-task joint adversarial optimization loss function;

[0109] Multi-task joint adversarial optimization loss function for:

[0110] ;

[0111] In the formula, , as well as These represent the hyperparameter weight coefficients of the three losses, used to balance the gradient contributions of each task;

[0112] This represents the object detection loss, which is used to ensure that the repaired features are accurately identified. for:

[0113] ;

[0114] In the formula, This represents the CloU bounding box regression loss. This represents the category classification loss, BCEWithLogits. Indicates the distribution focus loss. Represents the predicted bounding box. With real bounding box labels Together they were used to calculate the CloU bounding box regression loss to measure the accuracy of the network's prediction of fruit locations; This represents the predicted category distribution. With real category labels Together, they were used to calculate the BCEWithLogits class classification loss to measure the accuracy of the network in judging the target class.

[0115] This represents the feature-aware consistency loss, which is used to constrain the features generated by the feature-level generative instigation module to conform to the true physical distribution. It is calculated by... (The sentence is incomplete in the original text.) Features corresponding to the unoccluded original image The L2 norm distance between them is obtained, and the calculation method is as follows:

[0116] ;

[0117] This means using the original, unmasked image. Input the features extracted from the pre-trained frozen backbone network.

[0118] To represent the adversarial generation loss, for numerical stability, we will... Set to minimize the negative logarithm of the detection confidence:

[0119] ;

[0120] In the formula, Conf This represents the average confidence level of the detection network against the adversarial sample output.

[0121] Physical meaning: In the general formula The negative sign before (or gradient reversal in the optimization step) means that the more the adversarial mask generation module reduces the accuracy of the detection network, the smaller this loss will be.

[0122] S4. Use an alternating optimization strategy for training to obtain the detection network with the best robustness;

[0123] Phase 1:

[0124] Freeze the parameters of the adversarial mask generation module and input the adversarial sample. calculate +λcon Based on the calculated values, the parameters of the detection network and the feature-level generative instigation module are updated using gradient descent.

[0125] Phase Two:

[0126] Parameters of the frozen detection network and the feature-level generative instigation module are calculated. The parameters of the adversarial mask generation module are updated through gradient descent.

[0127] Through iterative cycles of the first and second stages, a Nash equilibrium is eventually reached, resulting in a detection network with optimal robustness.

[0128] S5. Detect severely shaded fruits in the field using a detection network with optimal robustness.

[0129] like Figure 2 As shown, this invention provides an intelligent sensing system for severely shaded fruits in the field. To achieve efficient collaboration between hardware and software, the system is physically divided into the following four core subsystems:

[0130] The multimodal vision acquisition subsystem is responsible for acquiring and digitizing raw environmental data, as detailed below:

[0131] The image acquisition unit, preferably a binocular RGB-D camera with a global shutter, is configured to acquire high-resolution video streams from the field. Its output data is defined as the input A of this invention, i.e., a high-dimensional tensor containing RGB color channels and depth channels. .

[0132] Data preprocessing unit: Built into the front-end FPGA (Field-Programmable Gate Array), it is used to perform automatic white balance and gamma correction to ensure the consistency of the input data distribution.

[0133] The edge computing inference subsystem, serving as the physical carrier of the algorithm, connects to the multimodal vision acquisition subsystem via a high-speed interface. Specifically:

[0134] AI computing power core: It adopts a high-performance embedded computing module based on ARM architecture and is equipped with TensorCore hardware acceleration unit.

[0135] Dynamic memory buffer: Used to temporarily store feature maps and gradient tensors of intermediate layers in deep neural networks.

[0136] The adversarial perception algorithm engine is the core software logic set running on the computing subsystem, comprising four core functional modules:

[0137] The confidence measurement module is used to calculate pixel-level information entropy.

[0138] The adversarial mask synthesis module is used to generate adversarial occlusion samples.

[0139] The feature-level generative repair module is used to repair damaged semantic information.

[0140] The target detection decoding module is used to output the final detection results.

[0141] The automatic operation control subsystem receives the three-dimensional coordinates (x, y, z) of the fruit and converts them into inverse kinematic control commands for the robotic arm.

[0142] The following provides a detailed description of an implementation of a model training method based on task-driven adversarial game theory.

[0143] Step 1: Measure the confidence level of the full-pixel discrimination;

[0144] The system inputs a batch of raw images into the detection network and extracts each spatial location on the feature map through the discrimination confidence measurement module. The corresponding category probability distribution vector The uncertainty value at this location was calculated based on the Shannon Entropy model. :

[0145] ;

[0146] In the formula, This indicates the total number of target categories for the detection task. This indicates that the detection network predicts that the pixel belongs to the first... The probability value of the class. It represents a constant.

[0147] This entropy value forms an "uncertainty heatmap" aligned with the original graph.

[0148] Step S2: Dynamic construction of difficult sample samples;

[0149] The anti-drowsing code synthesis module executes its attack strategy based on the aforementioned heatmap. First, it utilizes dynamic thresholds... Generate a binarized adversarial mask

[0150] ;

[0151] in For indicator functions, The target region is then determined using a differentiable texture tensor. Generate adversarial training samples

[0152] ;

[0153] Step 3: Contextual repair of feature-level semantic information;

[0154] adversarial examples Input the backbone network. The feature-level generative instigation module is activated, capturing the context through multi-scale dilated convolutions and outputting the instigated feature tensor via residual connections. :

[0155] ;

[0156] ;

[0157] in:

[0158] The input is the damaged feature map;

[0159] Indicates the expansion rate The dilated convolution operation;

[0160] for Projective convolutional layer This is the SiLU activation function.

[0161] Step S4: Calculation of joint loss for multiple objectives;

[0162] Construct a hybrid objective function with three constraints. :

[0163] ;

[0164] First item To detect accuracy loss, the constrained and repaired features can accurately regress the true label.

[0165] The second term is the perceived consistency loss, which uses L2 norm constraints to repair features. Approximating the features of the unobstructed original image

[0166] The third item is the loss incurred in combating attacks. This indicates the detection network's confidence in the adversarial examples. The negative sign before this term aims to drive the adversarial mask generation module to maximize the detection error. These are the hyperparameter weighting coefficients for each loss term.

[0167] The following provides a detailed description of an implementation example of online reasoning and feature self-healing processes.

[0168] During the actual field operation phase (i.e., inference mode), the system automatically freezes and bypasses the adversarial mask synthesis module.

[0169] At this point, the image containing real, natural occlusions, captured by the visual acquisition subsystem, flows through the backbone network. The feature-level generative inpainting module automatically responds to discontinuous regions in the feature map based on the contextual prior weights learned during the training phase, achieving "self-healing" of the feature map. Finally, the object detection decoding module outputs robust detection results.

[0170] like Figure 3 As shown, the feature-level generative repair module in this embodiment adopts a multi-branch parallel hole residual architecture, with the following specific configuration:

[0171] Parallel Branch 1 (Local Sensing Path): Configured as A standard convolutional layer with a dilation rate of r=1 is used to preserve high-frequency texture details in unoccluded areas.

[0172] Parallel Branch 2 (Mid-distance Context Path): Configured as A hollow convolutional layer with an expansion rate r=2 expands the equivalent receptive field to .

[0173] Parallel branch 3 (global semantic path): configured as A dilated convolutional layer with an expansion rate of r=4 is used to perceive the context of the tree structure across occlusions.

[0174] Feature aggregation layer: The output features of the above three branches are concatenated along the channel dimension, and then processed through a... The point convolutional layers are used for channel dimensionality reduction and feature fusion to achieve residual completion of the original feature flow.

[0175] Through the above-mentioned refined hardware architecture design and strict mathematical model constraints, the present invention effectively solves the problem of perception failure caused by occlusion in the existing technology in unstructured agricultural environments.

[0176] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for detecting severely shaded fruits in the field based on task-driven generative repair, characterized in that, include: S1. An adversarial mask generation module based on uncertainty generates adversarial samples; S101, the adversarial mask generation module is configured at the input end of the detection network. For the input original field image, it calculates the information entropy of the predicted probability distribution of each pixel in the image space. S102, Adversarial mask generated based on information entropy of the predicted probability distribution for each pixel; S103. Generate adversarial samples using simulated leaf texture tensors and adversarial masks. S2. Output the repaired feature map through the feature-level generative repair module coupled to the backbone network; S3. Construct a multi-task joint adversarial optimization loss function, which includes target detection loss, feature-aware consistency loss and adversarial generation loss. The target detection loss is used to ensure that the repaired features are accurately identified, and the feature-aware consistency loss is used to constrain the features generated by the feature-level generative repair module to conform to the real physical distribution. S4. Use an alternating optimization strategy for training to obtain the detection network with the best robustness; S5. Detect severely shaded fruits in the field using a detection network with optimal robustness.

2. The method for detecting heavily shaded fruits in the field based on task-driven generative repair according to claim 1, characterized in that, Step S101 specifically includes: If the input original field image is The category probability feature map output by the detection network is as follows: For image space coordinates For each pixel, calculate the information entropy of the predicted probability distribution for that pixel. ; ； In the formula, This indicates the total number of target categories for the detection task. This indicates that the detection network predicts that the pixel belongs to the first... The probability value of the class. It represents a constant.

3. The method for detecting severely shaded fruits in the field based on task-driven generative repair according to claim 2, characterized in that, Step S102 specifically includes: Define a binarized adversarial mask The generating function, through a preset dynamic uncertainty threshold. Filter out high-confidence regions: ； In the formula, Indicates an indicator function, This indicates the target area of ​​the fruit defined by the Ground Truth bounding box, restricting occlusion to only occur on the fruit.

4. The method for detecting severely shaded fruits in the field based on task-driven generative repair according to claim 3, characterized in that, In step S103, generating adversarial examples specifically includes: ； In the formula, This represents the simulated leaf texture tensor. Indicates adversarial examples, This represents element-wise multiplication of a matrix.

5. The method for detecting severely shaded fruits in the field based on task-driven generative repair according to claim 4, characterized in that, Step S2 specifically includes: If the damaged feature map input to the feature-level generative repair module is... First, a multi-scale dilated residual convolution combined with feature concatenation is used to obtain the concatenated feature map, as follows: ； In the formula, This represents the spliced ​​feature map. : indicates the expansion rate Dilated convolution operation, , This indicates a feature concatenation operation performed along the channel dimension; The repaired feature map is generated by constructing residual connections, as follows: ； In the formula, This represents the feature map after repair. express Convolutional layer This represents a non-linear activation function.

6. The method for detecting severely shaded fruits in the field based on task-driven generative repair according to claim 5, characterized in that, In step S3, the multi-task joint adversarial optimization loss function is performed. for: ； In the formula, , as well as These represent the hyperparameter weight coefficients of the three losses, used to balance the gradient contributions of each task; Indicates the target detection loss. for: ； In the formula, This represents the CloU bounding box regression loss. This represents the category classification loss, BCEWithLogits. Indicates the distribution focus loss. Represents the predicted bounding box. This represents the predicted category distribution. The label represents the actual bounding box. Indicates the actual category label; This represents the feature-aware consistency loss, calculated by the repaired features. Features corresponding to the unoccluded original image The L2 norm distance between them is obtained, and the calculation method is as follows: ； Indicates adversarial generation loss, Set to minimize the negative logarithm of the detection confidence: ； In the formula, Conf This represents the average confidence level of the detection network against the adversarial sample output.

7. The method for detecting severely shaded fruits in the field based on task-driven generative repair according to claim 6, characterized in that, Step S4 specifically includes: Phase 1: Freeze the parameters of the adversarial mask generation module and input the adversarial sample. calculate +λcon Based on the calculated values, the parameters of the detection network and the feature-level generative instigation module are updated using gradient descent. Phase Two: Parameters of the frozen detection network and the feature-level generative instigation module are calculated. The parameters of the adversarial mask generation module are updated through gradient descent. Through iterative cycles of the first and second stages, a Nash equilibrium is eventually reached, resulting in a detection network with optimal robustness.

Images