A method and system for illumination fill light for agricultural detection

Abstract

The present application relates to the technical field of agricultural lighting, and particularly relates to a lighting compensation method and system for agricultural detection. The present application takes shadow as a valuable information source containing spatial priori, and enhances the feature representation ability through explicit modeling of shadow detection task. A shadow guided gating fusion mechanism is proposed, which dynamically adjusts the feature weight according to whether the spatial position is in the shadow area, and realizes adaptive enhancement. The mechanical arm is taken as a light source carrier, and the flexible adjustment of the light compensation position and intensity is realized through double fuzzy PID control, and the shadow part of the target area in the line of sight is illuminated. The present application significantly improves the performance and robustness of the agricultural visual detection system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural lighting, in particular to a lighting compensation method and system for agricultural detection. BACKGROUND

[0002] Intelligent agricultural technology has become an inevitable trend of modern agricultural development. Computer vision technology is the core support of intelligent agriculture. In orchard and other agricultural environments, the projected shadows of leaves and branches will cover a large part of the target to be detected, resulting in a significant decrease in image contrast. The dynamic changes in illumination at different times such as sunny days, cloudy days, dawn and dusk will also cause the same target to exhibit different visual characteristics. Non-uniform lighting will further cause local overexposure or underexposure. These lighting problems will directly lead to a decrease in target area brightness and loss of texture information, and further cause missed detection, increase the difficulty of feature extraction, and affect detection stability. As the core support of computer vision technology in intelligent agriculture, its role in fruit detection, automatic sorting, and quality assessment relies on accurate target detection. The lighting compensation method improves the lighting environment and lays a solid foundation for all subsequent intelligent operations, making up for the shortcomings of traditional manual detection, such as strong subjectivity, low efficiency, and high cost. It also strengthens the advantages of intelligent visual detection systems, such as fast speed, objective judgment, and all-weather work.

[0003] To address the lighting problem of agricultural visual detection, existing research has formed a multi-dimensional solution path, and related exploration of lighting compensation has gradually expanded. At the algorithm level, existing research mainly enhances the lighting robustness from the algorithm level (data enhancement, domain adaptation) or from the hardware level. At the same time, agricultural target detection technology has shifted from the traditional manual feature recognition paradigm to a deep learning-based approach, with detection algorithms such as ResNet, Faster R-CNN, Mask R-CNN, and YOLO series providing core technical support for agricultural visual systems. In related application research, the DeepFruits system, related exploration for orchard detection, visual algorithms for mango picking, and occlusion perception size estimation have made significant progress. At the hardware level, fixed lighting methods have been used to improve lighting conditions, and adaptive lighting control research has begun to explore dynamic control modes based on image quality assessment. In addition, some research has analyzed the impact of lighting changes on detection performance, and other research has optimized the IATEFF-YOLO algorithm architecture to adapt to low-light conditions. These methods provide ideas for alleviating the impact of lighting on agricultural detection from different angles.

[0004] Although the existing method has made some progress, there are still many obvious defects, which is difficult to fully meet the actual needs of agricultural visual detection. From the overall research, the research on active light compensation for visual detection tasks is relatively lacking. In the algorithm level, the existing detection network generally regards the shadow as interference that needs to be eliminated, and does not fully exploit the prior information such as spatial structure and occlusion relationship contained in the shadow, resulting in limited effect under extreme light, and unable to accurately detect the shadow area to be illuminated and compensated for light, and the detection performance is insufficient. SUMMARY

[0005] Therefore, the technical problem to be solved by the present application is to overcome the defects that the existing detection network generally regards the shadow as interference that needs to be eliminated, and does not fully exploit the prior information such as spatial structure and occlusion relationship contained in the shadow, resulting in that under extreme light, the shadow area to be illuminated and compensated for light cannot be accurately detected, and the illumination compensation precision is insufficient.

[0006] To solve the above technical problems, the present application provides a lighting compensation method for agricultural detection, comprising:

[0007] obtaining binocular images of a target area, and generating a shadow mask of each image in the binocular images and a feature pyramid of different scales of each image;

[0008] performing down-sampling operation on the shadow mask of each image, so that the spatial resolution of each down-sampled shadow mask is consistent with the spatial resolution of the feature pyramid of each scale of the image, and a down-sampled shadow mask corresponding to each scale of the image is obtained;

[0009] based on the feature pyramid of each scale of each image, the down-sampled shadow mask corresponding to each scale, a minimum gating value and an exponential factor, a gated feature corresponding to each scale of each image is obtained through multiplication gating;

[0010] based on the gated feature corresponding to each scale of each image, a prediction result corresponding to each scale of each image is obtained by using a detection head, and the prediction result of each image is processed by non-maximum suppression to obtain a final shadow detection result of each image. Based on the final shadow detection result of the binocular images, the shadow part of the target area is illuminated and compensated for light.

[0011] Preferably, the method for obtaining the gated feature corresponding to each scale of each image based on the feature pyramid of each scale of each image, the down-sampled shadow mask corresponding to each scale, the minimum gating value and the exponential factor through multiplication gating comprises:

[0012] based on the down-sampled shadow mask corresponding to each scale of each image, the minimum gating value and the exponential factor, a gating function corresponding to each scale of each image is obtained;

[0013] The gating function corresponding to each scale of each image is multiplied element-wise with the feature pyramid of that scale to obtain the gating feature corresponding to each scale of each image.

[0014] Preferably, the gating function for each scale of each image is obtained based on the downsampled shadow mask, minimum gate value, and exponential factor corresponding to each scale of each image, as shown in the formula:

[0015] ,

[0016] in, For the first The gating function corresponding to each scale This is the scale index of the feature pyramid. Minimum gate value, As an exponential factor, For the first The shadow mask after downsampling corresponding to each scale.

[0017] Preferably, after obtaining the prediction results for each scale of each image using the detection head, the prediction results include: the original confidence level logits; the original confidence level logits for each scale of each image is optimized, including:

[0018] Based on a pre-set confidence modulation index, the original confidence logits corresponding to each scale of each image, and the downsampled shadow mask, a gating mechanism is used to obtain the target confidence logits corresponding to each scale of each image; where the pre-set confidence modulation index is greater than the exponential factor.

[0019] Preferably, the target confidence level logits for each scale of each image is obtained through a gating mechanism based on a preset confidence modulation index, the original confidence level logits for each scale of each image, and the downsampled shadow mask. The formula is as follows:

[0020] ,

[0021] in, For the first The target confidence level logits corresponding to each scale For the first The original confidence scores logits corresponding to each scale To preset the confidence modulation index, For the first The shadow mask after downsampling corresponding to each scale.

[0022] Preferably, the method for illuminating the shadow part of the target region based on the final shadow detection result of the binocular image comprises:

[0023] Based on the final shadow detection result of the binocular image, the three-dimensional position of the center point of the target object corresponding to the shadow part in the mechanical arm coordinate system is obtained as the three-dimensional position of the light compensation point.

[0024] The light source target is set to be located above the target object corresponding to the shadow part, and the light source target position is obtained based on the three-dimensional position of the light compensation point and the initial illumination distance of the light source.

[0025] Based on the brightness of the shadow part, the target brightness, the calibration coefficient, and the initial illumination distance of the light source, the initial light intensity is obtained through the illumination model.

[0026] Based on the light source target position and the initial position of the end effector of the mechanical arm, the end effector of the mechanical arm is driven from the initial position to the light source target position through the mechanical arm position controller.

[0027] Based on the target brightness and the initial light intensity, the PWM duty cycle of the light source carried by the mechanical arm is controlled through the light source brightness controller until the brightness of the shadow part of the target region enters the target range, so as to realize the illumination compensation of the shadow part of the target region.

[0028] Preferably, the method for controlling the end effector of the mechanical arm to drive from the initial position to the light source target position based on the light source target position and the initial position of the end effector of the mechanical arm through the mechanical arm position controller comprises:

[0029] A control period is set, and every time a control period passes, the PID parameters of the mechanical arm controller in the next control period are obtained through the fuzzy inference mechanism based on the position error of the current control period, the position error change rate of the current control period and the last control period, and the L1 norm of the position error of the current control period is the L1 norm of the light source target position and the current control period end effector position.

[0030] Based on the PID parameters of the mechanical arm controller in the next control period, the end effector of the mechanical arm is moved through the mechanical arm position controller until the end effector of the mechanical arm is driven from the initial position to the light source target position.

[0031] Preferably, the method for controlling the PWM duty cycle of the light source carried by the mechanical arm based on the target brightness and the initial light intensity through the light source brightness controller until the brightness of the shadow part of the target region enters the target range comprises:

[0032] The PID parameters of the light source brightness controller in the next control period are obtained through a fuzzy inference mechanism based on the brightness error of the current control period, the brightness error change rate of the current control period and the last control period, wherein the brightness error of the current control period is the difference between the target brightness and the brightness of the shadow part in the current control period;

[0033] The light source control output in the current control period is obtained through the light source control law.

[0034] Based on the light source control output in the current control period, the light intensity increment of the light source in the next control period is obtained until the brightness of the shadow part of the target region enters the target range.

[0035] Preferably, the method for generating the shadow mask of each image in the binocular image and the feature pyramid of different scales respectively comprises:

[0036] Each image in the binocular image is input into the shadow detection branch respectively, and the shadow mask of each image is output.

[0037] Each image in the binocular image is input into the target detection branch respectively, and the feature pyramid of different scales of each image is output.

[0038] The application also provides a lighting compensation system for agricultural detection, comprising:

[0039] The feature extraction module is used to obtain the binocular image of the target region, and generate the shadow mask of each image in the binocular image and the feature pyramid of different scales respectively.

[0040] The down-sampling module is used to perform down-sampling operation on the shadow mask of each image, so that the spatial resolution of each down-sampled shadow mask is consistent with the spatial resolution of the feature pyramid of each scale of the image respectively, and the down-sampled shadow mask corresponding to each scale of the image is obtained.

[0041] The gated feature acquisition module is used to obtain the gated feature corresponding to each scale of each image through multiplication gating based on the feature pyramid of each scale of each image, the down-sampled shadow mask corresponding to each scale, the minimum gating value and the exponential factor.

[0042] The shadow detection module is used to obtain the prediction result corresponding to each scale of each image by using the detection head based on the gated feature corresponding to each scale of each image, and the final shadow detection result of each image is obtained by performing non-maximum suppression processing on the prediction result of each image.

[0043] The light compensation module is used to perform lighting compensation on the shadow part of the target region based on the final shadow detection result of the binocular image.

[0044] The above technical solutions of the present application have the following beneficial effects compared with the prior art:

[0045] The illumination light compensation method and system for agricultural detection provided by the present application adopt a multiplication gating mechanism for adaptive feature modulation for each scale feature pyramid. The gating function builds a nonlinear modulation logic through the synergistic effect of the minimum gating value and the exponential factor. For the shadow area, the gating value tends to 1, so that the target features in the shadow area are completely retained or even strengthened, effectively making up for the defects of blurred target features and low recognition in the shadow area. For the non-shadow area, the gating value is constrained within a certain range, which avoids the complete suppression of features in the normal illumination area while moderately suppressing the background noise and false target features in the non-shadow area, thereby reducing the misidentification causes from the source. In addition, before being sent to the detection head, the present application additionally introduces a second layer of confidence modulation gating mechanism, which further amplifies the confidence difference between the shadow area and the non-shadow area by performing secondary strengthening on the target score implied in the features through the confidence modulation index. The confidence score of the target in the shadow area remains basically unchanged, while the confidence score of the misidentified target in the non-shadow area is significantly reduced. Combined with the subsequent non-maximum suppression processing, the present application finally realizes the accurate filtering of misidentification in the non-shadow area. The above technical means are progressive, from the accurate extraction of shadow information, cross-scale matching, to adaptive modulation at the feature level and secondary strengthening at the confidence level, forming a complete shadow prior utilization link. The present application solves the problems of easy missed detection in the shadow area and misidentification caused by false targets in the non-shadow area in the traditional detection network, and finally realizes the dual improvement of target detection performance in complex lighting environments, thereby improving the illumination light compensation precision.

[0046] Moreover, based on the final shadow detection result, the three-dimensional position of the shadow corresponding target object in the mechanical arm coordinate system is extracted as the three-dimensional position of the light supplement point, the target position of the light source is planned above the target object combined with the initial illumination distance of the light source, and the initial light intensity is calculated through the illumination model of the fusion of the brightness of the shadow part, the target brightness, the calibration coefficient and the initial illumination distance. This step not only determines the quantitative correlation of "light supplement position - light intensity - detection performance", providing a scientific basis for the optimization of the light supplement scheme, but also accurately positions the three-dimensional position of the target, so that the mechanical arm can realize adaptive light supplement for the scattered and distributed targets in complex agricultural scenes, solving the problem that fixed light supplement cannot adapt to the diverse distribution of targets and the separation of light supplement and detection processes; the control period is set, the L1 norm of the target position of the light source and the current position of the end effector of the mechanical arm is taken as the position error, the error change rate is combined to dynamically update the PID parameters of the next period through fuzzy reasoning, the end effector is driven to accurately move from the initial position to the target position of the light source, and in the brightness control, the difference between the target brightness and the current shadow brightness is taken as the brightness error, the PID parameters of the light source brightness controller in the next period are obtained through fuzzy reasoning combined with the change rate, the control output is calculated through the light source control law, and the light intensity increment is obtained, and the PWM duty cycle of the light source is continuously adjusted until the shadow brightness enters the target range. This double closed-loop fuzzy PID control strategy can dynamically adapt to the nonlinearity and time-varying characteristics of the illumination-image response, replacing the limitations of traditional single control mode; in the whole process, the planning of the light supplement parameters depends on the detection result, and after the light supplement is executed, the shadow brightness is fed back through real-time image acquisition to adjust the control parameters, forming a closed-loop mechanism of "detection - light supplement - feedback - optimization", and the design of the mechanical arm carrying the light source is matched with the target position exceeding the workspace, which is projected to the reachable boundary", greatly improving the spatial flexibility of light supplement. The mode of actively planning the light supplement scheme based on the detection result replaces the passive perception of the prior art, and an active intelligent illumination regulation mechanism is constructed, which not only solves the problem of lack of accurate standard for light supplement effect evaluation, but also effectively breaks through the barrier of agricultural visual detection performance improvement in complex illumination environment, effectively improving the real-time performance and accuracy of the light supplement. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to make the content of the present application easier to be clearly understood, the present application will be further described in detail below according to specific embodiments of the present application and in conjunction with the drawings, in which:

[0048] Figure 1 is a flowchart of a light supplement method for agricultural detection.

[0049] Figure 2 is a structure diagram of a shadow-guided fusion detection network.

[0050] Figure 3is a structural diagram of an adaptive illumination light compensation system.

[0051] Figure 4 is a qualitative comparison of different models in a shadow scene.

[0052] Figure 5 is a position error change curve.

[0053] Figure 6 is a fuzzy PID speed output curve.

[0054] Figure 7 is a shadow area brightness rise control curve.

[0055] Figure 8 is a light source intensity adjustment rise curve.

[0056] Figure 9 is a shadow area brightness drop control curve.

[0057] Figure 10 is a light source intensity adjustment drop curve. DETAILED DESCRIPTION

[0058] The present application will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it.

[0059] Referring to Figure 1 Fig. 1, the embodiment one provides an illumination light compensation method for agricultural detection, comprising:

[0060] Step S1: acquiring binocular images of a target area, respectively generating a shadow mask of each image in the binocular images and a feature pyramid of different scales;

[0061] As Figure 2 shown, Figure 2 is a shadow-guided fusion detection network. The shadow-guided fusion detection network (Shadow-Guided YOLO fusion network) proposed by the present application is composed of three core modules. The three core modules include: a shadow detection branch (Shadow Detection Branch), an object detection branch (Object Detection Branch) and a spatial mask gating module (Spatial Mask Gating Module).

[0062] In the embodiment, specifically, the method for respectively generating a shadow mask of each image in the binocular images and a feature pyramid of different scales comprises:

[0063] input each image in the binocular images into the shadow detection branch respectively, and output the shadow mask of each image in the binocular images ;

[0064] In this embodiment, the shadow detection branch optionally adopts a stacked conditional GAN (ST-CGAN).

[0065] In this embodiment, the shadow detection branch optionally adopts a DSDNet as a backbone network, which is pre-trained on an ISTD dataset and has strong shadow segmentation capability. The encoder of the DSDNet uses a ResNeXt-101, and multi-scale features of each image are extracted by the DSDNet encoder, and five feature maps of different resolutions of each image are output , the resolution gradually decreases from 320x320 to 20x20; wherein, is the feature map of the i-th resolution;

[0066] After each image in the binocular image is passed through the DSDNet encoder and respectively passes through the five feature extraction modules of the DSDNet encoder , , , , , five feature maps of different resolutions are obtained , the five feature maps of different resolutions are compressed by their corresponding down blocks , , , , , before direction-aware enhancement, the channel width of each scale is kept consistent; wherein, the down block includes: 3x3 convolution (step 1, padding 1) and 1x1 convolution connected in turn, the 3x3 convolution is used to reduce the channel number of the backbone network, and the 1x1 convolution is used to project the channel number to 32 channels, and BN layer and ReLU activation function are used after the 3x3 convolution and the 1x1 convolution.

[0067] The direction-aware spatial module (DS) of the DSDNet is a core component, including two complementary sub-modules: FN sub-module (False Negative Suppression) and FP sub-module (False Positive Suppression);

[0068] Each resolution of the compressed feature map of each image is passed through the FN sub-module, the discriminative feature expression of the shadow boundary is enhanced by using the attention mechanism, and the missed detection is reduced, and the FN sub-module output feature of each resolution of each image is obtained, and the formula is:​

[0069] ,

[0070] wherein, is the FN sub-module output feature, is the FN sub-module input feature, is the main feature map, represents the attention mechanism, is the sigmoid activation function;

[0071] The feature map of each resolution of each image is passed through the FP sub-module, and the feature is refined by convolution operation to suppress false negative effects, to obtain the FP sub-module output feature of each resolution of each image, and the formula is:

[0072] ,

[0073] wherein, is the FP sub-module output feature, is the FP sub-module input feature.

[0074] The FP sub-module output feature and the FP sub-module output feature of each resolution of each image are respectively fused by element-wise addition to generate the DS feature of each resolution of each image, and the formula is:

[0075] ,

[0076] In order to make full use of multi-scale information, the DS feature of each resolution generates a side output prediction, and these side outputs are aggregated by a fusion layer into the final shadow mask logits, and the formula is:

[0077] ,

[0078] wherein, represents the shadow mask logits, is the probability map after sigmoid activation, is the side output prediction of the th resolution, represents a multi-scale fusion operator (implemented by convolution aggregation).

[0079] Each image in the binocular image is input into the target detection branch respectively, and a feature pyramid of different scales of each image is output.

[0080] The direction-aware spatial module of DSDNet can more accurately locate the shadow boundary, and the output shadow mask can be directly integrated into the target detection feature as spatial prior information. Second, DSDNet is based on ResNeXt-101 encoder, which outputs multi-scale feature maps, and the resolution level can be naturally aligned with the feature pyramid of YOLOv5, which is convenient for the scale-by-scale fusion of the subsequent spatial mask gating module. Third, DSDNet is pre-trained on the ISTD dataset, which has fast inference speed and is more suitable for real-time agricultural visual systems.

[0081] In this embodiment, the target detection branch optionally uses the DeepFruits system, which uses deep neural networks for fruit detection. DeepFruits is based on the Faster R-CNN architecture and belongs to a two-stage detection method. It has verified the effectiveness of deep learning methods in agricultural visual tasks on multiple fruit types.

[0082] In this embodiment, the target detection branch optionally uses the YOLOv5 architecture, which achieves a good balance between speed and accuracy. The Backbone part is composed of a Focus layer and three Conv+CSP modules, which extract multi-scale features through step-by-step downsampling. The Focus layer first reorganizes the 640x640x3 input into a 320x320x64 feature map through slicing. This design reduces the amount of calculation while preserving information. Subsequently, three Conv+C3 modules combine convolution downsampling with a step size of 2 and CSP Bottleneck structure to reduce the feature resolution to 160x160, 80x80, and 40x40, respectively.

[0083] The Neck part uses a combination of SPPF (Spatial Pyramid Pooling - Fast) and PANet (Path Aggregation Network). SPPF expands the receptive field through multi-scale pooling, allowing the network to capture global context information. PANet aggregates features through bottom-up and top-down paths, enhancing the network's expression ability at different scales.

[0084] Finally, the network outputs three different scale feature pyramids , which correspond to different scale target detection, represents the three different scale feature pyramids, is the number of feature channels for each scale, , , .

[0085] Step S2: Obtain the shadow mask of each image performing downsampling operations to make the spatial resolution of each of the downsampled shadow masks consistent with the spatial resolution of the feature pyramid of each scale of the image respectively, to obtain a downsampled shadow mask corresponding to each scale of the image , the formula is:

[0086] ,

[0087] wherein, is a scale corresponding to the downsampled shadow mask, is a bilinear interpolation downsampling operator, respectively represent the height and width of the feature pyramid of scale .

[0088] Step S3: based on the feature pyramid of each scale of each image, the downsampled shadow mask corresponding to each scale, the minimum gating value, and the exponential factor, a gated feature corresponding to each scale of each image is obtained through multiplicative gating, including:

[0089] In this embodiment, the feature pyramid of each scale of each image is adaptively modulated through the spatial mask gating module SMGM, and the specific process is as follows:

[0090] Step S31: based on the downsampled shadow mask corresponding to each scale of each image, the minimum gating value, and the exponential factor, a gating function corresponding to each scale of each image is obtained, and the gating function adopts a parameterized nonlinear mapping, and the formula is:

[0091] ,

[0092] wherein, is a gating function corresponding to the th scale, is a scale index of the feature pyramid, is a minimum gating value, is an exponential factor, is a downsampled shadow mask corresponding to the th scale.

[0093] In this embodiment, is 0.2, which is used to prevent the features in the non-shadow area from being completely suppressed, thereby causing performance degradation under normal illumination, is 1.5, which is used to control the degree of nonlinearity of the gating curve, and a larger value can amplify the response difference between the shadow area and the non-shadow area.

[0094] In a physical sense, when Time (corresponding to the shaded area), gate value The feature retains its integrity and strength; when Time (corresponding to the non-shaded area), gate value The features are moderately suppressed but not completely eliminated. Indicates the first The downsampled shadow mask corresponding to the i-th scale is the i-th Line number The elements of the column.

[0095] Step S32: Multiply the gating function corresponding to each scale of each image element-wise with the feature pyramid of that scale to obtain the gating feature corresponding to each scale of each image. The formula is as follows:

[0096] ,

[0097] in, For the first Gated features corresponding to each scale For the first Feature pyramids at various scales This indicates element-wise multiplication.

[0098] To determine the optimal gating scheme, this invention compared two designs in ablation experiments and ultimately selected multiplicative gating, which is computationally efficient and performs well, as the core mechanism of SGFD-Net. In addition to the aforementioned multiplicative gating, this invention explores another gating scheme: an attention-based gating scheme. This scheme designs a lightweight cross-attention mechanism that guides the attention allocation of features by using the shadow mask as a key.

[0099] ,

[0100] in, Represents the attention weight matrix. These are query, key, and value vectors, respectively. The dimension of the key vector. Set the residual connection weight (to 0.3).

[0101] The query vector is generated from the feature pyramid at the current scale using a 1×1 convolution:

[0102] ,

[0103] in, For the projection matrix, the number of channels is compressed to 1 / 8 of the original to reduce the computational load.

[0104] The key vector is generated from the shadow mask:

[0105] ,

[0106] wherein the mapping matrix extends the single-channel mask to the same dimension as Query.

[0107] The value (Value) vector is also extracted from the feature pyramid of the current scale:

[0108] ,

[0109] wherein, is the value projection matrix.

[0110] Step S4: obtaining the gated feature of each scale corresponding to each image obtaining the prediction result of each scale corresponding to each image by using the detection head obtaining the final shadow detection result of each image by performing non-maximum suppression (NMS) processing on the prediction result of each image The final shadow detection result includes the detection frame of the target object corresponding to the shadow part, for example, the target fruit in the shadow, the range of the shadow is usually large, including branches and leaves and other objects that do not need to be lighted, and the present application only identifies the target fruit in the shadow.

[0111] In the embodiment, preferably, the prediction result of each scale corresponding to each image is obtained by using the detection head, and the prediction result includes original confidence logits; the original confidence logits are used to represent the possibility that a pixel belongs to a shadow area.

[0112] The original confidence logits of each scale corresponding to each image are optimized, including:

[0113] Based on the pre-set confidence modulation index, the original confidence logits of each scale corresponding to each image, and the down-sampled shadow mask, the target confidence logits of each scale corresponding to each image are obtained by using a gating mechanism; wherein the pre-set confidence modulation index is greater than the exponential factor.

[0114] In the embodiment, preferably, the target confidence logits of each scale corresponding to each image are obtained by using the gating mechanism based on the pre-set confidence modulation index, the original confidence logits of each scale corresponding to each image, and the down-sampled shadow mask, so as to suppress the false detection of non-shadow areas, and the formula is:

[0115] ,

[0116] wherein, is the first target confidence logits corresponding to each scale, the first original confidence logits corresponding to each scale, a preset confidence modulation index, the first down-sampled shadow mask corresponding to each scale.

[0117] In the embodiment, 2.0, and the preset confidence modulation index is greater than the index factor, so that the confidence modulation has a stronger suppression effect. The operation is only performed in the inference stage, and false detection is suppressed by significantly reducing the confidence score of the non-shadow area.

[0118] In order to realize end-to-end joint optimization, the application designs a total loss function including three parts for the shadow-guided fusion detection network:

[0119] ,

[0120] wherein, the total loss function, the detection loss, the shadow supervision loss, the regularization loss, the loss weight coefficient.

[0121] The detection loss adopts the YOLOv5 standard loss function, including three items of bounding box regression (CIoU Loss), target confidence (BCE) and class classification. The loss ensures that the model has basic target detection capability.

[0122] In order to prevent the shadow branch from degrading in joint training, the application introduces pixel-level shadow supervision. The shadow supervision loss is composed of binary cross entropy (BCE) and Dice loss :

[0123] ,

[0124] wherein, the shadow ground truth mask, the predicted mask logits, the predicted mask probability map.

[0125] The Dice loss is defined as:

[0126] ,

[0127] wherein, Dice loss is more robust to the class imbalance of shadow regions (usually only 10-30% of the image).

[0128] To suppress false detections in non-shadow regions, the present application introduces a regularization loss that encourages the model to output lower confidence in non-shadow regions:

[0129] ,

[0130] where, denotes the set of non-shadow regions at the th scale (defined as ), is the target score at position .

[0131] The loss weights are determined by grid search: , These weights avoid performance degradation caused by over-regularization while maintaining shadow detection accuracy.

[0132] Step S5: Based on the final shadow detection result of the binocular image, the shadow part of the target region is lightened.

[0133] The quantitative relationship between the lightening position, light intensity and detection performance of the existing lightening method is not clear, and the quantitative relationship between the lightening position, light intensity and detection performance is not clear, which makes the optimization of the lightening scheme lack of scientific basis. Specifically; the fixed lightening at the hardware level cannot be personalized for different targets, and the traditional PID method in adaptive lightening control is also difficult to deal with the nonlinear and time-varying light-image response relationship. Further focusing on the intelligent lightening research itself, its shortcomings mainly manifest in four aspects: first, there is a lack of quantitative optimization target for the detection task, and the lightening effect evaluation lacks accurate standards; second, the lightening and detection processes are mutually separated, and the closed-loop mechanism of "lightening-detection-feedback-optimization" has not been formed; third, the spatial flexibility of the lightening scheme is insufficient, making it difficult to adapt to the diverse distribution of targets in complex agricultural scenes; fourth, the control strategy is relatively single, and cannot cope with the dynamic changes of the light environment and detection needs. At the same time, the existing scheme as a whole relies on passive sensing mode, lacks active and intelligent light regulation mechanism, which also becomes the main obstacle for the performance improvement of the current agricultural visual detection system under complex light environment. Therefore, as shown in Figure 3 , the present application designs a complete adaptive intelligent lightening system, constructs an active lightening system based on a double-closed-loop control architecture, and realizes the whole-process closed-loop control from perception to execution. The system adopts double fuzzy PID closed-loop control to realize accurate and adaptive lightening of the shadow region, specifically as follows:

[0134] The adaptive intelligent lighting compensation system contains two parallel closed-loop control circuits:

[0135] (1) The mechanical arm position control circuit, through the fusion network target detection and three-dimensional reconstruction, adjusts the PID parameters through fuzzy reasoning, drives the mechanical arm to move the light source to the optimal lighting position, and the position feedback is provided by the encoder;

[0136] (2) The light source brightness control circuit, through ROI extraction and Lab color space brightness evaluation, generates target brightness, adjusts the LED light intensity through fuzzy PID control, and the brightness feedback is obtained by calculating the image collected by the camera.

[0137] The two circuits work together to realize accurate and adaptive compensation of the shadow area. Figure 3 The complete signal flow, control parameters and physical entities of the actuator are shown.

[0138] The primary problem of light compensation is to determine where the light source should be placed and how much the initial light intensity should be set. The invention designs a lighting strategy planner based on the illumination geometric model and the inverse square law.

[0139] Step S51: Based on the final shadow detection result of the binocular image, the three-dimensional position of the target object center point corresponding to the shadow part in the mechanical arm coordinate system is obtained through stereo vision reconstruction as the three-dimensional position of the compensation point; although monocular vision is needed to identify the shadow, the shadow image area obtained is not enough for positioning. The invention uses binocular images, mainly for spatial positioning, and uses the method of binocular vision positioning to obtain the coordinates of the center point of the target object corresponding to the shadow part.

[0140] Step S52: Set the light source target to be located above the target object corresponding to the shadow part, based on the three-dimensional position of the compensation point and the initial lighting distance of the light source, obtain the target position of the light source, the formula is:

[0141] ,

[0142] wherein, is the target position of the light source, is the initial lighting distance of the light source, in this embodiment, is 150mm, is a unit vector vertically upward.

[0143] Due to the constraints of the mechanical arm base position and the working space, the light source actually irradiates the target area in a diagonal manner, effectively eliminating the surrounding shadows. When the calculated target position exceeds the working space range, a projection strategy is adopted to map it to the reachable boundary.

[0144] Step S53: Based on the shadow part brightness, target brightness, calibration coefficient, light source initial illumination distance, the initial light intensity (PWM duty ratio) is obtained through the illumination model as:

[0145] ,

[0146] wherein, is the initial light intensity (PWM duty ratio, range [30, 255]), is the current shadow area brightness, is the target brightness, is the calibration coefficient (set to 5000).

[0147] Based on the human eye perception characteristics, the present application sets the target brightness , and the brightness entering this range during control is considered to meet the standard, and the strategy output by the planner provides a clear target for the subsequent control layer.

[0148] The core task of the double closed-loop fuzzy PID controller control layer is to convert the target given by the planning layer into actual execution actions. In this paper, a double fuzzy PID closed-loop control architecture is adopted to realize mechanical arm position control and light source brightness control. Compared with the traditional fixed gain PID controller, fuzzy PID can dynamically adjust the control parameters according to the current error state, so as to better cope with the nonlinear and time-varying characteristics of the system. The specific scheme is as follows:

[0149] Step S54: Based on the light source target position and the initial position of the mechanical arm end effector, the mechanical arm end effector is driven from the initial position to the light source target position through the mechanical arm position controller, including:

[0150] Step S541: Set the control period, and every time a control period passes, based on the position error of the current control period, the position error change rate of the current control period and the last control period, the PID parameters of the mechanical arm controller of the next control period are obtained through the fuzzy reasoning mechanism; wherein the position error of the current control period is the L1 norm (the sum of the absolute values of each axis error) of the light source target position and the end effector position of the current control period, and the formula is: , is the position error of the th control period, is the light source target position, is the end effector position of the th control period;

[0151] Step S542: Based on the PID parameters of the mechanical arm controller of the next control period, the mechanical arm end effector is moved through the mechanical arm position controller until the mechanical arm end effector is driven from the initial position to the light source target position.

[0152] To solve the problem that fixed gain PID is difficult to balance response speed and stability, the application introduces fuzzy reasoning mechanism, and according to error and error change rate Self-adaptive adjustment PID parameters:

[0153] ,

[0154] Wherein, respectively represent the proportional gain, integral gain and differential gain of the manipulator controller at the time, is the basic gain of the manipulator controller, is the gain adjustment amount of fuzzy reasoning output, is the position error change rate.

[0155] The fuzzy reasoning mechanism adopts Mamdani type reasoning mechanism, and defines seven fuzzy sets:

[0156] ,

[0157] Wherein, NB (Negative Big) represents negative big, NM (Negative Medium) represents negative medium, NS (Negative Small) represents negative small, ZO (Zero) represents zero, PS (Positive Small) represents positive small, PM (Positive Medium) represents positive medium, and PB (Positive Big) represents positive big, which are described by triangular membership functions.

[0158] The error quantization function maps the physical quantity to the domain [-3,3], wherein the X and Y axes of the manipulator, .

[0159] The fuzzy rule base contains 49 rules. The rule design follows the classical principle: increase the proportional gain to speed up the response when the error is large, and reduce the gain to avoid overshoot when approaching the target.

[0160] As shown in Table 1, Table 1 shows the complete rule base of .

[0161] Table 1

[0162]

[0163] Defuzzification adopts the barycenter method:

[0164] ,

[0165] Wherein, This indicates the gain adjustment amount of the robotic arm controller. (corresponding to proportional, integral, and differential respectively) For the first The activation level of the rule, For the first The output membership center of each rule The number of rules that are activated.

[0166] and Using a similar rule base, the robotic arm employs a decoupled control strategy for its three axes X, Y, and Z, with each axis running independently under a fuzzy PID controller.

[0167] The control law for the robotic arm is:

[0168] ,

[0169] in, This indicates the control output of the robotic arm.

[0170] In this embodiment, the control cycle duration is set. The process involves iterative updates; the convergence condition for the robotic arm end effector to drive from its initial position to the target position of the light source is: That is, when At that time, it is determined that the end effector of the robotic arm has been driven from the initial position to the target position of the light source.

[0171] Step S55: Based on the target brightness and initial light intensity, the PWM duty cycle of the light source carried by the robotic arm is controlled by the light source brightness controller until the brightness of the shadow area of ​​the target area enters the target range, so as to realize the illumination of the shadow area of ​​the target area, including:

[0172] Step S551: Based on the brightness error of the current control cycle and the rate of change of brightness error between the current control cycle and the previous control cycle, obtain the PID parameters of the light source brightness controller for the next control cycle through a fuzzy inference mechanism; wherein, the brightness error of the current control cycle is the difference between the target brightness and the brightness of the shadow area in the current control cycle.

[0173] The light source brightness controller adjusts the PWM duty cycle of the LEDs to bring the shadow brightness into the target range. To facilitate PID error calculation, the midpoint of the range is used as a reference value, and the brightness error is defined as:

[0174] ,

[0175] in, For the first Brightness error per control cycle For target brightness, The luminance of the shadow part for the first control cycle. The luminance of the shadow part for the first control cycle.

[0176] Step S552: Obtain the light source control output of the current control cycle through the light source control law.

[0177] Step S553: Based on the light source control output of the current control cycle, obtain the light intensity increment of the light source of the next control cycle until the luminance of the shadow part of the target region enters the target range. The convergence condition is that the luminance enters the target range, and the adjustment is stopped. The luminance is extracted through the L channel of the Lab color space (range [0, 100], linearly corresponding to human eye perception).

[0178] The light source luminance controller has a bidirectional adjustment capability: when the luminance of the shadow part is lower than the target range, the light intensity is increased to make the luminance rise; when the luminance of the shadow part is higher than the target range (such as environmental light change or distance adjustment leading to over-brightness), the light intensity is reduced to make the luminance drop. This bidirectional control ensures that the luminance is always stable in the target range, and adapts to different initial conditions and environmental disturbances.

[0179] Similar to the mechanical arm controller, the light source controller also uses fuzzy PID, and the PID parameters are adaptively adjusted according to the luminance error And the luminance error change rate

[0180] ,

[0181] Wherein, respectively represent the proportional gain, integral gain, and differential gain of the light source controller at the time, is the basic gain of the light source controller, is the gain adjustment amount of the fuzzy reasoning output, is the luminance error change rate.

[0182] As shown in Table 2, Table 2 shows the fuzzy rule base of the light source control.

[0183] Table 2

[0184]

[0185] The light intensity is updated gradually:

[0186] ,

[0187] Wherein, represents the light intensity (PWM duty cycle) at the time, is the light intensity increment, ​​For the step factor (set to 1.5), and introduce dead zone threshold Avoid frequent adjustment:

[0188] ,

[0189] Wherein, The light source control law output, The dead zone threshold.

[0190] The light source control law is:

[0191] ,

[0192] Wherein The light source control output.

[0193] The complete control flow of the application is divided into four stages:

[0194] First, the detection and positioning stage, SGFD-Net predicts the shadow and detects the target, stereo matching calculates the depth, coordinate transformation to the mechanical arm coordinate system, and extracts the initial brightness.

[0195] Second, the strategy planning stage, calculate the lighting direction, light source target position and initial light intensity, if it exceeds the working space, then project correction.

[0196] Then the dual fuzzy PID control stage, as the core control stage, this stage is divided into three sub-stages: first, the mechanical arm positioning, using three-axis fuzzy PID control, with a control cycle length of Iterate to ; Then the distance optimization, from Start, shorten 20mm each time, until the brightness meets the standard or reaches the minimum distance 100mm; Finally, the brightness control, using light source fuzzy PID control, with a cycle length of Iterate to . Finally, the data saving stage, turn off the light source and save the control data.

[0197] In the system initialization phase, the binocular camera acquisition system is started sequentially, a serial communication connection to the robotic arm is established, the pre-trained shadow-guided fusion detection network is loaded into the GPU memory, and the parameters of the dual-layer fuzzy PID controller are initialized. The second phase is detection and localization. The system acquires binocular images and inputs them into the fusion network for forward inference. Simultaneously, stereo vision algorithms are used to calculate the three-dimensional spatial coordinates of the fruit and to evaluate the brightness of the shadow area. The third phase is strategy planning. Based on the detection results, the optimal lighting position is calculated, the initial light intensity setpoint is estimated, and the robotic arm's motion trajectory is planned. The fourth phase is dual fuzzy PID control. First, the robotic arm is moved to the vicinity of the target position. Then, the distance from the light source to the target is optimized through a position loop PID. Finally, the LED brightness is adjusted to the desired value through an illuminance fuzzy PID. The final phase is result evaluation and storage. The system captures images after supplemental lighting and saves data such as position, speed, and light intensity during the control process. The method of this invention can significantly improve detection performance. The shadow-guided fusion detection network greatly improves the recall rate in shadow scenes, and active supplemental lighting significantly increases the detection rate. Furthermore, the system can maintain stable performance in complex lighting environments, extending the operating time and significantly improving equipment utilization. Furthermore, the method of this invention has good versatility and can be applied to other agricultural vision tasks such as disease diagnosis and growth monitoring. Finally, multi-module collaborative control provides a technical reference for the intelligentization of agricultural robots and promotes the development of collaborative control technology.

[0198] This invention proposes an innovative algorithm that integrates shadow detection and target detection, as well as an innovative active supplementary lighting method that combines visual perception with a robotic arm. It collaboratively addresses lighting issues from two levels: AI-based low-light target recognition and intelligent supplementary lighting control, aiming to significantly improve the performance and robustness of agricultural visual inspection systems. By treating shadows as a valuable source of spatial prior information, the feature representation capability is enhanced through explicit modeling of the shadow detection task. A shadow-guided gating fusion mechanism is proposed, dynamically adjusting feature weights based on whether the spatial location is within a shadow area to achieve adaptive enhancement. Furthermore, an active supplementary lighting closed-loop control system is constructed, using the robotic arm as the light source carrier, and employing dual fuzzy PID control to flexibly adjust the position and intensity of the supplementary lighting.

[0199] Based on Embodiment 1, this Embodiment 2 verifies the method of the present invention through experiments, as detailed below:

[0200] The experimental setup was meticulously designed to meet the system's requirements for visual perception and mechanical execution. A binocular camera was mounted above the workbench to ensure complete coverage of the work area. The robotic arm base was installed at the front edge of the workbench, with its workspace fully overlapping the camera's field of view.

[0201] like Figure 4 As shown, Figure 4 It demonstrates a qualitative comparison of different models in shadow scenes.

[0202] The performance of the control system directly affects the supplementary lighting effect and the overall system efficiency. This embodiment evaluates the system based on the positioning accuracy of the robotic arm and the control of the light source brightness.

[0203] (1) Positioning accuracy of robotic arm

[0204] The positioning accuracy of the robotic arm determines whether the light source can accurately move to the target position. This was demonstrated through 100 repeated positioning experiments at random target points. The position error variation curve is shown below. Figure 5 As shown, the fuzzy PID speed output curve is as follows: Figure 6 As shown.

[0205] (2) Light source brightness control accuracy

[0206] The goal of light source brightness control is to adjust the Lab-L* value of the shadow area to the target range [45, 60]. Figure 7 The typical brightness increase control process is shown: the initial brightness is about 27 (far below the target range), the system gradually increases the light intensity through fuzzy PID control, and the brightness rises to 47 after about 13 seconds and enters the target range [45, 60]. Figure 8 The corresponding light source intensity adjustment curve is shown, with red dots marking key adjustment moments. The light intensity gradually increases from the initial 127 to about 250, and the adjustment process is smooth and without oscillation.

[0207] like Figure 9 As shown, the initial brightness was approximately 42 (higher than the upper limit of the target range). The system automatically reduced the light intensity, and the brightness dropped to 38 within about 4 seconds and stabilized within the target range. This verifies the controller's bidirectional adjustment capability, effectively handling situations with excessively high brightness. It is worth noting that the system's target range can be flexibly set according to different application scenarios. Figure 9 This demonstrates the control effect when the target range is set to [25,40] (green area). Figure 10 The corresponding light source intensity adjustment curves were displayed, verifying the bidirectional control capability and the configurability of the target range.

[0208] Based on the above experiments, it can be demonstrated that the overall performance of the system in this application can fully meet the practical application requirements of intelligent supplementary lighting in shadow scenes. In terms of system collaboration, the robotic arm positioning system and the light source brightness control system work together efficiently and accurately, forming a closed-loop control system that not only ensures the spatial accuracy of the supplementary lighting action but also achieves dynamic adaptation of the supplementary lighting intensity. The overall performance is stable and controllable, and it has significant technical advantages over traditional supplementary lighting solutions in terms of supplementary lighting accuracy, response speed, and scene adaptability.

[0209] This third embodiment provides an illumination supplement system for agricultural inspection, including:

[0210] The feature extraction module is configured to acquire binocular images of the target region, and generate a shadow mask of each of the binocular images and a feature pyramid of different scales of each of the binocular images.

[0211] The downsampling module is configured to perform a downsampling operation on the shadow mask of each of the binocular images, so that the spatial resolution of each of the downsampled shadow masks is consistent with the spatial resolution of the feature pyramid of each of the scales of the binocular image, and obtain a downsampled shadow mask corresponding to each of the scales of the binocular image.

[0212] The gated feature acquisition module is configured to obtain gated features corresponding to each of the scales of each of the binocular images by multiplication gating based on the feature pyramid of each of the scales of each of the binocular images, the downsampled shadow mask corresponding to each of the scales, a minimum gating value, and an exponential factor.

[0213] The shadow detection module is configured to obtain a prediction result corresponding to each of the scales of each of the binocular images by using a detection head based on the gated features corresponding to each of the scales of each of the binocular images, and perform non-maximum suppression processing on the prediction result of each of the binocular images to obtain a final shadow detection result of each of the binocular images.

[0214] The light compensation module is configured to perform illumination compensation on the shadow part of the target region based on the final shadow detection result of the binocular images.

[0215] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0216] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The device that implements the functions specified in one or more flows or blocks.

[0217] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0218] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0219] Obviously, the above-described embodiments are only examples and are not intended to limit the present application. Based on the above description, one of ordinary skill in the art can make other variations and changes without departing from the present application. It is not necessary or possible to enumerate all the embodiments. The obvious variations and changes derived therefrom are still within the scope of the present application.

Claims

1. An illumination method for agricultural detection, characterized in that, The method comprises the following steps: obtaining binocular images of a target region, and generating a shadow mask of each image in the binocular images and a feature pyramid of different scales respectively; performing down-sampling operation on the shadow mask of each image, so that the spatial resolution of each down-sampled shadow mask is consistent with the spatial resolution of the feature pyramid of each scale of the image, and a down-sampled shadow mask corresponding to each scale of the image is obtained; based on the feature pyramid of each scale of each image, the down-sampled shadow mask corresponding to each scale, a minimum gating value and an exponential factor, a gated feature corresponding to each scale of each image is obtained through multiplication gating; based on the gated feature corresponding to each scale of each image, a prediction result corresponding to each scale of each image is obtained by using a detection head, and the prediction result of each image is processed through non-maximum suppression to obtain a final shadow detection result of each image; the prediction result comprises original confidence logits, and the original confidence logits corresponding to each scale of each image are optimized, which comprises: based on a pre-set confidence modulation index, the original confidence logits corresponding to each scale of each image and the down-sampled shadow mask, a target confidence logits corresponding to each scale of each image is obtained through a gating mechanism; wherein the pre-set confidence modulation index is greater than the exponential factor; based on the final shadow detection result of the binocular images, a shadow part of the target region is illuminated, which comprises: based on the final shadow detection result of the binocular images, a three-dimensional position of a target object center point corresponding to the shadow part in a mechanical arm coordinate system is obtained as a light compensation point three-dimensional position; a light source target is set to be located above the target object corresponding to the shadow part, and based on the light compensation point three-dimensional position and an initial illumination distance of the light source, a light source target position is obtained; based on the brightness of the shadow part, a target brightness, a calibration coefficient and the initial illumination distance of the light source, an initial light intensity is obtained through an illumination model; based on the light source target position and an initial position of a mechanical arm end effector, the mechanical arm end effector is driven from the initial position to the light source target position through a mechanical arm position controller; based on the target brightness and the initial light intensity, a PWM duty cycle of the light source carried by the mechanical arm is controlled through a light source brightness controller until the brightness of the shadow part of the target region enters a target range, so that the shadow part of the target region is illuminated.

2. The method for illumination compensation for agricultural detection according to claim 1, wherein, The method comprises the following steps: based on the feature pyramid of each scale of each image, the down-sampled shadow mask corresponding to each scale, the minimum gating value and the exponential factor, a gated feature corresponding to each scale of each image is obtained through multiplication gating; based on the down-sampled shadow mask corresponding to each scale of each image, the minimum gating value and the exponential factor, a gating function corresponding to each scale of each image is obtained; 3. The method for illumination compensation for agricultural detection according to claim 2, wherein, the gating function corresponding to each scale of each image is multiplied with the feature pyramid of the scale element by element to obtain the gated feature corresponding to each scale of each image. The formula for obtaining the gating function corresponding to each scale of each image based on the down-sampled shadow mask corresponding to each scale of each image, the minimum gating value and the exponential factor is ， wherein, is a gating function corresponding to the th scale, is a scale index of the feature pyramid, is a minimum gating value, is an exponential factor, is a down-sampled shadow mask corresponding to the th scale.

4. The method for illumination compensation for agricultural detection according to claim 1, wherein, The original confidence logits corresponding to each scale of each image are obtained based on the preset confidence modulation index, the original confidence logits corresponding to each scale of each image, and the down-sampled shadow mask through a gating mechanism, and the formula is: ， in, For the first The target confidence level logits corresponding to each scale For the first The original confidence scores logits corresponding to each scale. To preset the confidence modulation index, For the first The shadow mask after downsampling corresponding to each scale.

5. The method for illumination compensation for agricultural detection according to claim 1, wherein, The method for driving the end effector of the mechanical arm from the initial position to the target position of the light source based on the target position of the light source and the initial position of the end effector of the mechanical arm includes: A control period is set, and every time a control period elapses, the PID parameters of the mechanical arm controller in the next control period are obtained based on the position error in the current control period, the position error change rate between the current control period and the previous control period, and a fuzzy inference mechanism; wherein the position error in the current control period is the L1 norm of the difference between the target position of the light source and the position of the end effector in the current control period; The end effector of the mechanical arm is controlled to move based on the PID parameters of the mechanical arm controller in the next control period through the mechanical arm position controller until the end effector of the mechanical arm is driven from the initial position to the target position of the light source.

6. The method for illumination compensation for agricultural detection according to claim 5, wherein, The method for controlling the PWM duty cycle of the light source carried by the mechanical arm based on the target brightness and the initial light intensity through a light source brightness controller until the brightness of the shadow part of the target region enters the target range includes: The PID parameters of the light source brightness controller in the next control period are obtained based on the brightness error in the current control period, the brightness error change rate between the current control period and the previous control period, and a fuzzy inference mechanism; wherein the brightness error in the current control period is the difference between the target brightness and the brightness of the shadow part in the current control period; The light source control output in the current control period is obtained through a light source control law; The light intensity increment of the light source in the next control period is obtained based on the light source control output in the current control period until the brightness of the shadow part of the target region enters the target range.

7. The method for illumination compensation for agricultural detection according to claim 1, wherein, The method for respectively generating a shadow mask for each image in the binocular image and a feature pyramid of different scales includes: Each image in the binocular image is respectively input into a shadow detection branch, and the shadow mask of each image is output; Each image in the binocular image is respectively input into a target detection branch, and a feature pyramid of different scales for each image is output.

8. An illumination light supplement system for agricultural detection, characterized in that, It includes: A feature extraction module is configured to obtain a binocular image of a target region, and respectively generate a shadow mask for each image in the binocular image and a feature pyramid of different scales; A down-sampling module is configured to perform a down-sampling operation on the shadow mask of each image, so that the spatial resolution of each down-sampled shadow mask is consistent with the spatial resolution of the feature pyramid of each scale of the image, and obtain a down-sampled shadow mask corresponding to each scale of the image; A gated feature acquisition module is configured to obtain gated features corresponding to each scale of each image by multiplication gating based on the feature pyramid of each scale of each image, the down-sampled shadow mask corresponding to each scale, a minimum gating value, and an exponential factor. The shadow detection module is configured to obtain, by using the detection head, a prediction result corresponding to each scale of each image based on the gating feature corresponding to each scale of each image, and obtain a final shadow detection result of each image by performing a non-maximum suppression process on the prediction result of each image. The prediction result includes original confidence logits, and the original confidence logits corresponding to each scale of each image are optimized, including: Based on the pre-set confidence modulation index, the original confidence logits corresponding to each scale of each image, and the down-sampled shadow mask, a target confidence logits corresponding to each scale of each image is obtained through a gating mechanism, wherein the pre-set confidence modulation index is greater than an exponential factor. The light compensation module is configured to illuminate and compensate the shadow part of the target region based on the final shadow detection result of the binocular image, including: Based on the final shadow detection result of the binocular image, a three-dimensional position of a target object center point corresponding to the shadow part in a mechanical arm coordinate system is obtained as a compensation point three-dimensional position. The light source target is set to be located above the target object corresponding to the shadow part, and a light source target position is obtained based on the compensation point three-dimensional position and an initial illumination distance of the light source. Based on the brightness of the shadow part, the target brightness, a calibration coefficient, and the initial illumination distance of the light source, an initial light intensity is obtained through an illumination model. Based on the light source target position and an initial position of an end effector of the mechanical arm, the end effector of the mechanical arm is driven from the initial position to the light source target position through a mechanical arm position controller. Based on the target brightness and the initial light intensity, a PWM duty cycle of the light source carried by the mechanical arm is controlled through a light source brightness controller until the brightness of the shadow part of the target region enters a target range, so as to realize illumination and compensation of the shadow part of the target region.

Images