Camellia fruit recognition optimization system based on attention mechanism

Abstract

The application discloses a kind of based on attention mechanism's camellia fruit recognition optimization system, the present application relates to agricultural computer vision technology field, comprising: dynamic occlusion perception attention module, through space-time double-flow occlusion reasoning unit is connected to the input layer of cross-modal spectral feature collaborative fusion network.The based on attention mechanism's camellia fruit recognition optimization system, significantly improve the feature discrimination ability under the dense environment of branch and leaf, effectively enhance the target recognition robustness under low contrast scene, reduce the risk of mismatch caused by light mutation, accurately capture the small gradient change of young fruit, balance semantic and detail features, while embedded hardware deployment realizes efficient parallel computing, ensure real-time response under complex scene, iteration strengthens weak link of model, overall improve the stability and generalization ability of system, provide high-precision, low-latency camellia fruit recognition solution for intelligent agriculture.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural computer vision, in particular to a camellia fruit recognition optimization system based on attention mechanism. BACKGROUND

[0002] As an important economic crop in southern China, the intelligent monitoring and picking technology of camellia fruit is crucial to the development of the industry. However, the growth environment of camellia fruit is complex, with dense branches and leaves, variable light, such as backlight, shadow interlacing, and dynamic changes in fruit shape. The volume of young fruits is small, and the color of mature fruits is close to that of branches and leaves, which poses a serious challenge to traditional computer vision algorithms. The existing mainstream methods are designed based on RGB images, which have the following significant defects: low contrast scene recognition is insufficient: young fruits have similar colors to branches and leaves, and uneven lighting further weakens the target outline, making it difficult for traditional convolutional networks to extract distinguishing features, resulting in high false detection and missed detection rates; dense shading leads to feature confusion: under the interlaced shading of branches and leaves, static attention mechanisms such as SE modules cannot dynamically adjust the focus area and are easily disturbed by the background, resulting in a decrease in the discriminability of shading boundary features; small target detection precision-efficiency imbalance: young fruits account for only 0.1% to 1%, and the fixed feature pyramid strategy cannot balance semantic and detailed information, and the standard loss function ignores size differences, resulting in insufficient coordination and optimization of positioning and classification accuracy. Existing improvement schemes attempt to introduce multi-modal data, such as near-infrared imaging or attention mechanisms, but simply concatenate multi-spectral features, fail to exploit the complementary nature of cross-modal data, and lack dynamic response to shading. In addition, small target optimization methods, such as super-resolution reconstruction, increase computational complexity and are difficult to meet the real-time requirements of field work. There is an urgent need for a system that integrates dynamic multi-modal attention, adaptive loss distribution, and efficient cross-spectral fusion to break through the technical bottleneck of camellia fruit recognition in complex environments. SUMMARY

[0003] (I) Technical problems solved

[0004] To solve the problems of weak feature discriminability, inefficient cross-spectral fusion, and small target detection precision-speed imbalance under shading interference, the present application provides a camellia fruit recognition optimization system based on attention mechanism.

[0005] (II) Technical solutions

[0006] To achieve the above purposes, the present application is implemented by the following technical solutions: a camellia fruit recognition optimization system based on attention mechanism, comprising:

[0007] The dynamic occlusion perception attention module is connected to an input layer of a cross-modal spectral feature collaborative fusion network through a space-time double-flow occlusion reasoning unit, an output end of the cross-modal spectral feature collaborative fusion network is connected to a heterogeneous pyramid fusion layer of a small target adaptive optimization engine, and a positioning loss calculation unit of the small target adaptive optimization engine is feedback connected to a competitive loss function module of a collaborative verification framework.

[0008] The dynamic occlusion perception attention module comprises:

[0009] The spatial dimension processing unit: input C3-C5 level multi-scale feature maps, and generate an occlusion probability heat map through a deformable convolution layer, wherein a convolution kernel offset is dynamically calculated by a local gradient of the feature map.

[0010] The time dimension processing unit: based on a light-weight LSTM, fuse the current frame heat map and the historical frame optical flow feature, and output a 3-5 frame occlusion trend prediction in the future;

[0011] The cross-modal spectral feature collaborative fusion network comprises a spectral reflectivity matching verification module for screening effective spectral features through physical prior constraints, and further comprises a cross-attention gate mechanism of an RGB branch and a near-infrared branch, wherein:

[0012] The RGB branch adopts a MobileNetV3 inverted residual structure to extract spatial features;

[0013] The near-infrared branch adopts a depth separable convolution to extract spectral reflectivity features;

[0014] The micro-scale feature enhancement pyramid of the small target adaptive optimization engine inserts a hollow convolution layer with a hollow rate of 3 / 6 / 9 between the C3-C5 levels of the FPN, and dynamically allocates level weights through a feature reweighting module.

[0015] The collaborative verification framework connects each module through a shared bottom feature extractor, and adopts a two-stage adversarial verification process to iteratively optimize weak areas of model decision.

[0016] Preferably, the dynamic occlusion perception attention module has a divide-and-conquer attention strategy, including occlusion region division and differential processing:

[0017] The occlusion region division comprises: setting a high threshold of 0.7 and a low threshold of 0.3 by using the occlusion probability heat map, and dividing the feature map into an occlusion region, a transition region and a visible region;

[0018] The differential processing comprises:

[0019] a) The occlusion region adopts a channel attention module to suppress background channel response, and is connected to a spatial attention module, which adopts a hollow convolution with a hollow rate of 6 for local context aggregation;

[0020] b) Transition zone deploys local contrast enhancement module, which can learn the parameters of threshold function optimized by joint regional average gradient and standard deviation, and is parallel with direction gradient histogram feature extractor;

[0021] c) Visible zone superimposes non-local neural network to establish long-distance semantic association between fruits and branches and leaves;

[0022] The training phase of the dynamic occlusion perception attention module injects random branch and leaf masks with coverage of 10% to 60% through an adversarial occlusion simulator, and the inference phase adopts a double-threshold voting mechanism to retain the overlapping area predicted by 0.6 and 0.4 thresholds.

[0023] Preferably, the cross-modal spectral feature collaborative fusion network comprises:

[0024] Spectral guidance unit: the spectral features output by the near-infrared branch are compressed in channel dimension through 1×1 convolution to generate a Sigmoid spatial mask, which is multiplied with the feature maps of the corresponding level of the RGB branch channel by channel;

[0025] Spatial constraint unit: the semantic features of the RGB branch generate a spatial weight matrix through 3×3 convolution to regionally weight the deep features of the near-infrared branch;

[0026] Dynamic router: according to the weighted score of target confidence and Sobel gradient amplitude, the formula is: Score = 0.7×confidence + 0.3×Sobel gradient amplitude, to select the bottom layer super-resolution fusion path or the high layer joint up-sampling path;

[0027] In the heterogeneous pyramid fusion layer:

[0028] High-level features, i.e. C5 level, are bilinearly interpolated and up-sampled after aligning the spectral-spatial features through deformable convolution;

[0029] Bottom-level features, i.e. C3 level, fuse low-resolution NIR and high-resolution RGB features through sub-pixel convolution;

[0030] During training, single-modal input is shielded with a probability of 50%, and the spectral reflectance matching verification module is used to constrain the near-infrared features within the range of 0.65-0.82, wherein the reflectance of young fruits is 0.65-0.75, the reflectance of mature fruits is 0.70-0.82, and the reflectance of fruits in the swelling stage is the union of the two ranges.

[0031] Preferably, the micro-scale feature enhancement pyramid of the small target adaptive optimization engine comprises:

[0032] Cave convolution layer configuration: C3 layer with a cavity rate of 3, C4 layer with a cavity rate of 6, and C5 layer with a cavity rate of 9;

[0033] Feature reweighting module: generate a normalized weight vector according to the target relative area input into the full connection layer, and the calculation formula is:

[0034]

[0035] Where s i is the average confidence of the i-th layer small target, g i is the Sobel gradient amplitude, and II is an indicator function, which is 1 when g i <0.4, otherwise 0.

[0036] Gaussian-exponential hybrid loss function: classification loss weight formula:

[0037] L cls =-(1+e -D )ylog(p);

[0038] D = a (1-Area) + b OR + g (1-Contrast);

[0039] The positioning loss adopts a center point Gaussian weighting, σ is inversely proportional to the target size.

[0040] Preferably, the spectral reflectance matching verification module comprises:

[0041] Differentiable constraint unit: perform segmented linear activation on the 61-channel feature map (corresponding to the 650-950nm waveband) output by the near-infrared branch, and when the channel value is within the measured threshold [0.65, 0.82] of the waveband, the original gradient is retained, and the gradient of the exceeding part is zeroed.

[0042] Penalty loss term formula:

[0043]

[0044] Inference verification mechanism: 16x16 sliding window sampling is performed on the fused features, and when the effective reflectivity channel proportion of the continuous 3 windows is <60%, the C3 layer super-resolution reconstruction compensation of the RGB branch is triggered.

[0045] Wherein, the spectral reflectance matching verification module is based on the ASD FieldSpec4 spectrometer to build a database.

[0046] Wherein, the calculation method of the effective reflectivity channel proportion is:

[0047]

[0048] Preferably, the sharing-competition mechanism of the collaborative verification framework comprises:

[0049] Parameter freezing strategy: the bottom parameters of MobileNetV3 and depthwise separable convolution are fixed in the first 3 training stages;

[0050] Dynamic weight competitive loss formula:

[0051] L total = αL doaa + βL cmsf + γL stae ;

[0052] The weight adjustment rule is:

[0053] When Δ = (R doaa - R cmsf ) / R all > 0.05, α <- α × 0.9, β <- β × 1.1;

[0054] When AP stae < 80%, γ <- γ + 0.05·(80% - AP stae );

[0055] Gradient norm constraint:

[0056] If , a 0.7 decay factor is applied to L doaa .

[0057] Preferably, the two-stage adversarial verification process comprises:

[0058] Extreme scenario simulation set generation:

[0059] a) Occlusion synthesis: generate dynamic occlusion patches based on CycleGAN, coverage 60%~80%;

[0060] b) Light simulation: combine the Phong light model to generate mixed images of overexposure (brightness > 240) and low illumination (brightness < 50);

[0061] c) Dense small target: generate 50-100 young fruits with an area <0.5% per image through super-resolution reconstruction;

[0062] Error feedback mechanism: use Grad-CAM to locate feature confusion areas, and through curriculum learning, strengthen training in three stages:

[0063] The first stage freezes the CMSF and STAE parameters, focusing on DOAA optimization;

[0064] The second stage unfreezes the CMSF and sets the cross-modal similarity threshold to 0.7;

[0065] The third stage optimizes the STAE loss weight and feature pyramid parameters jointly;

[0066] Wherein, the positioning loss gradient is back-propagated to the error feedback mechanism of the adversarial verification process, and the optimization weight of the curriculum learning stage is dynamically adjusted.

[0067] Preferably, the physical data driven method of the spectral reflectance matching verification module comprises:

[0068] Spectral database construction: ASD FieldSpec4 spectrometer is used to collect reflectance data in the 650-950 nm band at different growth stages, i.e. young fruit, swelling stage, and mature stage;

[0069] Back-propagation linkage optimization: when the near-infrared feature exceeds the threshold [0.68, 0.79] at 780 nm channel, the spatial attention weight promotion coefficient of the RGB branch corresponding position is triggered 1.3 times;

[0070] Real-time exception handling process: after freezing the near-infrared branch parameters, the C3 layer hollow convolution feature of the RGB branch is enabled for 3 times of sub-pixel convolution reconstruction until the reflectance effective channel ratio is ≥60%.

[0071] Preferably, the hardware deployment of the attention mechanism based camellia fruit recognition optimization system comprises:

[0072] Image acquisition unit: IMX477 RGB sensor and SWIR-640 near-infrared camera are synchronously triggered for acquisition;

[0073] Processing unit: Jetson Xavier NX embedded platform, configured with a multi-threaded task scheduler:

[0074] a) The first thread runs the dynamic occlusion perception attention module, allocating 2 CPU cores;

[0075] b) The second thread runs the cross-modal fusion network, allocating 128 CUDA cores;

[0076] c) The third thread runs the small target optimization engine, allocating 64 CUDA cores;

[0077] Output interface: the recognition result is transmitted to the agricultural machinery controller through CAN bus at a frequency of 100 Hz, with a response delay of <200 ms.

[0078] Preferably, the system in the orchard monitoring scene comprises:

[0079] Dynamic acquisition unit of unmanned aerial vehicle: quadcopter unmanned aerial vehicle equipped with the image acquisition unit performs serpentine path cruising at a constant height, realizes centimeter level track tracking through RTK positioning module, and dynamically adapts the image acquisition frequency to the flight speed;

[0080] Real-time processing-feedback control loop:

[0081] a) The processing unit completes single-frame camellia fruit recognition within 150 ms, and transmits the positioning coordinates to the agricultural machinery through the LoRa wireless module;

[0082] b) When the mechanical arm controller generates a picking path based on the fruit coordinates, it synchronously receives the future 3-frame occlusion trend predicted by the dynamic occlusion perception module, and dynamically adjusts the end effector motion trajectory;

[0083] Spectrum-space joint verification mechanism: when the spectral reflectance matching verification module detects continuous 5-frame reflectance abnormalities, it triggers the unmanned aerial vehicle to hover and starts multi-angle compensation shooting, including:

[0084] Pitch angle ±15° deflection shooting 3 groups of images;

[0085] Turn on the near-infrared fill light to capture features again;

[0086] Input the multi-view features into the collaborative verification framework for confidence voting, and only adopt the recognition results consistent with the two sets of models.

[0087] (Three) beneficial effects

[0088] The application provides an oil tea fruit recognition optimization system based on an attention mechanism. It has the following beneficial effects:

[0089] The oil tea fruit recognition optimization system based on the attention mechanism, through the dynamic divide-and-conquer attention mechanism, strengthens the fruit edge features in the occlusion area division and differential processing, combines the spatiotemporal prediction model to capture the dynamic changes of occlusion, and significantly improves the feature discrimination ability in the dense environment of branches and leaves. The cross-modal spectral fusion network adopts cross-attention gating and physical prior constraint, deeply fuses RGB texture and near-infrared reflectance features, effectively enhances the target recognition robustness in low-contrast scenes, and reduces the risk of mismatch caused by sudden changes in light. In view of the small target detection problem, the system enhances the multi-scale receptive field and dynamically allocates the loss weight, accurately captures the tiny gradient changes of young fruits, balances the semantic and detail features, and simultaneously realizes efficient parallel computing through embedded hardware deployment, ensuring real-time response in complex scenes. The collaborative verification framework combines adversarial optimization and module competition mechanism, iteratively strengthens the weak links of the model, and improves the stability and generalization ability of the system as a whole, providing a high-precision, low-latency oil tea fruit recognition solution for smart agriculture. BRIEF DESCRIPTION OF DRAWINGS

[0090] Figure 1 It is the overall framework diagram of the application;

[0091] Figure 2 It is the control logic timing diagram of the application. Detailed Implementation

[0092] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0093] Please see Figure 1 and Figure 2 This invention provides a technical solution: an attention-based camellia fruit recognition optimization system, comprising:

[0094] The dynamic occlusion perception attention module is connected to the input layer of the cross-modal spectral feature collaborative fusion network through the spatiotemporal dual-stream occlusion inference unit. The output of the cross-modal spectral feature collaborative fusion network is connected to the heterogeneous pyramid fusion layer of the small target adaptive optimization engine. The localization loss calculation unit of the small target adaptive optimization engine is fed back to the competitive loss function module of the collaborative verification framework.

[0095] The dynamic occlusion awareness attention module includes:

[0096] Spatial dimension processing unit: Input multi-scale feature maps of C3-C5 levels, generate occlusion probability heatmaps through deformable convolutional layers, where the convolutional kernel offset is dynamically calculated by the local gradient of the feature map;

[0097] Temporal processing unit: Based on lightweight LSTM, it fuses the current frame heatmap and historical frame optical flow features to output occlusion trend prediction for the next 3-5 frames;

[0098] The cross-modal spectral feature collaborative fusion network includes a spectral reflectance matching verification module for screening effective spectral features through physical prior constraints, and a cross-attention gating mechanism between the RGB and near-infrared branches, wherein:

[0099] The RGB branch uses the inverse residual structure of MobileNetV3 to extract spatial features;

[0100] The near-infrared branch uses depthwise separable convolution to extract spectral reflectance features;

[0101] The Microscale Feature Enhancement Pyramid (MSE-FPN) of the Small Target Adaptive Optimization Engine inserts dilated convolutional layers with dilation rates of 3 / 6 / 9 between the C3-C5 layers of the FPN, and dynamically allocates layer weights through the feature reweighting module.

[0102] The synergistic verification framework connects each module by sharing the underlying feature extractor, and adopts a two-stage adversarial verification process to iteratively optimize weak decision areas of the model.

[0103] It should be further explained that, in the specific implementation process, the dynamic occlusion perception attention module has a divide-and-conquer attention strategy, including occlusion area division and differential processing:

[0104] The occlusion area division includes: inputting the C3-C5 layer multi-scale feature map into the deformable convolution layer, calculating the convolution kernel offset through local gradient, and generating an occlusion probability heat map.

[0105] Set double thresholds, i.e. high threshold 0.7, low threshold 0.3, divide the heat map into: occlusion area greater than or equal to 0.7, i.e. complex background, need to strengthen the fruit edge features; transition area, i.e. 0.3≤probability<0.7, i.e. the occlusion boundary is fuzzy, and the contrast needs to be enhanced; visible area, i.e. less than 0.3, i.e. simple background, need to associate global semantics;

[0106] Differential processing includes:

[0107] a) For the occlusion area, a channel attention module, i.e. SE module, is used to suppress the background channel response, such as reducing the branch texture channel response by 30%-50%, and then connecting a spatial attention module, which uses a spatial attention module with a hollow rate of 6 to perform local context aggregation and extract residual fruit edges;

[0108] b) For the transition area, deploy a local contrast enhancement module, which can learn the parameters of the threshold function jointly optimized by the area average gradient and the standard deviation, and is connected in parallel with the direction gradient histogram (HOG) feature extractor; wherein, the optimization algorithm formula of the learnable threshold function is:

[0109] Where k is a learnable parameter, is the area average gradient, σ is the standard deviation, and μ is the mean;

[0110] c) The visible area superimposes a non-local neural network (Non-local Network) to establish long-distance semantic association between fruits and branches;

[0111] The training phase of the dynamic occlusion perception attention module injects a random branch mask with a coverage rate of 10%-60% through the adversarial occlusion simulator, and the inference phase uses a double-threshold voting mechanism to retain the overlapping area predicted by the 0.6 and 0.4 thresholds.

[0112] It needs to be further explained that in the dynamic occlusion perception attention module, first, a multi-dimensional occlusion prediction mechanism is constructed through the space-time double-flow occlusion reasoning unit. In the spatial dimension, multi-scale feature maps of C3-C5 levels are used as input, and a deformable convolution layer is used to adaptively sample the occlusion boundary, wherein the convolution kernel offset is dynamically calculated by the local gradient of the feature map, so as to accurately extract the contour features of the fruits occluded by the leaves; at the same time, an occlusion probability heat map is generated through three parallel convolutions to quantify the occlusion possibility of each pixel region. In the time dimension, for video streams or continuous image sequences, a light-weight LSTM unit is introduced to fuse the spatial occlusion heat map of the current frame and the time sequence change features (motion vectors of adjacent frames extracted by the optical flow method) of the historical frames, predict the dynamic expansion or contraction trend of the occlusion region in the future 3-5 frames, and realize the space-time consistency of occlusion compensation. Based on the occlusion probability heat map, double thresholds (high threshold 0.7, low threshold 0.3) are set to divide the feature map into an occlusion area, a transition area and a visible area, and different attention mechanisms are applied respectively: in the occlusion area, channel-space double-dimensional attention is used, the background-related channel response is suppressed through the SE module, and the residual fruit edge features are enhanced by combining the local context aggregation of the hole convolution; in the transition area, a local contrast enhancement module is deployed, a learnable threshold function is used to perform nonlinear transformation on the feature map, and the edge response is enhanced combined with HOG prior information; in the visible area, a non-local network is superimposed to capture long-distance dependency and establish semantic association between fruits and surrounding leaves. In the training stage, the dynamic occlusion perception attention module is simulated by an adversarial occlusion, and a real leaf mask (coverage rate 10% to 60%) is injected. In the inference stage, a double-threshold voting mechanism is used to filter out misjudgment areas. This scheme improves the accuracy of occlusion area recognition and reduces the confusion rate of occlusion boundary features.

[0113] Through the space-time double-flow reasoning and divide-and-conquer strategy of the dynamic occlusion perception attention module, the system realizes accurate recognition in the orchard environment with severe leaf occlusion. Based on the double-threshold occlusion probability heat map, the feature map is divided into an occlusion area, a transition area and a visible area, and channel-space attention, local contrast enhancement and global semantic association strategies are used respectively to effectively solve the occlusion boundary feature confusion problem. Combined with LSTM time series prediction and adversarial occlusion simulation training, the system's adaptability to dynamic occlusion is significantly enhanced, the fruit recognition accuracy in the occlusion scene is improved, and the false detection rate is reduced.

[0114] It needs to be further explained that in the specific implementation process, the cross-modal spectral feature collaborative fusion network adopts a dual-branch heterogeneous feature extraction architecture: the RGB branch extracts fruit texture and shadow distribution features based on the inverted residual structure of MobileNetV3, and the near-infrared (NIR) branch extracts the reflectivity features of tea fruits in the near-infrared band (650-950 nm) through deep separable convolution. Bidirectional feature guidance is achieved through a cross-attention gating mechanism: the spectral features output by the near-infrared branch are compressed through 1x1 convolution to generate a Sigmoid spatial mask, which is multiplied with the feature maps of the corresponding level of the RGB branch channel by channel, dynamically enhancing low-contrast areas such as the silhouette of a backlit fruit. At the same time, the semantic features of the RGB branch generate a spatial weight matrix through 3x3 convolution, which weights the deep features of the near-infrared branch in the region, suppressing the redundant response of branches and leaves similar to the fruit. In the heterogeneous pyramid fusion layer, the high-level C5 layer is bilinearly interpolated and upsampled after aligning the spectral-spatial features through deformable convolution, and the low-level C3 layer fuses the low-resolution NIR and high-resolution RGB features through sub-pixel convolution. The dynamic routing algorithm selects the optimal fusion path according to the target confidence and the Sobel gradient amplitude, which improves the target contrast in strong backlight scenes and reduces the cross-modal mismatch rate.

[0115] The spatial mask generated by the near-infrared branch dynamically enhances the low-contrast areas in the RGB image, improving the target contrast in strong backlight scenes, and the spectral reflectivity matching verification module uses a differentiable threshold function and hard constraint feature fusion to suppress the spectral mismatch between branches and leaves and fruits, reducing the cross-modal misjudgment rate. Through the heterogeneous feature collaborative fusion mechanism of RGB and near-infrared spectra, the system maintains robustness in complex lighting conditions.

[0116] It needs to be further explained that in the specific implementation process, the small target adaptive optimization engine improves the detection capability of young fruits through the reconstruction of micro-scale feature enhancement pyramid (MSE-FPN). The empty convolution layers with empty rates of 3, 6, and 9 are inserted between the C3-C5 levels of the standard FPN, and the C3 layer empty convolution (receptive field 7x7) captures the small gradient changes of the young fruit edge, and the C5 layer (receptive field 19x19) correlates the global semantics of the occluded target. The feature reweighting module dynamically allocates level weights according to the target size, and the calculation formula is:

[0117]

[0118] where wi is the normalized weight coefficient of the i-th level feature map, which is used to dynamically allocate the contribution of different levels to small target detection; s i is the average confidence of the i-th small target, which is calculated by weighted summation of the foreground probability output by the classification head; the formula is:

[0119] where pk is the confidence of the kth bounding box, area k is the target relative image area, and II is an indicator function that takes value 1 when the condition g i <0.4, otherwise 0.

[0120] g i is the average Sobel gradient magnitude of the ith feature map, reflecting the edge sharpness of the feature, and the calculation formula is:

[0121]

[0122] ; where G x , G y are the gradient values of the Sobel operator in horizontal and vertical directions; II(g i <0.4): gradient constraint indicator function, taking value 1 when g i <0.4, otherwise 0, used to apply a decay factor of 0.8 to the weight of the fuzzy feature level.

[0123] In the Gaussian-exponential hybrid loss function: the classification loss is through the difficulty coefficient:

[0124] L cls =-(1+e -D )y log(p);

[0125] D = a(l-Area) + bOR + g(l-Contrast); the difficulty coefficient is exponentially weighted for difficult samples, and the positioning loss adopts a center point Gaussian weighting, the formula is:

[0126] Prioritize optimization of center offset, where D: sample difficulty coefficient, is a linear combination of the following parameters:

[0127] Area: target relative image area, such as young fruit Area <1%.

[0128] OR: Occlusion Rate, the calculation formula is: OR = number of occluded pixels / total number of target pixels.

[0129] Contrast: local contrast, calculated as the ratio of the gray scale standard deviation of the fruit area to the background area: Contrast = s obj / (s bg +e); e = le-5 is a small constant to prevent division by zero; a, b, g: weight coefficients, default values a = 0.6, b = 0.3, g = 0.1, determined by grid search optimization on the validation set.

[0130] w reg: Position loss weight, related to the offset of the predicted box center point: Δx, Δy: Euclidean distance between the predicted box center point and the true box (unit: pixels); σ: Gaussian kernel scale parameter, σ is inversely proportional to the target size: σ = k / √Area, K = 0.1 is an empirical constant, and Area is the relative area of the target; this scheme improves the young fruit detection IoU.

[0131] By expanding the receptive field through C3-C5 level hollow convolution and combining the feature re-weighting mechanism, the young fruit detection IoU is improved, and by optimizing the young fruit, medium fruit and mature fruit detection heads in stages and using the pseudo-label generated by knowledge distillation, the young fruit missed detection rate is reduced, so that for the young fruit detection problem of camellia oleifera, the small target adaptive optimization engine realizes breakthrough through multi-scale feature enhancement and dynamic loss distribution.

[0132] It should be further explained that in the specific implementation process, the spectral reflectance matching verification module ensures the rationality of cross-modal fusion through physical prior constraints. The 61-channel feature map output by the near-infrared branch (corresponding to the 650-950nm wavelength band, 5nm interval) is subjected to segmented linear activation: when the channel value is within the measured threshold [0.65, 0.82] of the wavelength band, the original gradient is retained, and the gradient of the exceeding part is set to zero and is included in the penalty loss term;

[0133] The penalty loss term formula is:

[0134]

[0135] Wherein, f c : The feature value of the cth channel of the near-infrared branch, corresponding to the wavelength 650+5(c-1)nm, when c=1, 650nm, and c=61, 950nm; λ: balance coefficient, default value λ=0.3, determined by cross-validation; 0.65-0.82: the measured reflectivity effective threshold of camellia oleifera in the 650-950nm wavelength band.

[0136] In the inference stage, a 16x16 sliding window is used to check the proportion of effective reflectivity channels, and if 3 consecutive windows are below 60%, the near-infrared branch parameters are frozen and the RGB branch C3 layer super-resolution reconstruction compensation is enabled. This mechanism applies a hard penalty to features that exceed the physical reflectivity range, ensuring that the spectral response meets the biological characteristics of the fruit, reduces the cross-modal mismatch rate, and maintains the spectral effectiveness under strong backlight.

[0137] It should be further explained that in the specific implementation process, the shared-competitive mechanism of the collaborative verification framework realizes parameter reuse of the modules by sharing the bottom feature extractor (MobileNetV3 and depth separable convolution), and the bottom parameters are frozen in the first 3 training stages to accelerate convergence.

[0138] Dynamic weight competitive loss function:

[0139] L ttotal αL aoaa +βL cmsf γL stae wherein, α, β, γ: dynamic weight coefficients, adjustment rules are as follows: occlusion module weight α: α new = α old × 0.9, if Δ = [(R doaa -R cmsf ) / R all ]> 0.05; modal module weight β: β new = β old × 1.1, if Δ > 0.05; small target module weight γ: γ new = γ old + 0.05 × (80% - AP stae ), linear compensation is triggered when AP stae < 80%; R doaa : recall rate of dynamic occlusion-aware module in occlusion scene, R cmsf : recall rate of cross-modal fusion module in uneven illumination scene, R all : overall recall rate of system; AP stae : average precision of small target optimization engine on young fruit test set; wherein, the test set contains 20,000 labeled images, covering 6 illumination conditions and 4 occlusion types, and the training-validation-test set is divided according to 6:2:2.

[0140] L total : total loss function, which is the overall loss function of the system, composed of the weighted sum of the loss functions of multiple sub-modules, used for global optimization of model parameters; through dynamic weights α, β, γ, the training objectives of different modules are balanced to avoid single module dominating the optimization direction, and the gradient is allocated in the back propagation to ensure collaborative optimization of each module and improve the overall performance of the system.

[0141] L doaa : dynamic occlusion-aware attention loss, which can improve the recognition accuracy of occlusion areas and optimize the boundary positioning ability; the special loss function of the dynamic occlusion-aware attention module includes the occlusion area classification loss and the boundary regression loss, the classification loss, i.e. the occlusion probability prediction formula:

[0142] wherein, y i is the occlusion label (0 / 1), and p i is the predicted probability;

[0143] Regression loss, i.e. occlusion boundary positioning formula:

[0144] Wherein, b i pred is the predicted bounding box, b i gt is the real bounding box;

[0145] Total loss: L doaa = L cls + 0.5L reg .

[0146] L cmsf : Cross-Modal Spectral Fusion Loss, used to ensure that the spectral features conform to the physical law and enhance the consistency of cross-modal features, the loss function of the cross-modal fusion network, including spectral matching loss and feature alignment loss, consisting of:

[0147] Spectral matching loss, i.e. reflectivity physical constraint formula:

[0148]

[0149] Feature alignment loss, i.e. cross-modal consistency formula:

[0150]

[0151] Wherein, f rgb and f nir are the same level feature maps of RGB and near-infrared branches.

[0152] Total loss:

[0153] L cmsf = L spec + 0.3L align ;

[0154] Lstae: Small target adaptive optimization engine loss, used to strengthen the classification confidence of small targets and improve the center point positioning accuracy; Small target detection dedicated loss function, including classification loss and positioning loss, focusing on optimizing small target detection; consisting of:

[0155] Classification loss, i.e. Gaussian-exponential weighted focal loss formula:

[0156]

[0157] Wherein, D i = α(1-Area i )+βOR i+ γ (1 - Contrast i ) is the sample difficulty coefficient.

[0158] Positioning loss, i.e. the center point Gaussian weighted loss formula:

[0159]

[0160] wherein,

[0161] Total loss: L stae = L cls + 0.7L reg .

[0162] The weight adjustment is driven by the performance difference of the modules, which suppresses the single module dominant tendency. The cosine annealing strategy is combined, i.e. resetting the initial weights α = 0.4, β = 0.3, γ = 0.3 every 20 epochs, to avoid local optimum.

[0163] The weight is dynamically adjusted according to the performance of the module, i.e. dynamic weight adjustment logic:

[0164] When the occlusion module L doaa dominates the training, its weight is reduced to force the model to pay attention to cross-modal fusion; for example, when the occlusion module L doaa recall rate advantage exceeds 5%, the weight coefficient α is reduced and the cross-modal module (CMSF) weight β is increased, i.e. when Δ = (R doaa -R cmsf ) / R all > 0.05, α ← α × 0.9, β ← β × 1.1.

[0165] When the small target module L stae performance is insufficient, its weight is increased; for example, when the small target module L stae weight γ increases with AP stae < 80%, γ ← γ + 0.05·(80% - AP stae );

[0166] The gradient norm constraint mechanism applies a 0.7 decay factor to the dominant module gradient, i.e. if then a 0.7 decay factor is applied to , so that the false detection rate in extreme scenarios is reduced and the module conflict rate is reduced.

[0167] It needs to be further explained that in the specific implementation process, the two-stage confrontation verification process includes standard test set evaluation and extreme scene stress test; the extreme scene simulation set generates dynamic occlusion patches (coverage 60%-80%) through CycleGAN; the Phong illumination model simulates mixed environment of overexposure (brightness > 240) and low illumination (brightness < 50); and dense young fruits are generated by super-resolution, 50-100 per image, area <0.5%.

[0168] After locating the feature confusion area by Grad-CAM, three-stage optimization is adopted through curriculum learning: the first stage strengthens the occlusion compensation ability and improves the recall rate, the second stage optimizes the cross-modal alignment and reduces the feature error, and the last stage adjusts the small target loss weight and improves the AP, finally reducing the performance fluctuation range of the model.

[0169] The dynamic weight competitive loss function is used to balance the contribution of each module, reduce the false detection rate in extreme scenes, and generate dynamic occlusion patches and dense small targets based on CycleGAN. Through curriculum learning, the weak links of the model are strengthened in stages, and the feature conflict rate between modules is reduced. That is, through the dynamic competition mechanism and the antagonistic iterative optimization of the collaborative verification framework, the stability of the model in extreme scenes is ensured. Among them, CycleGAN may generate noise that does not conform to the true branch shape when generating dynamic occlusion patches. To this end, supplementary constraints are added: when generating occlusion patches, it is forced to follow the true branch shape database, such as leaf vein direction and branch angle.

[0170] It needs to be further explained that in the specific implementation process, the physical data driven method of the spectral reflectance matching verification module is based on ASDFieldSpec4 spectrometer to build an oil tea fruit reflectivity database, covering 650-950nm band data under different growth periods (young fruit / swelling period / mature period) and light conditions (noon / cloudy / dusk), sampling interval 5nm; when the 780nm channel feature exceeds the threshold [0.68, 0.79], trigger the RGB branch to increase the spatial attention weight of the corresponding position by 1.3 times, and recover the reflectivity effective channel ratio to 82% through 3 sub-pixel convolutions. This method reduces the spectral misjudgment rate and shortens the response time.

[0171] It needs to be further explained that in the specific implementation process, the hardware deployment scheme of the attention mechanism-based camellia fruit recognition optimization system: the system hardware uses IMX477 RGB sensor and SWIR-640 near-infrared camera to synchronously collect images, and Jetson XavierNX embedded platform allocates 2 CPU cores to run the occlusion perception module (delay <30ms), and 128 CUDA cores to process cross-modal fusion (occupancy rate ≤85%). The recognition result is transmitted to the mechanical arm controller through the CAN bus, and the response delay is <200ms, the overall power consumption is small, and it is lower than the traditional scheme; the output interface transmits the recognition result to the agricultural machinery controller through the CAN bus at a frequency of 100Hz, and the response delay is <200ms.

[0172] It needs to be further explained that in the specific implementation process, the implementation of the system in the orchard monitoring scene includes:

[0173] Unmanned aerial dynamic acquisition unit: the quadrotor unmanned aerial vehicle carrying the image acquisition unit cruises in a serpentine path at a constant height, realizes centimeter-level track tracking through the RTK positioning module, and dynamically adapts the image acquisition frequency to the flight speed;

[0174] Real-time processing-feedback control loop:

[0175] a) The processing unit completes single-frame camellia fruit recognition within 150ms, and transmits the positioning coordinates to the agricultural machinery through the LoRa wireless module;

[0176] b) When the mechanical arm controller generates the picking path based on the fruit coordinates, it synchronously receives the predicted future 3 frames of occlusion trend of the dynamic occlusion perception module, and dynamically adjusts the end effector motion trajectory;

[0177] Spectrum-space joint verification mechanism: when the spectral reflectance matching verification module detects 5 consecutive frames of abnormal reflectance, it triggers the unmanned aerial vehicle to hover and starts multi-angle compensation shooting, including:

[0178] Pitch angle ±15° deflection shooting 3 groups of images;

[0179] Turn on the near-infrared fill light to re-capture features;

[0180] Input multi-view features into the collaborative verification framework for confidence voting, and only adopt the recognition result consistent with the prediction of the two sets of models.

[0181] It needs to be further explained that in the specific implementation process, in the orchard deployment, the quadcopter unmanned aerial vehicle cruises at a height of 3±0.5m, the speed is 1-2m / s, the frame rate is adaptively adjusted (10-15fps), the RTK positioning error is <2cm. The recognition result is transmitted to the mechanical arm through LoRa within 150ms, and the picking path is dynamically adjusted in combination with the future 3 frames of shielding prediction. When detecting 5 consecutive frames of reflectivity anomaly, the unmanned aerial vehicle hovers and deflects ±15° to take 3 groups of multi-view images, cooperates with 850nm fill light (300lux) to recapture features, and after confidence voting decision, the missing detection rate is reduced, and the probability of mechanical arm touching leaves is reduced.

[0182] The system improves the accuracy of camellia fruit recognition in the occlusion scene through the dynamic divide-and-conquer attention mechanism, cross-modal spectral fusion, small target optimization engine and collaborative verification framework. The cross-modal mismatch rate is reduced, the young fruit detection IoU is improved, and the real-time response is realized through embedded hardware deployment, which improves the orchard inspection efficiency and provides a high-precision and low-power solution for smart agriculture.

[0183] The system enhances the fruit edge features in the occlusion area division and differential processing through the dynamic divide-and-conquer attention mechanism, and captures the dynamic changes of occlusion in combination with the spatio-temporal prediction model, which significantly improves the feature discrimination ability in the dense branch and leaf environment. The cross-modal spectral fusion network uses cross-attention gating and physical prior constraint to deeply fuse RGB texture and near-infrared reflectivity features, effectively enhancing the target recognition robustness in low-contrast scenes and reducing the risk of mismatch caused by sudden changes in light.

[0184] For the small target detection problem, the system enhances the multi-scale receptive field and dynamically allocates the loss weight to accurately capture the small gradient changes of young fruits and balance the semantic and detail features. At the same time, embedded hardware deployment realizes efficient parallel computing to ensure real-time response in complex scenes. The collaborative verification framework combines adversarial optimization and module competition mechanism to iteratively strengthen the weak links of the model and improve the stability and generalization ability of the system, providing a high-precision and low-latency camellia fruit recognition solution for smart agriculture.

[0185] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and implementations, it is to be understood that the terminology used is for the purpose of descriptive clarity and that it is intended to be limited only by the words recited in the appended claims. It is to be understood that the terms "including", "comprising", "consisting" and variations thereof do not preclude the addition of further integers to the combinations of integers specified in the claims. It is to be understood that the terms "including", "comprising", "consisting" and variations thereof encompass the various features of the embodiments described herein. It is not intended that the application be limited to the implementation that is described in detail and / or shown in the drawings. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and implementations, it is to be understood that the terminology used is for the purpose of descriptive clarity and that it is intended to be limited only by the words recited in the appended claims.

[0186] While the embodiments of the application have been shown and described herein, it is to be understood that the application is not limited to the embodiments described, and it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and implementations, it is to be understood that the terminology used is for the purpose of descriptive clarity and that it is intended to be limited only by the words recited in the appended claims.

Claims

1. An attention-based camellia fruit recognition optimization system, characterized in that, include: The dynamic occlusion perception attention module is connected to the input layer of the cross-modal spectral feature collaborative fusion network through the spatiotemporal dual-stream occlusion inference unit. The output of the cross-modal spectral feature collaborative fusion network is connected to the heterogeneous pyramid fusion layer of the small target adaptive optimization engine. The localization loss calculation unit of the small target adaptive optimization engine is fed back to the competitive loss function module of the collaborative verification framework. The dynamic occlusion perception attention module includes: Spatial dimension processing unit: Input multi-scale feature maps and generate occlusion probability heatmaps through deformable convolutional layers; Temporal processing unit: Based on lightweight LSTM, it fuses the current frame heatmap and historical frame optical flow features to output occlusion trend prediction for the next 3-5 frames; The cross-modal spectral feature collaborative fusion network includes a spectral reflectance matching verification module, which is used to screen effective spectral features through physical prior constraints, and also includes a cross-attention gating mechanism between the RGB branch and the near-infrared branch; The microscale feature enhancement pyramid of the small target adaptive optimization engine inserts a multiscale receptive field enhancement layer into the feature pyramid network and dynamically allocates the layer weights through the feature reweighting module. The collaborative verification framework connects the modules through a shared underlying feature extractor and uses an adversarial verification process to iteratively optimize the weak areas of the model's decision-making.

2. The camellia fruit recognition optimization system based on attention mechanism according to claim 1, characterized in that: The dynamic occlusion perception attention module has a divide-and-conquer attention strategy, including occlusion region segmentation and differential processing: Occlusion region segmentation includes: using an occlusion probability heatmap to set dual thresholds to divide the feature map into occlusion region, transition region, and visible region; Differentiation processing includes: a) The occluded area uses a channel attention module to suppress the background channel response, which is then connected to a spatial attention module; b) Deploy a local contrast enhancement module in the transition region and connect it in parallel with the directional gradient histogram feature extractor; c) Establishing long-distance semantic associations by overlaying nonlocal neural networks over the visible area; During the training phase, the dynamic occlusion perception attention module injects random masks through an adversarial occlusion simulator, and during the inference phase, it employs a dual-threshold voting mechanism to retain overlapping regions.

3. The camellia fruit recognition optimization system based on attention mechanism according to claim 1, characterized in that: The cross-modal spectral feature collaborative fusion network includes: Spectral guiding unit: The spectral features output from the near-infrared branch generate a spatial mask, which is then multiplied with the RGB branch features; Spatial constraint unit: The semantic features of the RGB branch generate a weight matrix, and the near-infrared features are weighted by region; Dynamic route selector: Selects the fusion path based on target confidence and gradient magnitude score; In the heterogeneous pyramid fusion layer, high-level features are upsampled after feature alignment, while low-level features are fused with high-resolution spatial features through super-resolution fusion.

4. The camellia fruit recognition optimization system based on attention mechanism according to claim 1, characterized in that: The microscale feature enhancement pyramid of the small-objective adaptive optimization engine includes: Multi-scale receptive field enhancement layers are configured with different receptive field sizes; The feature reweighting module dynamically generates hierarchical weights based on the target size; Hybrid loss function: classification loss is dynamically weighted based on target attributes, while localization loss adopts a center point weighting strategy.

5. The camellia fruit recognition optimization system based on attention mechanism according to claim 1, characterized in that: The spectral reflectance matching verification module includes: Differentiable constraint unit: For the 61-channel feature map of the near-infrared branch output, corresponding to the 650-950nm band, piecewise linear activation is performed. When the channel value is within the measured threshold [0.65, 0.82] of this band, the original gradient is retained, and the gradient of the part exceeding the threshold is returned to zero. Inference verification mechanism: The fused features are sampled using a 16×16 sliding window. When the effective reflectance channel ratio of three consecutive windows is less than 60%, the super-resolution reconstruction compensation of the C3 layer of the RGB branch is triggered. The spectral reflectance matching verification module is based on the ASD FieldSpec4 spectrometer and uses a database to build the database.

6. The camellia fruit recognition optimization system based on attention mechanism according to claim 1, characterized in that: The sharing-competition mechanism of the collaborative verification framework includes: Parameter freezing strategy: The underlying parameters of MobileNetV3 and depthwise separable convolution are fixed during the first three training phases.

7. The camellia fruit recognition optimization system based on attention mechanism according to claim 6, characterized in that: The adversarial verification process includes the generation of extreme scenario simulation sets and an error feedback mechanism; The extreme scene simulation set generation includes generating dynamic occlusion patches based on CycleGAN with a coverage of 60% to 80%; generating overexposed and underexposed mixed images by combining the Phong illumination model; and generating 50-100 young fruits with an area of ​​less than 0.5% per image through super-resolution reconstruction. The error feedback mechanism includes: using Grad-CAM to locate feature confusion regions, and strengthening training in three stages through course learning: the first stage freezes the CMSF and STAE parameters and focuses on DOAA optimization; the second stage unfreezes the CMSF and sets the cross-modal similarity threshold to 0.7; the third stage jointly optimizes the STAE loss weights and feature pyramid parameters. Among them, the error feedback mechanism of backpropagating the location loss gradient to the adversarial verification process dynamically adjusts the optimization weights in the course learning stage.

8. The camellia fruit recognition optimization system based on attention mechanism according to claim 5, characterized in that: The physical data-driven method of the spectral reflectance matching verification module includes: Spectral database construction: Reflectance data in the 650-950nm band at different growth stages were collected using an ASD FieldSpec4 spectrometer; Backpropagation linkage optimization: When the near-infrared feature exceeds the threshold [0.68, 0.79] in the 780nm channel, the spatial attention weight of the corresponding position in the RGB branch is increased by 1.3 times; Real-time verification anomaly handling process: After freezing the near-infrared branch parameters, enable the C3 layer dilated convolution feature of the RGB branch to perform 3 sub-pixel convolution reconstructions until the effective reflectance channel accounts for ≥60%.

9. The camellia fruit recognition optimization system based on attention mechanism according to claim 1, characterized in that: The hardware deployment of the attention-based camellia fruit recognition optimization system includes: Image acquisition unit: IMX477 RGB sensor and SWIR-640 near-infrared camera synchronously trigger acquisition; Processing unit: Jetson Xavier NX embedded platform, configured with a multi-threaded task scheduler: a) The first thread runs the dynamic occlusion perception and attention module, which is allocated 2 CPU cores; b) The second thread runs the cross-modal fusion network, allocated 128 CUDA cores; c) The third thread runs the small target optimization engine, which is allocated 64 CUDA cores; Output interface: The identification results are transmitted to the agricultural machinery controller via the CAN bus.

10. The camellia fruit recognition optimization system based on an attention mechanism according to claim 9, characterized in that: The implementation of the system in an orchard monitoring scenario includes: Dynamic image acquisition unit for drones: The quadcopter drone equipped with the image acquisition unit cruises along a serpentine path at a constant altitude, and achieves centimeter-level trajectory tracking through the RTK positioning module. The image acquisition frequency is dynamically adapted to the flight speed. Real-time processing - feedback control loop: a) The processing unit completes single-frame camellia fruit identification within 150ms and transmits the positioning coordinates to agricultural machinery via the LoRa wireless module; b) When the robotic arm controller generates the picking path based on the fruit coordinates, it simultaneously receives the occlusion trend predicted by the dynamic occlusion perception module for the next 3 frames and dynamically adjusts the motion trajectory of the end effector. Spectral-spatial joint verification mechanism: When the spectral reflectance matching verification module detects five consecutive frames of reflectance anomalies, it triggers the drone to hover and initiates multi-angle compensation shooting, including: Three sets of images were captured by tilting the camera at an angle of ±15°. Turn on the near-infrared illuminator to recapture features; Multi-view features are input into the collaborative verification framework for confidence voting, and only the recognition results predicted by the two models that are consistent are adopted.

Images