A fruit counting method based on region focusing and scale correlation

Abstract

The application provides a fruit counting method based on region focusing and scale correlation, comprising the following steps: obtaining a fruit image to be counted; inputting the fruit image into a pre-trained region scale coordination network to generate a corresponding fruit density map; and performing integral operation on the fruit density map to obtain the number of fruits; wherein the region scale coordination network comprises an encoder, a region focusing module, a scale correlation module and a decoder. The region focusing module models local and global collaborative attention of deep semantic features, combines spatial gating to enhance fruit response and suppress background interference; the scale correlation module captures multi-scale fruit morphology through multi-expert deformable convolution pooling and dynamic weighted fusion; and the decoder fuses multi-level features to gradually restore resolution. The method realizes collaborative optimization of background interference suppression and multi-scale representation at the feature level, effectively solving the problem of low fruit counting accuracy in complex scenes such as dense, occlusion and light change.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a fruit counting method based on region focusing and scale correlation. Background Technology

[0002] With the deepening development of agricultural modernization and smart agriculture, automatic fruit counting has become a key technological link in achieving precision agricultural management. Accurate fruit quantity estimation is the foundation for yield forecasting, formulating scientific harvesting plans, optimizing water and fertilizer resource allocation, and assessing crop growth status, and is of great significance for improving orchard management efficiency and economic benefits.

[0003] Traditional fruit counting relies primarily on manual inspection or threshold segmentation methods based on simple image processing. The former is inefficient, costly, and highly subjective, while the latter is extremely sensitive to changes in ambient lighting and color, exhibiting poor robustness and making it difficult to apply to complex natural orchard scenarios. In recent years, deep learning-based technologies have become the mainstream solution for automatic fruit counting. These methods can be broadly categorized into two types: detection-based counting methods and density map regression-based counting methods. While detection-based counting methods can directly locate fruits, in orchard scenarios with dense fruit, severe occlusion, and significant background interference, the counting accuracy is easily reduced due to overlapping detection boxes and false detections. Although density map regression-based counting methods can alleviate the target occlusion problem, they still struggle to effectively distinguish background interference that is highly similar to the fruit and cannot simultaneously represent fruits with vastly different scales in the image, resulting in limited density map quality and decreased counting accuracy.

[0004] In summary, although existing technologies, especially density map methods, provide a feasible path for automatic fruit counting, their capabilities in suppressing interference from complex backgrounds and adaptively representing multi-scale fruit features remain insufficient. Therefore, a novel counting framework is needed to collaboratively address these two major challenges in order to achieve high-precision and robust automatic fruit counting in complex natural environments. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the present invention provides a fruit counting method based on region focusing and scale correlation to solve the technical problem that it is difficult to accurately represent the multi-scale features of fruits in complex backgrounds.

[0006] To achieve the above and other related objectives, this invention provides a fruit counting method based on region focusing and scale correlation, comprising the following steps: acquiring an image of the fruit to be counted; inputting the fruit image into a pre-trained region-scale coordination network to generate a corresponding fruit density map; performing an integral operation on the fruit density map to obtain the number of fruits; wherein, the region-scale coordination network comprises: an encoder for progressively downsampling the input image to extract multi-scale features; a region focusing module for receiving the deepest semantic features output by the encoder, enhancing the feature response of the fruit region and suppressing background interference through a local and global collaborative attention mechanism and spatial gating, and outputting region-enhanced features; a scale correlation module for receiving the region-enhanced features, outputting scale-coordinated features through multi-expert parallel deformable convolutional pooling and dynamic weighted fusion, and simultaneously outputting multiple single-scale expert features; and a decoder for receiving and fusing multi-level features from the scale correlation module and the encoder, progressively restoring spatial resolution through multi-stage multi-feature fusion, upsampling, and channel attention gating, and finally outputting the fruit density map.

[0007] In one embodiment of the present invention, the encoder constructs a multi-level feature pyramid by downsampling step by step to extract the multi-scale features.

[0008] In one embodiment of the present invention, the region focusing module includes: a first attention branch, used to perform attention modeling on the deepest semantic features of the input with a first receptive field range, and output a first enhanced feature; a second attention branch, used to perform attention modeling on the deepest semantic features of the input with a second receptive field range larger than the first receptive field range, and output a second enhanced feature; a feature fusion unit, used to perform adaptive weighted fusion of the first enhanced feature and the second enhanced feature, and output a fused feature; a spatial gating unit, used to generate a spatial weight map from the deepest semantic features of the input, and use the spatial weight map to perform spatial selective enhancement on the fused feature, and output a gated feature; and a residual connection unit, used to fuse the gated feature with the deepest semantic features of the input, and output the region enhanced feature.

[0009] In one embodiment of the present invention, the first attention branch is a local window attention branch, which is used to divide the deepest semantic features of the input into windows and apply multi-head self-attention independently within each window; the second attention branch is a global downsampling attention branch, which is used to perform spatial downsampling on the deepest semantic features of the input, apply multi-head self-attention, and then restore the original resolution through upsampling.

[0010] In one embodiment of the present invention, the scale association module includes: a scale perceptron, used to perform global context analysis on the input region enhancement features and generate a set of scale weight vectors; a deformable convolutional expert pool, containing multiple parallel deformable convolutional experts, each expert having a different base dilation rate, used to extract features from the input region enhancement features at different scale ranges and output multiple single-scale expert features; and a weighted fusion unit, used to perform weighted fusion of the multiple single-scale expert features using the scale weight vectors and output the scale-coordinated features; wherein the multiple single-scale expert features are also sent as independent outputs to different stages of the decoder.

[0011] In one embodiment of the present invention, the scale perceptron includes: a global average pooling layer for performing global average pooling on the input region enhancement features in spatial dimension to obtain channel descriptors; a multilayer perceptron for performing nonlinear transformation on the channel descriptors; and a Softmax activation function for mapping the output of the multilayer perceptron to a scale weight vector of probability distribution.

[0012] In one embodiment of the present invention, each deformable convolutional expert includes: a spatial offset learning branch for learning a two-dimensional spatial offset of sampling points from the input region enhancement features; an adaptive dilation rate learning branch for learning a dynamic dilation rate parameter from the input region enhancement features; a modulation mechanism learning branch for learning a modulation scalar of sampling points from the input region enhancement features; and a deformable convolution operation unit for performing deformable convolution operations on the input region enhancement features according to the spatial offset, the adaptive dilation rate parameter, and the modulation scalar, and outputting corresponding expert features.

[0013] In one embodiment of the present invention, the decoder includes three cascaded fusion stages, corresponding to deep, mid-level, and shallow feature processing, respectively. The first stage receives the scale-coordinated features, large-scale expert features from the plurality of single-scale expert features, and deep semantic features output by the encoder, and outputs a first fused feature. The second stage receives the first fused feature, mid-scale expert features from the plurality of single-scale expert features, and mid-level features output by the encoder, and outputs a second fused feature. The third stage receives the second fused feature, small-scale expert features from the plurality of single-scale expert features, and shallow detail features output by the encoder, and outputs a third fused feature. The third fused feature is processed to output the fruit density map.

[0014] In one embodiment of the present invention, each fusion stage includes: a multi-feature fusion unit for fusing multiple input features received in this stage; an upsampling unit for upsampling the fused features at spatial resolution; a convolution unit for performing convolution transformation on the upsampled features; a normalization unit for normalizing the convolution output; an activation function unit for performing nonlinear activation on the normalized features; and a channel attention gating unit for adaptively weighting the activated features according to the channel dimension and outputting the fused features of this stage.

[0015] The beneficial effects of this invention are as follows: This invention proposes a fruit counting method based on region focusing and scale correlation. This method constructs a region-scale coordination network. First, it uses a region focusing module to perform local and global collaborative attention modeling on the deepest semantic features of the encoder, and combines a spatial gating mechanism to adaptively enhance the fruit region response and suppress background interference such as leaf reflection. Then, through multi-expert parallel deformable convolutional pooling and dynamic weighted fusion in the scale correlation module, it simultaneously captures fruit morphological features at different scales. Finally, the decoder fuses multi-level features to restore spatial resolution step by step. This method achieves collaborative optimization of background interference suppression and multi-scale representation at the feature level, effectively solving the problem of low fruit counting accuracy in complex scenes such as dense, occluded, and changing lighting conditions. The mean absolute error on the grape dataset is reduced to 7.51, which is significantly better than existing methods. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The accompanying drawings are incorporated in and constitute a part of this specification, illustrating embodiments consistent with this application, and are used together with the description to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a fruit counting method provided in an embodiment of the present invention;

[0018] Figure 2 This is an architecture diagram of a regional-scale coordination network provided in an embodiment of the present invention;

[0019] Figure 3 This is an architectural diagram of a region focusing module provided in an embodiment of the present invention;

[0020] Figure 4 This is an architecture diagram of a scale association module provided in an embodiment of the present invention;

[0021] Figure 5 This is an architecture diagram of a decoder provided in an embodiment of the present invention. Detailed Implementation

[0022] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. It should be noted that, unless otherwise specified, the following embodiments and features can be combined with each other. In addition to the specific methods, equipment, and materials used in the embodiments, based on the knowledge of the prior art and the description of the present invention by those skilled in the art, any prior art methods, equipment, and materials similar to or equivalent to the methods, equipment, and materials in the embodiments of the present invention can be used to implement the present invention.

[0023] It should be understood that the terminology used in the embodiments of this invention is for describing specific implementations and not for limiting the scope of protection of this invention. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art.

[0024] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In some embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0025] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations that may be implemented in the methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0026] Please see Figure 1 , Figure 1An embodiment of the present invention provides a fruit counting method based on region focusing and scale correlation, comprising the following steps: (1) acquiring images of the fruits to be counted; (2) inputting the fruit images into a pre-trained region-scale coordination network to generate a corresponding fruit density map; (3) performing an integral operation on the fruit density map to obtain the number of fruits. The main improvement of the present invention lies in the region-scale coordination network, the architecture of which is shown in the figure below. Figure 2 As shown, it includes an encoder, a region focusing module (RFM), a scale association module (SAM), and a decoder. The encoder is used to downsample the input image step by step to extract multi-scale features; the region focusing module is used to receive the deepest semantic features output by the encoder, enhance the feature response of the fruit region and suppress background interference through a local and global collaborative attention mechanism and spatial gating, and output region-enhanced features; the scale association module is used to receive the region-enhanced features, output scale-coordinated features through multi-expert parallel deformable convolutional pooling and dynamic weighted fusion, and simultaneously output multiple single-scale expert features; the decoder is used to receive and fuse multi-level features from the scale association module and the encoder, restore the spatial resolution step by step through multi-stage multi-feature fusion, upsampling and channel attention gating, and finally output the fruit density map.

[0027] The region-scale coordination network is not simply a stack of attention and convolutional modules, but rather employs a collaborative enhancement mechanism of region focusing and scale association. The region focusing module filters fruit-related regions from deep semantic features, providing clean input for subsequent scale modeling; the scale association module then adaptively fuses multi-scale features; and the decoder further fuses shallow detail features from the encoder, achieving progressively refined reconstruction from global to local levels. These three components work together to enable the network to suppress background interference while accurately representing fruits at different scales, fundamentally improving the quality of the density map and the accuracy of counting.

[0028] It is important to note that Figure 1 Subsequent experiments all use grapes as an example for illustration, but the method provided by this invention is not limited to counting grapes; it is also applicable to the automatic counting of various spherical fruits such as cherries, strawberries, and apples. It is understood that for different types of fruits, if the network is trained using the corresponding dataset, the resulting model will have better counting performance on that type of fruit; if the same model is trained using a dataset containing multiple types of fruits, a more general counting model can be obtained, but its counting reliability on a single category may decrease slightly.

[0029] In one specific embodiment of the present invention, the encoder constructs a multi-level feature pyramid through stepwise downsampling to extract multi-scale features. The deepest semantic feature output can be denoted as F. E It can be represented as: FE =H Enc (I), where H Enc The encoder function is represented by , where I is the original image input to the encoder. This process maps the original image from pixel space to feature space, providing a stable and semantically rich input for subsequent modules. Through progressive downsampling by the encoder, the network can construct a multi-level feature pyramid from high-resolution details to low-resolution semantics, providing the deepest semantically rich features for subsequent region focusing modules, and also providing multi-scale skip connection features for the decoder. This is the foundation for achieving global semantic perception and local detail restoration.

[0030] Please see Figure 3 In a specific embodiment of the present invention, the region focusing module includes a first attention branch, a second attention branch, a feature fusion unit, a spatial gating unit, and a residual connection unit. The first attention branch is used to process the deepest semantic feature F of the input within a first receptive field. E Perform attention modeling and output the first enhanced feature F. local The second attention branch is used to process the deepest semantic features F of the input with a second receptive field larger than the first receptive field. E Perform attention modeling and output the second enhanced feature F. global The feature fusion unit is used to adaptively weight and fuse the first and second enhanced features, outputting the fused feature, which can be expressed by the formula: In the formula, α∈[0,1] represents the parameters automatically learned by the network, which can dynamically adjust the contribution ratio of local and global information based on the input feature content. The spatial gating unit is used to generate a spatial weight map G from the deepest semantic features of the input, which can be expressed by the formula: In the formula, σ represents the Sigmoid activation function; and the spatial weight map G is used to evaluate the fused features F. fused Perform spatially selective enhancement and output gated feature F gated It can be expressed by the formula: In the formula, the Repeat operation means copying the single-channel spatial weight map G C times along the channel dimension, so that it is compared with the multi-channel feature map F. fused The shapes are consistent to allow for element-wise multiplication, enabling differentiated enhancement at different spatial locations. Residual connection units are used to connect the gated feature F. gated With the deepest semantic features F of the input E The fusion process can be expressed by the following formula: In the formula, feature fusion is achieved through residual connections and layer normalization, and the output region enhancement feature F is obtained. R .

[0031] The region focusing module is the core component of this invention for suppressing background interference. This module captures the local structure of the fruit through a first attention branch and perceives the global distribution through a second attention branch. After adaptive fusion using learnable weights, the contribution ratio of local and global information can be dynamically adjusted according to the image content. Based on this, a spatial gating unit generates a spatial weight map from the original input features, spatially selectively enhancing the fused features, allowing the network to focus on areas where the fruit may be present. Residual connections ensure training stability and the integrity of feature transfer. Through this design, complex background interference such as branches, leaves, and reflections is effectively suppressed at the feature level, and the feature response of the fruit region is significantly enhanced, providing high-quality input for the subsequent scale correlation module.

[0032] In a specific embodiment of the present invention, the first attention branch is a local window attention branch, used to divide the deepest semantic features of the input into windows and independently apply multi-head self-attention within each window; the second attention branch is a global downsampling attention branch, used to spatially downsample the deepest semantic features of the input, apply multi-head self-attention, and then restore the original resolution through upsampling. The local window attention branch, by dividing the feature map into windows and independently calculating attention within each window, can efficiently capture local details and structural features of the fruit region with low computational cost; the global downsampling attention branch, by spatially downsampling the feature map and then calculating global attention, can model the dependencies between distant pixels and perceive the overall fruit distribution pattern of the image. The two branches work together to achieve comprehensive feature enhancement from local to global perspectives.

[0033] The multi-head attention mechanism linearly transforms the input features into multiple sets of queries, keys, and values. Each set independently computes scaled dot product attention, and the outputs of all attention heads are concatenated and linearly transformed to obtain the final result. This mechanism can simultaneously capture the dependencies between features from multiple different subspaces, enhancing the model's ability to model contextual information of the fruit region. It is the foundation for the effective operation of local window attention and global downsampling attention.

[0034] Please see Figure 4 In a specific embodiment of the present invention, the scale association module includes a scale perceptron, a deformable convolutional expert pool, and a weighted fusion unit; the scale perceptron is used to perform global context analysis on the input region enhancement features and generate a set of scale weight vectors w; the deformable convolutional expert pool contains multiple parallel deformable convolutional experts, each with a different base dilation rate, used to extract features from the input region enhancement features at different scale ranges and output multiple single-scale expert features O. k The weighted fusion unit is used to weight and fuse multiple single-scale expert features using scale weight vectors, outputting scale-coordinated features, which can be expressed by the formula: In addition, multiple single-scale expert features are also sent as independent outputs to different stages of the decoder.

[0035] The scale association module is the core component of this invention for solving the problem of fruit scale variation. The scale perceptron performs global context analysis on the input features, generating a set of K-dimensional scale weight vectors to represent the dependence of the current image on different scale contexts. The deformable convolutional expert pool contains K parallel deformable convolutional experts, each with a different base dilation rate, capable of capturing contextual information across different ranges. Unlike traditional fixed-grid convolution, deformable convolution can adaptively adjust the sampling position according to the irregular shape of the fruit, learning a two-dimensional spatial offset field from the input features through a small convolutional sub-network, allowing the receptive field to flexibly fit the target contour. The weighted fusion unit uses scale weights to weightedly sum the outputs of each expert, obtaining the final scale-coordinated features. Simultaneously, the intermediate outputs of each expert are independently fed as single-scale expert features to different stages of the decoder, achieving hierarchical utilization of scale information. This design enables the network to autonomously strengthen the feature responses of the most relevant scales and suppress interference from irrelevant scales based on the actual distribution of fruit in the input image.

[0036] In a specific embodiment of the present invention, the scale perceptron includes a global average pooling layer, a multilayer perceptron, and a Softmax activation function. The global average pooling layer performs spatial-dimensional global average pooling on the input region enhancement features to obtain channel descriptors. The multilayer perceptron performs nonlinear transformations on the channel descriptors, and its specific architecture is as follows: Figure 4 As shown in the dashed box above; the Softmax activation function is used to map the output of the multilayer perceptron to a scale weight vector of the probability distribution, which can be expressed by the formula: z = GAP(F R ), w=Softmax(MLR(z)). Global average pooling compresses spatial dimension information into channel descriptors, preserving the relative response strength between channels; the multilayer perceptron performs a nonlinear transformation on the channel descriptors to learn the complex interaction relationships between channels; the Softmax activation function maps the output to a scale weight vector in the form of a probability distribution, ensuring that the sum of the weights of each expert is 1, which facilitates stable training of subsequent weighted fusion.

[0037] In a specific embodiment of the present invention, the architecture of each deformable convolutional expert is as follows: Figure 4As shown in the dashed box below, it includes a spatial offset learning branch, an adaptive dilation rate learning branch, a modulation mechanism learning branch, and a deformable convolution operation unit. The spatial offset learning branch learns the two-dimensional spatial offset of the sampling points from the input region enhancement features; the adaptive dilation rate learning branch learns the dynamic dilation rate parameter from the input region enhancement features; the modulation mechanism learning branch learns the modulation scalar of the sampling points from the input region enhancement features; and the deformable convolution operation unit performs deformable convolution operations on the input region enhancement features based on the spatial offset, the adaptive dilation rate parameter, and the modulation scalar, outputting the corresponding expert feature O. K Each expert's calculation process can be represented as: ,in, For F, a small convolutional subnetwork is used to generate F R The learned two-dimensional offset field makes the convolution sampling points no longer limited to a fixed grid.

[0038] Through the spatial offset learning branch, each deformable convolutional expert can dynamically adjust the sampling grid according to the irregular shape of the fruit, breaking through the geometric limitations of fixed convolutional kernels. The adaptive dilation rate learning branch enables the expert to dynamically adjust the receptive field size according to the scale of the fruit. The modulation mechanism learning branch further weights and modulates the feature contributions of the sampling points, enhancing the model's attention to important sampling points. The combination of these three approaches allows the deformable convolutional expert to flexibly adapt to the morphological differences of the fruit caused by distance, occlusion, and changes in viewing angle, significantly improving the accuracy and robustness of multi-scale feature extraction.

[0039] Please see Figure 5 In a specific embodiment of the present invention, the decoder includes three cascaded fusion stages, corresponding to deep, mid, and shallow feature processing, respectively. The first stage receives scale-coordinated features, large-scale expert features from multiple single-scale expert features, and deep semantic features output by the encoder, outputting a first fused feature. The second stage receives the first fused feature, mid-scale expert features from multiple single-scale expert features, and mid-level features output by the encoder, outputting a second fused feature. The third stage receives the second fused feature, small-scale expert features from multiple single-scale expert features, and shallow detail features output by the encoder, outputting a third fused feature. The third fused feature is then processed to output a fruit density map. The decoder adopts a three-stage cascaded structure, corresponding to deep, mid, and shallow feature processing, respectively. The deep stage fuses global semantic information to grasp the overall fruit distribution, the mid-stage fuses structural information to distinguish adjacent fruit clusters, and the shallow stage fuses detail information to accurately depict the edges of individual fruits. This design achieves decoupled expression and progressive fusion of scale information, avoiding the loss of scale information caused by a single fusion method.

[0040] In a specific embodiment of the present invention, each fusion stage includes a multi-feature fusion unit, an upsampling unit, a convolution unit, a normalization unit, an activation function unit, and a channel attention gating unit. The multi-feature fusion unit fuses multiple input features received in this stage; the upsampling unit upsamples the fused features at spatial resolution; the convolution unit performs convolutional transformations on the upsampled features; the normalization unit normalizes the convolutional output; the activation function unit performs nonlinear activation on the normalized features; and the channel attention gating unit adaptively weights the activated features along the channel dimension, outputting the fused features for this stage. In each fusion stage, the multi-feature fusion unit integrates features from different sources; the upsampling unit gradually restores spatial resolution; the combination of convolution, normalization, and activation functions performs nonlinear transformations and standardization on the features, stabilizing the training process; and the channel attention gating unit adaptively weights the features along the channel dimension, further enhancing the feature responses of important channels. This standardization process ensures the stability and effectiveness of feature fusion at each stage.

[0041] The effects of the present invention will be further explained below with reference to experiments.

[0042] (1) Experimental conditions: The experimental system of this invention uses the Ubuntu 20.04 LTS operating system, the deep learning framework is PyTorch 1.11.0, and is equipped with the CUDA 11.2 acceleration library to support GPU computing. In terms of hardware, the AMD EPYC7642 processor (2.3GHz, 48 cores) and NVIDIA GeForce RTX 3090 graphics card (24GB VRAM) are mainly used to complete the model training and inference tasks. The grape images used in this study are all taken from a vineyard. Images were collected in the actual production environment, and a total of 263 original images with a size of 1024×1024 pixels were obtained. The images include various lighting environments such as normal lighting, overexposure and low lighting, and also include fruit targets of varying scales, including close-up shots of a single fruit cluster and distant compositions of multiple fruit clusters, so as to ensure that the trained model can adapt to different target scales. To further expand the sample size, image augmentation techniques were used to extend the original dataset, including flipping, rotating, and color adjustment, to simulate lighting variations and device imaging differences that may occur in real-world applications. After data augmentation, the total number of images increased to 1052. Proportionally, 80% of the images were used as the training set, and the remaining 20% ​​were used as the test set.

[0043] (2) Evaluation index: In order to objectively evaluate the accuracy advantage of the present invention in fruit counting method, mean absolute error (MAE) and root mean square error (RMSE) are used as performance measurement indexes. These two indexes are widely used in the evaluation of error measurement in counting tasks, and their specific definitions are as follows.

[0044] ,

[0045] ,

[0046] In the above formula, N is the number of test images. For predicting the count, Count the truth values ​​for the i-th image.

[0047] (3) Experimental Content and Result Analysis: Comparison with State-of-the-art Methods on Grape Datasets. The Region-Scale Coordination Network (RSHNet) proposed in this invention was compared with seven other density map counting models (MCNN, CSRNet, SFCN, BL, DMCount, MAN, UOT). The comparison results are shown in Table 1. As can be seen from Table 1, our RSHNet performed best in all experimental results, demonstrating its superiority in the grape fruit counting task.

[0048] Table 1: Comparison of results with other density map models on the grape dataset.

[0049] MCNN 18.71 25.30 CSRNet 16.92 23.41 SFCN 15.21 21.56 BL 14.53 20.84 DMCount 13.64 18.52 MAN 11.38 17.98 UOT 10.25 16.85 RSHNet (Ours) 7.51 13.21

[0050] In summary, this invention addresses two core challenges in fruit counting in natural orchard scenarios: severe background interference and large differences in fruit scale. It proposes a counting method that synergistically enhances the effects of region focusing and scale correlation. The region focusing module adaptively suppresses background interference, while the scale correlation module adaptively fuses multi-scale features. These two modules work synergistically within the network, significantly improving the quality of the density map and the accuracy of the counting. Experiments demonstrate that this method achieves state-of-the-art performance on a grape dataset, with a mean absolute error as low as 7.51, exhibiting excellent robustness and generalization ability.

[0051] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A fruit counting method based on region focusing and scale correlation, characterized in that, Includes the following steps: Obtain the image of the fruit to be counted; The fruit image is input into a pre-trained region-scale coordination network to generate a corresponding fruit density map; The number of fruits is obtained by integrating the fruit density map. The regional scale coordination network includes: An encoder is used to downsample the input image step by step to extract multi-scale features; The region focusing module is used to receive the deepest semantic features output by the encoder, enhance the feature response of the fruit region and suppress background interference through a local and global collaborative attention mechanism and spatial gating, and output region-enhanced features. A scale correlation module is used to receive the region enhancement features, output scale-coordinated features through multi-expert parallel deformable convolutional pooling and dynamic weighted fusion, and simultaneously output multiple single-scale expert features; and The decoder is used to receive and fuse multi-level features from the scale association module and the encoder. Through multi-stage multi-feature fusion, upsampling and channel attention gating, the spatial resolution is restored step by step, and finally the fruit density map is output. The scale association module includes: The scale perceptron is used to perform global context analysis on the input region enhancement features and generate a set of scale weight vectors. The deformable convolutional expert pool contains multiple parallel deformable convolutional experts, each with a different base dilation rate, used to extract features from the input region enhancement features at different scales, outputting multiple expert features at a single scale; and The weighted fusion unit is used to perform weighted fusion of the multiple single-scale expert features using the scale weight vector, and output the scale-coordinated features; Among them, the multiple single-scale expert features are also sent as independent outputs to different stages of the decoder; The scale perceptron includes: The global average pooling layer is used to perform global average pooling on the input region enhancement features in the spatial dimension to obtain channel descriptors; A multilayer perceptron is used to perform a nonlinear transformation on the channel descriptor; and The Softmax activation function is used to map the output of a multilayer perceptron to a scale weight vector of a probability distribution. Each of the aforementioned deformable convolution experts includes: The spatial offset learning branch is used to learn the two-dimensional spatial offset of the sampling points from the input region enhancement features; An adaptive dilation rate learning branch is used to learn dynamic dilation rate parameters from the input region augmentation features; A modulation mechanism learning branch is used to learn the modulation scalar of sampling points from the input region enhancement features; and The deformable convolution operation unit is used to perform deformable convolution operation on the input region enhancement features according to the spatial offset, the adaptive dilation rate parameter and the modulation scalar, and output the corresponding expert features.

2. The fruit counting method based on region focusing and scale correlation according to claim 1, characterized in that, The encoder constructs a multi-level feature pyramid through step-by-step downsampling to extract the multi-scale features.

3. The fruit counting method based on region focusing and scale correlation according to claim 1, characterized in that, The region focusing module includes: The first attention branch is used to model the deepest semantic features of the input with the first receptive field and output the first enhanced feature. The second attention branch is used to perform attention modeling on the deepest semantic features of the input with a second receptive field range that is larger than the first receptive field range, and outputs the second enhanced feature. The feature fusion unit is used to adaptively weight and fuse the first enhanced feature and the second enhanced feature to output the fused feature; A spatial gating unit is used to generate a spatial weight map from the deepest semantic features of the input, and to perform spatial selective enhancement on the fused features using the spatial weight map, outputting gated features; and The residual connection unit is used to fuse the gated features with the deepest input semantic features to output the region enhancement features.

4. The fruit counting method based on region focusing and scale correlation according to claim 3, characterized in that, The first attention branch is a local window attention branch, which is used to divide the deepest semantic features of the input into windows and apply multi-head self-attention independently within each window; the second attention branch is a global downsampling attention branch, which is used to spatially downsample the deepest semantic features of the input, apply multi-head self-attention, and then restore the original resolution through upsampling.

5. The fruit counting method based on region focusing and scale correlation according to claim 1, characterized in that, The decoder includes three cascaded fusion stages, corresponding to deep, middle, and shallow feature processing, respectively. The first stage is used to receive the scale coordination features, the large-scale expert features from the multiple single-scale expert features, and the deep semantic features output by the encoder, and output the first fusion feature. The second stage is used to receive the first fused feature, the mesoscale expert feature from the plurality of single-scale expert features, and the mid-level feature output by the encoder, and output the second fused feature. The third stage is used to receive the second fused feature, the small-scale expert feature from the multiple single-scale expert features, and the shallow detail feature output by the encoder, and output the third fused feature. The third fusion feature is processed to output the fruit density map.

6. The fruit counting method based on region focusing and scale correlation according to claim 5, characterized in that, Each of the aforementioned fusion stages includes: The multi-feature fusion unit is used to fuse the multiple input features received in this stage; The upsampling unit is used to perform spatial resolution upsampling on the fused features; A convolutional unit is used to perform convolutional transformations on the upsampled features. The normalization unit is used to normalize the convolution output; Activation function units are used to perform nonlinear activation on the normalized features; and The channel attention gating unit is used to adaptively weight the activated features according to the channel dimension and output the fused features for this stage.

Images