Small target local sensing and relation reasoning detection system and method

Abstract

The invention relates to the technical field of computer vision, in particular to a small target local sensing and relation reasoning detection system and method. The system comprises a target detection backbone network and a local visible area sensing and relation reasoning module which are cascaded, the target detection backbone network comprises a cross-stage feature fusion unit which is arranged in a down-sampling stage and is used for carrying out cross fusion on a feature after down-sampling and a reserved high-resolution feature and outputting an enhanced feature; the cross-domain collaborative attention unit is arranged in the up-sampling stage and is used for comprehensively enhancing the features and the up-sampling features to carry out attention calculation; and the local visible region sensing and relation reasoning module is used for receiving the enhanced high-resolution feature map, performing local feature enhancement and aggregation and outputting a final detection result. The method can actively enhance the detail features of the small target, and has the characteristic that the local visible information can be utilized to carry out relation reasoning.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a small target local perception and relational reasoning detection system and method. Background Technology

[0002] With the rapid development of artificial intelligence technology, target detection technology based on deep convolutional neural networks has achieved remarkable success in fields such as autonomous driving, security monitoring, and industrial quality inspection. In recent years, this technology has been gradually applied to agricultural production to realize intelligent operations such as monitoring crop growth status, yield prediction, and automated harvesting. As a high-value fruit, strawberries and blueberries have relatively controllable planting environments (such as greenhouses and sheds), but the visual detection scenarios are still very complex, specifically manifested as: (1) Large scale differences: There are mature and unripe fruits on the same target fruit plant at the same time, and their size differences are significant. Even mature fruits show great scale changes in the image due to different distances from the camera. (2) Dense and small target problems: There are many targets to be detected and they are densely distributed, which makes it easy for traditional detection methods to cause missed detections and false detections.

[0003] While existing general-purpose deep learning-based object detection algorithms (such as Faster R-CNN, YOLO, and the SSD series) perform excellently on publicly available datasets, their accuracy and robustness are still insufficient when directly applied to the complex agricultural environments described above, particularly for densely packed small targets and severely occluded targets. Therefore, there is an urgent need for a method specifically designed for small target detection in complex environments to improve accuracy and recall, providing reliable technical support for subsequent automated agricultural operations. Summary of the Invention

[0004] Based on this, it is necessary to provide a small target local perception and relational reasoning detection system and method that can actively enhance the detailed features of small targets, and has the feature of using locally visible information to perform relational reasoning.

[0005] To achieve the above-mentioned objectives of this invention, the technical solution adopted is as follows: A small target local perception and relation reasoning detection system includes a cascaded target detection backbone network and a local visible region perception and relation reasoning module. The target detection backbone network is used to extract features from the input image and includes: a cross-stage feature fusion unit, located in the downsampling stage of the backbone network, which generates feature retention branches and downsampling branches in parallel at each downsampling stage, and cross-fused the downsampled features with the retained high-resolution features to output enhanced features; and a cross-domain collaborative attention unit, located in the upsampling stage of the backbone network, which performs attention calculation by integrating the enhanced features and upsampling features to output an enhanced high-resolution feature map. The local visible region perception and relation reasoning module is connected to the output of the target detection backbone network and receives the enhanced high-resolution feature map, performs local feature enhancement and aggregation, and outputs the final detection result.

[0006] Preferably, the locally visible region perception and relationship reasoning module specifically includes: A local region proposal network is used to receive the enhanced high-resolution feature map and predict local candidate regions present in the image. The feature extraction and enhancement unit is used to map the local candidate regions to fixed-size features and use deformable convolution to enhance local features; the local relationship reasoning submodule is used to receive the enhanced local features, establish spatial context relationships between different candidate regions and aggregate features, perform classification and regression, and output the final detection result.

[0007] A small target local perception and relation reasoning detection method based on the system includes the following steps: acquiring a target scene image to be detected; inputting the image to be detected into the target detection backbone network; in the downsampling stage, generating a feature retention branch and a downsampling branch in parallel through a cross-stage feature fusion unit; cross-fusion the downsampled features with the retained high-resolution features to output enhanced features; in the upsampling stage, inputting the enhanced features into a cross-domain collaborative attention unit; performing attention calculation by combining the enhanced features and the upsampling features to output an enhanced high-resolution feature map; inputting the high-resolution feature map into a local visible area perception and relation reasoning module for local feature enhancement and aggregation; and outputting the final detection result.

[0008] Preferably, the specific method for outputting enhanced features by the cross-stage feature fusion unit is as follows: the image to be detected enters the downsampling stage through convolution and pooling operations; in the downsampling stage, the resolution of the image gradually decreases from the original size, while the corresponding number of feature channels gradually increases; In the Each downsampling stage outputs a high-resolution feature map. Then, the feature preservation branch and the downsampling branch are started in parallel; the feature preservation branch directly preserves... The downsampling branch is passed through convolution pairs with a stride of 2. Downsampling is performed to obtain low-resolution feature maps. ;Will After being sized by bilinear interpolation and Enhanced high-resolution features are obtained by stitching together along the channel dimension and integrating them through a convolutional layer. This serves as a feature of skip connections, used for upsampling by subsequent decoders; simultaneously... and The output after fusion is the enhanced low-resolution feature of this stage. .

[0009] Furthermore, the specific steps for enhancing the high-resolution feature map output by the cross-domain collaborative attention unit are as follows: for each stage... High-frequency component extraction: =

[0010] in, This represents a learnable high-pass filter convolution kernel. This indicates the corresponding bias term. This represents the convolution operation. Represents the high-frequency component diagram; A learnable high-pass filter convolution kernel is used to linearly amplify the high-frequency component image, resulting in an amplified high-frequency component image. : in, For the threshold, As the amplification factor, and For trainable parameters, It is a nonlinear amplification function; A heterogeneous dual-path, three-branch attention architecture is adopted, where the dual paths include a low-level detail path and a high-level semantic path; the low-level detail path specifically includes a Key branch for calculating similarity. The Value branch is used to provide the feature content that needs to be weighted. High-level semantic paths specifically include Branches; For the lowest-level Nth stage cross-domain collaborative attention unit: right Upsampling is performed using subpixel convolution to make its spatial size consistent with... Maintain consistency: = PixelShuffle ( ) Will Generated by two independent 3×3 convolutional layers Branch output and Branch output; right Perform a transpose operation to generate Branch output; Perform attention calculations and output enhanced high-resolution feature maps. :

[0011] in This is the scaling factor; For other cross-domain collaborative attention units: right Upsampling is performed using subpixel convolution to make its spatial size consistent with... Maintain consistency: = PixelShuffle ( ) Will Generated by two independent 3×3 convolutional layers Branch output and Branch output; right Perform a transpose operation to generate Branch output; Perform attention calculations and output enhanced high-resolution feature maps. : .

[0012] Furthermore, the specific steps of the local visible region perception and relation reasoning module in enhancing and aggregating local features to output the final detection result include: Receive the enhanced high-resolution feature map and predict local candidate regions present in the image; The local candidate regions are mapped to fixed-size features, and deformable convolution is used for local feature enhancement. The system receives enhanced local features, establishes spatial context relationships between different candidate regions, aggregates features, performs classification and regression, and outputs the final detection results.

[0013] Furthermore, the specific steps for predicting local candidate regions include: predicting each spatial location on the feature map using a region confidence prediction branch. The confidence level of the local target;

[0014] in, For convolution kernel weights, To bias it, The Sigmoid activation function outputs... The confidence score is calculated for each spatial location as a local target; through a regional coordinate regression branch, for each location, four regression parameters representing coordinate offsets are output. , representing the offset of the center point's horizontal and vertical coordinates, and the scaling factors for width and height, respectively. , As a preset constant, For regression weights, Offset it; calculate the center coordinates of the candidate region. and size :

[0015]

[0016]

[0017]

[0018] in, and This is a scale control factor. and The preset base width and height; Based on the confidence level of each candidate region The N regions with the highest confidence levels are selected as local candidate regions. Each of them .

[0019] Furthermore, the steps for classification using deformable convolution include: for each candidate region Extracting fixed-size feature blocks using RoIAlign :

[0020] The output is a fixed size. K×K×C Feature blocks Feature aggregation is performed using deformable convolution formulas. in For regular sampling grid, As weight, The learned offset; the offset is learned from the input features through additional convolutional layers:

[0021] in Includes the (x,y) offset of each sampling point; for the aggregated features Perform global average pooling to obtain the enhanced feature vector. : .

[0022] Furthermore, the classification step includes: dividing the enhanced feature vector... Projected as query vectors respectively Key vector Sum value vector :

[0023]

[0024] in The projection matrix is ​​learnable; Calculate the first The candidate region and the first Attention weights among candidate regions : in, The dimensions after projection; Feature aggregation is performed using attention weights to obtain enhanced aggregated features. :

[0025] Intermediate features are obtained through residual connections and layer normalization: Enhanced features are obtained and output through feedforward networks and residual connections: .

[0026] Furthermore, the regression steps include: transforming the scores into a probability distribution using Softmax. in, Here is the weight matrix of the fully connected layer for the learnable classification branch. The bias term for the classification branch; Determine the bounding box refinement parameters :

[0027] This is the weight matrix for the regression branch. This is the bias term for the regression branch. It includes parameters in four dimensions: ; Fine-tuning the initial candidate regions yields the final bounding boxes. :

[0028]

[0029]

[0030] .

[0031] The beneficial effects of this invention are as follows: This invention innovatively designs a cross-stage feature fusion and cross-domain collaborative attention mechanism in the backbone network. By retaining high-resolution branches and performing real-time interaction during the downsampling stage, the loss of high-frequency details of small targets during continuous downsampling is effectively avoided. The cross-domain collaborative attention module uses high-level semantic features as queries to actively guide and filter high-frequency detail features enhanced by nonlinear amplification, achieving proactive feature reconstruction. This enables the network to retain rich details while possessing strong semantic discriminative power, significantly improving the detection accuracy of small and blurry targets. At the same time, through the local visible region perception and relation reasoning module, even if the target is mostly occluded, the model can still obtain the complete existence of the target through relation reasoning using local features, greatly improving the recall rate in complex scenes. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of a small target local perception and relation reasoning detection system in one embodiment; Figure 2 This is a flowchart illustrating a method for small target local perception and relational reasoning detection in one embodiment; Figure 3 This is a schematic diagram of a cross-stage feature fusion unit in one embodiment; Figure 4 This is a schematic diagram of a cross-domain collaborative attention unit in one embodiment. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] Example 1 like Figure 1As shown, a small target local perception and relation reasoning detection system includes a cascaded target detection backbone network and a local visible region perception and relation reasoning module. The target detection backbone network is used to extract features from the input image, including: a cross-stage feature fusion unit, set in the downsampling stage of the backbone network, used to generate feature retention branches and downsampling branches in parallel at each downsampling stage, and cross-fuse the downsampled features with the retained high-resolution features to output enhanced features; a cross-domain collaborative attention unit, set in the upsampling stage of the backbone network, used to perform attention calculation by combining enhanced features and upsampling features, and output an enhanced high-resolution feature map; the local visible region perception and relation reasoning module is connected to the output end of the target detection backbone network, used to receive the enhanced high-resolution feature map, perform local feature enhancement and aggregation, and output the final detection result.

[0035] In one specific embodiment, the local visible area perception and relation reasoning module specifically includes: A local region proposal network is used to receive the enhanced high-resolution feature map and predict local candidate regions present in the image. The feature extraction and enhancement unit is used to map the local candidate regions to fixed-size features and use deformable convolution to enhance local features; the local relationship reasoning submodule is used to receive the enhanced local features, establish spatial context relationships between different candidate regions and aggregate features, perform classification and regression, and output the final detection result.

[0036] In this embodiment, the object detection backbone network is based on the classic U-Net architecture, an encoder-decoder structure. Its encoder, the downsampling path, uses a series of convolution and pooling operations to gradually reduce the resolution of the input image from its original size to 1 / 4, 1 / 8, 1 / 16, and even 1 / 32, while the corresponding number of feature channels increases to C, 2C, 4C, and 8C, respectively, thereby gradually expanding the receptive field and extracting high-level semantic features. Its decoder, the upsampling path, fuses the upsampling operation with skip connections from the same-scale feature map of the encoder to restore spatial details.

[0037] However, the classic U-Net architecture has inherent limitations when dealing with small target detection in complex agricultural environments, as described in Problem 1. First, during downsampling, pooling or stride convolution, while pursuing high-level semantics, inevitably filters out high-frequency spatial details—precisely crucial for accurately distinguishing small targets from complex backgrounds. Second, in the upsampling path, simple feature map addition and linear interpolation upsampling only achieve linear and passive compensation for spatial information, failing to actively recover the high-frequency details lost during downsampling. This results in the fused feature map still showing blurred outlines of small targets and weak semantic representation.

[0038] In this embodiment, a cross-stage feature fusion mechanism is introduced during the downsampling process. This mechanism does not simply pass features to skip connections, but instead immediately cross-fuses the downsampled features with the high-resolution features from the previous stage after each downsampling stage. Simultaneously, this invention designs an active, non-linear cross-domain collaborative attention module specifically for reconstructing and enhancing the high-frequency details of small targets at the feature level. This compensates for the information loss caused by the inherent network structure and solves the problem that existing multi-scale feature fusion methods (such as FPN) fuse features through upsampling and addition, resulting in blurred details and weak semantics for small targets in the fused feature map.

[0039] Example 2 like Figure 2 As shown, a small target local perception and relation reasoning detection method based on the system includes the following steps: acquiring a target scene image to be detected; inputting the image to be detected into the target detection backbone network; in the downsampling stage, generating a feature retention branch and a downsampling branch in parallel through a cross-stage feature fusion unit; cross-fusioning the downsampled features with the retained high-resolution features to output enhanced features; in the upsampling stage, inputting the enhanced features into a cross-domain collaborative attention unit; performing attention calculation by combining the enhanced features and the upsampling features to output an enhanced high-resolution feature map; inputting the high-resolution feature map into a local visible area perception and relation reasoning module for local feature enhancement and aggregation; and outputting the final detection result.

[0040] In one specific embodiment, such as Figure 3 As shown, the specific method for outputting enhanced features by the cross-stage feature fusion unit is as follows: the image to be detected enters the downsampling stage through convolution and pooling operations; in the downsampling stage, the resolution of the image gradually decreases from the original size, while the corresponding number of feature channels gradually increases; In the Each downsampling stage outputs a high-resolution feature map. Then, the feature preservation branch and the downsampling branch are started in parallel; the feature preservation branch directly preserves... The downsampling branch is passed through convolution pairs with a stride of 2. Downsampling is performed to obtain low-resolution feature maps. ; In this embodiment, using The expanded receptive field and more abstract semantic information have been used to enhance and refine the current high-resolution features. This allows it to retain details while also possessing stronger semantic representation capabilities.

[0041] Will After being sized by bilinear interpolation and Enhanced high-resolution features are obtained by stitching together along the channel dimension and integrating them through a convolutional layer. This serves as a feature of skip connections, used for upsampling by subsequent decoders; simultaneously... and The output after fusion is the enhanced low-resolution feature of this stage. .

[0042] In this embodiment, the inputs to the cross-domain collaborative attention module include: Low-level detailed features: such as feature maps from the third stage It has rich pixel-level details, but large semantic noise, that is, "there may be a lot of things here, but it is not sure what they are"; High-level semantic features: such as feature maps from the fourth stage Each pixel carries strong semantic information within a broad receptive field, enabling reliable determination of "what is here".

[0043] In one specific embodiment, such as Figure 4 As shown, the specific steps for the enhanced high-resolution feature map output by the cross-domain collaborative attention unit in the third stage are as follows: High-frequency component extraction:

[0044] in, This represents a learnable high-pass filter convolution kernel, whose initial value is set to an approximate form of the Laplacian operator. During training, it will be continuously optimized to better adapt to the feature patterns of small objects such as strawberries and blueberries. This indicates the corresponding bias term. This represents a convolution operation whose kernel is initialized as a Laplacian kernel approximation that strengthens the center and weakens the surrounding area (e.g., [[0,-1,0],[-1,4,-1],[0,-1,0]]). During training, this convolution kernel adaptively learns how to... It extracts the high-frequency components most sensitive to small targets; it is no longer a fixed mathematical operator, but a data-driven, task-oriented feature extractor. Its output... It can be understood as a "detail saliency map", where bright points correspond to areas of dramatic change in the original features. These areas are likely the edges or corners of small targets. A learnable high-pass filter convolution kernel is used to linearly amplify the high-frequency component image, resulting in an amplified high-frequency component image. : in, For the threshold, As the amplification factor, and For trainable parameters, It is a nonlinear amplification function. and All of these can be used as network parameters learned during training. For response values ​​exceeding... Strong features (which may correspond to sharp edges or noise of large targets) should be preserved as is to avoid over-amplification that could lead to feature distortion or excessive noise. For response values ​​below... The weak features (which are precisely the small target signals of interest that are on the verge of disappearing) are used It was significantly magnified. It can effectively improve the saliency of small object edges and textures. Upsampling is performed using subpixel convolution to make its spatial size consistent with... Maintain consistency:

[0045] in, For upsampling convolution kernels, PixelShuffle This represents a rearrangement operation from the channel dimension to the spatial dimension, the function of which is to improve resolution; in this embodiment, The spatial dimensions were increased from H / 32×W / 32 to H / 16×W / 16, and the number of channels was adjusted from 8C to 4C. This means that the network can learn how to use the contextual information between channels to better reconstruct the edges and shapes of small targets. The results are clearer and more conducive to the accurate localization of small targets than bilinear interpolation. In this embodiment, two distinct feature representations with their own advantages have been obtained through two independent paths: the features output by the high-frequency excitation path are rich in enhanced detail information but lack sufficient semantic context; the features output by the upsampling path carry high-level semantic information but have relatively blurred spatial details. However, directly using any single feature source to generate the Q, K, and V branches all have significant limitations. If all are generated from high-frequency excitation features, the attention weight calculation will be interfered with by detail noise, making it difficult to accurately grasp global semantic associations; if all are generated from upsampling features, the ability to use detail information for precise localization is lost. Furthermore, the homologous QKV generation method cannot meet the requirement of using semantic information to guide the selection and recombination of detail features because it essentially performs autocorrelation operations within the same feature space and lacks a cross-feature space guidance mechanism.

[0046] In this embodiment, the present invention proposes a heterogeneous dual-path three-branch attention architecture. Its core motivation lies in fully utilizing the advantages of the dual-path features obtained in the previous processing, and constructing an attention computation system with a clear functional orientation through specialized branch division of labor. Specifically, the high-frequency excitation features rich in detail focus on providing the queried content K and the features V to be weighted, while the upsampled features rich in semantics focus on constructing the query instruction Q. This achieves precise guidance and filtering of detailed information by semantic information in the attention computation process.

[0047] Generate the Key branch from two independent 3×3 convolutional layers. and Value branch These two branches carry enhanced details, with the Key branch used to calculate similarity and the Value branch used to provide the feature content that needs to be weighted. right After processing with a 3×3 convolutional layer, a specific transpose operation is performed: ,generate Branch; this transpose operation will transfer the feature from Convert to This changes the way features are organized, making them more suitable as query vectors; Perform attention calculations and output enhanced high-resolution feature maps. :

[0048] in This is the scaling factor; This design elevates the attention mechanism beyond simple feature autocorrelation to a semantic-detail interaction system across the feature space, providing more accurate and robust feature representation capabilities for small object detection. In complex occlusion scenarios, the geometric center of small objects is often invisible and has weak features, but their locally visible parts (such as exposed red fruit tips, white petals, or green sepals) still possess clear and identifiable features on high-resolution feature maps. Therefore, we abandon the traditional strategy of directly regressing the blurred center point and instead utilize and enhance the salient features of these locally visible regions. The input to this module is a high spatial resolution feature map from the backbone network. This feature preserves rich low-level details (such as edges and textures) and incorporates high-level semantic information.

[0049] In one specific embodiment, the steps of the local visible region perception and relation reasoning module in performing local feature enhancement and aggregation to output the final detection result include: Receive the enhanced high-resolution feature map and predict local candidate regions present in the image; The local candidate regions are mapped to fixed-size features, and deformable convolution is used for local feature enhancement. The system receives enhanced local features, establishes spatial context relationships between different candidate regions, aggregates features, performs classification and regression, and outputs the final detection results.

[0050] In one specific embodiment, the steps for predicting local candidate regions include: predicting each spatial location on the feature map using a region confidence prediction branch. The confidence level of the local target;

[0051] in, For convolution kernel weights, For bias, The Sigmoid activation function outputs... The confidence score is calculated for each spatial location as a local target; through a regional coordinate regression branch, for each location, four regression parameters representing coordinate offsets are output. , representing the offset of the center point's horizontal and vertical coordinates, and the scaling factors for width and height, respectively. , As a preset constant, Output ; Calculate the center coordinates of the candidate region and size :

[0052]

[0053]

[0054]

[0055] in, and This is a scale control factor. and The preset base width and height; This means adding the learned x-direction offset to the original column coordinate j; To add the learned y-direction offset to the original column coordinate i; To the base width Exponential scaling is applied to the base model; To be at the base height Exponential scaling is applied to the base model; Based on the confidence level of each candidate region The N regions with the highest confidence levels are selected as local candidate regions. Each of them .

[0056] Furthermore, the steps for classification using deformable convolution include: for each candidate region Extracting fixed-size feature blocks using RoIAlign :

[0057] The output is a fixed size. K×K×C Feature blocks Deformable convolution feature enhancement breaks the fixed geometric structure limitations of traditional convolution, allowing the model to adaptively adjust the shape and position of the receptive field, thereby more accurately focusing on the discriminative local features of irregular, occluded targets. Feature aggregation is performed using the deformable convolution formula. in For regular sampling grid, As weight, The learned offset; the offset is learned from the input features through additional convolutional layers:

[0058] in This includes the (x, y) offset of each sampling point; the enhanced feature vector is obtained through global average pooling, compressing the K×K×C feature map into a D-dimensional vector. The aim is to aggregate the enhanced spatial features extracted by deformable convolution into a compact global semantic representation, significantly reducing subsequent computational complexity while retaining the core discriminative information of each candidate region. : .

[0059] In this embodiment, GAP compresses a K×K feature map into a single value, a process that erases the specific distribution of features within the K×K grid. For example, information such as whether a high-response feature appears in the center or top left corner of the region is lost. It perfectly preserves the overall activation intensity of each channel (feature filter) within the region. If a channel is specifically used to detect "red," then the value after GAP represents "the overall redness of this region." The candidate region R_n = (x_n, y_n, w_n, h_n) processed by the local feature enhancement module has its position and size initially determined by the preceding "region proposal network." The regression branch of the detection head no longer performs "out of thin air" localization but instead fine-tunes this initial bounding box. Therefore, it does not require a pixel-level precise spatial map, but only a powerful feature vector to predict four fine-tuning quantities (δx, δy, δw, δh). This determines whether the region is a "ripe strawberry," an "unripe strawberry," or a "flower." This task relies more on global semantic features (e.g., "Is this area generally red?", "Does it have petal texture?") rather than the specific distribution of features within the region. GAP provides this kind of global semantic generalization. Therefore, as mentioned above, it's fine-tuning rather than coarse localization. A strong feature vector is sufficient to learn how to finely adjust based on the content. The risk of information loss is greatly mitigated by the preceding "deformable convolution." Deformable convolution acts like an intelligent focusing system. It actively shifts the sampling points of the convolution kernel to the most discriminative parts of the region (e.g., the red tip of a strawberry, the edge of the sepals). After this "focusing," the output feature map Y_n is already a highly condensed representation. Key features are enhanced, while irrelevant background and occlusions are weakened. Applying GAP at this point is equivalent to "summarizing" this already refined feature map. Therefore, it's pooling the "essence," not pooling the "raw, messy information."

[0060] In one specific embodiment, the classification step includes: classifying the enhanced feature vector. Projected as query vectors respectively Key vector Sum value vector :

[0061]

[0062] in The projection matrix is ​​learnable; Calculate the first The candidate region and the first Attention weights among candidate regions : in, The dimensions after projection; Feature aggregation is performed using attention weights to obtain enhanced aggregated features. :

[0063] Intermediate features are obtained through residual connections and layer normalization: Enhanced features are obtained and output through feedforward networks and residual connections: .

[0064] Furthermore, the regression steps include: transforming the scores into a probability distribution using Softmax. in, Here is the weight matrix of the fully connected layer for the learnable classification branch. The bias term for the classification branch; Determine the bounding box refinement parameters :

[0065] This is the weight matrix for the regression branch. This is the bias term for the regression branch. It includes parameters in four dimensions: ; Fine-tuning the initial candidate regions yields the final bounding boxes. :

[0066]

[0067]

[0068] .

[0069] Example 3 This embodiment applies the small target local perception and relation reasoning detection method to an intelligent strawberry picking robot. The robot is equipped with a binocular depth camera and an edge computing terminal (such as the NVIDIA Jetson series) to accurately identify the ripeness and location of strawberries in the complex lighting and dense foliage conditions of a greenhouse, guiding the robotic arm to complete the non-destructive harvesting.

[0070] The detection method mainly includes the following steps: The image acquisition and multi-scale feature extraction robot moves between crop rows, and the camera acquires RGB images (images to be detected) containing strawberry plants. The images contain strawberries at different growth stages (small targets), overlapping fruits, and fruits obscured by leaves. The images are then input into a pre-built target detection backbone network.

[0071] During the encoder's downsampling process, to prevent the loss of high-frequency details such as seed texture and edges on the strawberry surface during continuous pooling, the network performs cross-stage feature fusion. For the current stage... Stage, output feature map At this point, the system executes two paths in parallel: Path A: Direct caching Preserve the current high-resolution detail information.

[0072] Path B: via a step size of 2 Convolution pairs Downsampling is performed to obtain a low-resolution feature map. This allows for the extraction of deeper semantic information. Subsequently, cross-fusion is performed: After upsampling and The process involves concatenation, using deep semantics to refine shallow details, and outputting the enhanced features via a skip connection to the decoder; simultaneously, the fused result is passed to the next layer encoder. During the decoder's resolution restoration process, to address the issue of small objects being "unclear," a cross-domain collaborative attention unit is introduced. This unit reconstructs features by semantically guiding details. High-frequency components (edges and corners) are extracted from features from the encoder using an initialized Laplacian approximate convolution kernel. A nonlinear amplification function is then applied. For response values ​​less than the threshold The minute detail is equal to 0.1, multiplied by the magnification factor. Significant enhancements are applied to strong responses; however, they remain unchanged. These features are used to generate the key vector K and value vector V.

[0073] High-level semantic feature generation query source: The features from the previous level of the decoder are generated through sub-pixel convolution. PixelShuffle Upsampling generates features that include global context information, which are used to generate the query vector Q.

[0074] calculate and The similarity weights, and the ... Weighting is applied. The network uses the semantic Q instruction "This looks like a fruit structure" to filter the detail V "The red edge texture here," thus outputting an enhanced high-resolution feature map that is both clear and semantically meaningful. After obtaining the enhanced high-resolution feature map, it is input into the local visible region perception and relation reasoning module. Since many strawberries are obscured by leaves, the system no longer looks for the geometric center of the strawberry, but rather for the "visible part." The local region proposal network scans every spatial location of the feature map. : Determine whether a location, such as the exposed red tip, is part of a strawberry.

[0075] Output offset .

[0076] Using formula After calculating multiple potential local candidate regions, the system selects the one with the highest confidence level. One region was selected as a candidate.

[0077] The system extracts features from the generated candidate regions: Map candidate regions of different sizes to A feature block of a fixed size.

[0078] Considering the irregular shape and varied postures of strawberries, standard convolution is difficult to adapt. The system utilizes deformable convolution to learn the offset of sampling points. The receptive field shape of the convolution kernel is adaptively adjusted to closely follow the edge of the fruit or the direction of the texture, thereby extracting more discriminative feature vectors. .

[0079] In this embodiment, the system does not only independently judge each box, but also considers the relationship between boxes through the local relationship reasoning submodule.

[0080] Constructing a relationship graph: Integrating the features of each candidate region Projections are Q, K, and V; computational domain With the region Attention weights between .

[0081] Region A is the "red fruit tip," and region B is the adjacent "green calyx." Although neither can be seen as complete on its own, the attention mechanism will find that they are spatially close and texturally related, thus assigning them a high association weight.

[0082] Based on the information from the surrounding area aggregated by weights, the features lost due to occlusion in the current area are supplemented to obtain the enhanced aggregated features. .

[0083] based on Determine the target category (e.g., "ripe strawberry" or "unripe strawberry").

[0084] Output the final bounding box. The robotic arm's end effector is guided to precisely position itself at the strawberry stem for cutting.

Claims

1. A small object local perception and relationship reasoning detection system, comprising: The target detection backbone network and the local visible region perception and relationship reasoning module are cascaded. The target detection backbone network is configured to perform feature extraction on an input image, and includes: a cross-stage feature fusion unit arranged at a down-sampling stage of the backbone network, configured to generate a feature reservation branch and a down-sampling branch in parallel at each down-sampling stage, cross-fuse the down-sampled features and the reserved high-resolution features, and output enhanced features; and a cross-domain collaborative attention unit arranged at an up-sampling stage of the backbone network, configured to perform attention calculation on the enhanced features and up-sampled features, and output an enhanced high-resolution feature map; and the local visible region perception and relationship reasoning module is connected to an output end of the target detection backbone network, configured to receive the enhanced high-resolution feature map, perform local feature enhancement and aggregation, and output a final detection result.

2. The system of claim 1, wherein, The local visible region perception and relationship reasoning module specifically includes: a local region proposal network configured to receive the enhanced high-resolution feature map, and predict local candidate regions existing in the image; a feature extraction and enhancement unit configured to map the local candidate regions into fixed-size features, and perform local feature enhancement by using deformable convolution; and a local relationship reasoning sub-module configured to receive the enhanced local features, establish spatial context relationships between different candidate regions, aggregate the features, perform classification and regression, and output the final detection result.

3. A method for detecting small target local perception and relationship reasoning based on the system, characterized in that, The method includes the following steps: collecting a to-be-detected image of a target scene, inputting the to-be-detected image into the target detection backbone network, at a down-sampling stage, generating a feature reservation branch and a down-sampling branch in parallel by using the cross-stage feature fusion unit, cross-fusing the down-sampled features and the reserved high-resolution features, and outputting enhanced features; at an up-sampling stage, inputting the enhanced features into the cross-domain collaborative attention unit, performing attention calculation on the enhanced features and up-sampled features, and outputting an enhanced high-resolution feature map; and inputting the high-resolution feature map into the local visible region perception and relationship reasoning module, performing local feature enhancement and aggregation, and outputting a final detection result.

4. The method of claim 3, wherein, The cross-stage feature fusion unit outputs enhanced features in the following manner: the to-be-detected image enters a down-sampling stage through convolution and pooling operations; at the down-sampling stage, the resolution of the image gradually decreases from the original size, and the number of corresponding feature channels gradually increases; In the first high-resolution feature map is output in the downsampling stage The feature reservation branch and the downsampling branch are started in parallel after the high-resolution feature map is output; the feature reservation branch directly reserves ; The down-sampling branch down-samples the high-resolution feature map by a convolution with a step of 2 to obtain a low-resolution feature map ; and adjusts the size of the low-resolution feature map by bilinear interpolation , and concatenates the low-resolution feature map and the high-resolution feature map in the channel dimension, and integrates the two feature maps by a convolution layer to obtain an enhanced high-resolution feature as a feature of the skip connection for use in subsequent up-sampling by the decoder; and fuses the enhanced high-resolution feature and the low-resolution feature to output an enhanced low-resolution feature of the current stage .​​​​ 5. The method of claim 3, wherein, The cross-domain collaborative attention unit outputs an enhanced high-resolution feature map in the following manner: for each stage performing high frequency component extraction: = wherein, denotes a learnable high-pass filtering convolution kernel, denotes a corresponding bias term, denotes a convolution operation, denotes a high-frequency component map; The high-frequency component graph is linearly enlarged by using a learnable high-pass filtering convolution kernel to obtain an enlarged high-frequency component graph : wherein, is a threshold value, is an amplification factor, and is a trainable parameter, is a non-linear amplification function; Adopt a three-branch attention architecture of a heterologous double path, wherein the double path includes a bottom layer detail path and a high layer semantic path; wherein the bottom layer detail path specifically includes a Key branch for calculating similarity , a Value branch for providing feature content that needs to be weighted ; the high layer semantic path specifically includes branches; For the cross-domain collaborative attention unit of the bottommost Nth stage: To Apply sub-pixel convolution for up-sampling to make its spatial size consistent with the original image: = PixelShuffle （ ） will be described below. are generated by two independent 3x3 convolutional layers, respectively the outputs of the branches and the outputs of the branches; the output of the branch;​​ performing attention computation, outputting enhanced high-resolution feature maps : wherein is a scaling factor; For other cross-domain collaborative attention units: To Apply sub-pixel convolution for up-sampling to make its spatial size consistent with the original image: = PixelShuffle （ ） The outputs of the branches are combined to generate the output of the network. The outputs of the branches are combined to generate the output of the network. The outputs of the branches are combined to generate the output of the network. The outputs of the branches are combined to generate the output of the network. the transpose operation, generating​​ performing attention computation, outputting enhanced high-resolution feature maps : 。 6. The method of claim 3, wherein, The local visible region perception and relationship reasoning module performs local feature enhancement and aggregation in the following manner to output a final detection result: receiving the enhanced high-resolution feature map, and predicting local candidate regions existing in the image; mapping the local candidate regions into fixed-size features, and performing local feature enhancement by using deformable convolution; The enhanced local features are received, spatial context relationships between different candidate regions are established and features are aggregated, classification and regression are performed, and final detection results are output.

7. The method of claim 5, wherein, The specific steps for predicting local candidate regions include: predicting each spatial location on the feature map using a region confidence prediction branch. The confidence level of the local target; wherein, is a convolution kernel weight, is a bias thereof, is a Sigmoid activation function, output is a confidence of each spatial position being a local target; through the region coordinate regression branch, for each position, 4 regression parameters representing coordinate offset are output , respectively representing the offset of the horizontal and vertical coordinates of the center point and the scaling factor of the width and height, wherein , is a preset constant, is a regression weight, is a bias thereof; the center coordinates of the candidate region and the size : wherein, and is a scale control factor, and is a preset base width-height; based on the confidence of each candidate region , select the N regions with the highest confidence as local candidate regions where each .

8. The method of claim 7, wherein, The step of classifying using deformable convolution includes: for each candidate region extracting a fixed-size feature block by RoIAlign : where the output is of fixed size K x K x C of the feature blocks feature aggregation with deformable convolution formula: where is a regular sampling grid used to compress the picture into one value; is a weight, is a learned offset; the offset is learned from the input features by an additional convolutional layer: wherein includes an (x, y) offset for each sample point; on the polymerized features performing global average pooling to obtain the enhanced feature vector : .

9. The method of claim 8, wherein, The step of classifying comprises projecting the enhanced feature vector as a query vector , a key vector and a value vector respectively: wherein is a learnable projection matrix; computing attention weights between the first candidate region and the second candidate region : wherein, is the dimension after projection; The enhanced aggregated features are obtained by completing feature aggregation through attention weights : An intermediate feature is obtained by residual connection and layer normalization processing: An enhanced feature is obtained and output by feedforward network and residual connection processing: .

10. The method of claim 9, wherein, The step of regression includes converting scores into probability distributions by Softmax: wherein, is a fully connected layer weight matrix of the learnable classification branch, is a bias term of the classification branch; Obtaining bounding box refinement parameters : a weight matrix for the regression branch, a bias term for the regression branch, including parameters of 4 dimensions: ; Fine-tuning the initial candidate regions to obtain final bounding boxes : 。

Images