A small target detection method against motion blur

Abstract

The application discloses a small target detection method for resisting motion blur, and a target detection model based on an improved YOLOv8n is constructed; the target detection model comprises a backbone network, a neck network and a detection end connected in sequence, a second C2f module of the backbone network is replaced by a LEG module, and an output feature of splicing the LEG module is replaced for splicing middle features in a bottom-up fusion path of the neck network; the LEG module splits input features into a semantic branch and an edge branch, the edge branch extracts a gradient graph from features, weights of pooling are calculated from the gradient graph through a normalization function, and features of the semantic branch are modulated according to the weights; a to-be-detected image acquired in real time is input into the trained target detection model, and a boundary box of a target is output. The YOLOv8n model is optimized and improved, the LEG module is introduced, and blurred features are filtered out, so that the model has strong anti-blurring capability and solves the problem of serious small target feature loss.

Description

Technical Field

[0001] This invention relates to robot visual localization, specifically to a method for detecting small targets that resists motion blur. Background Technology

[0002] Visual grasping systems are widely used in industrial automation grasping and flexible manufacturing, with the "eye-in-hand" configuration being particularly favored due to its high flexibility and minimal occlusion. In such systems, the accuracy and inference speed of the embedded target detection algorithm directly determine the stability of the closed-loop control and the success or failure of the grasping process. Currently, while single-stage detection algorithms, represented by the YOLO series, are widely used, they still face the following significant challenges in practical dynamic grasping and edge computing scenarios:

[0003] First, it has weak resistance to motion blur. During visual grasping, the camera rapidly approaches the target along with the end effector, inevitably causing image motion blur. Existing convolutional neural networks typically rely on clear gradient textures for feature extraction. In blurred scenes, high-frequency information at the target's edges is smoothed, making it difficult for the network to distinguish between foreground and background. This easily leads to missed detections or positioning drift, severely affecting the success rate of grasping at high speeds.

[0004] Second, there is a contradiction between lightweight design and the preservation of small target features. Limited computing power of edge computing devices (Raspberry Pi) in industrial settings necessitates the use of lightweight models. However, to reduce computation, lightweight models typically employ large-step downsampling and channel compression. This process leads to a significant loss of subtle geometric features of small targets in deep feature maps. While existing Feature Pyramid Network (FPN) structures can fuse multi-scale information, in extremely compressed models, they still struggle to retain sufficient pixel-level edge location information to support precise grasping tasks.

[0005] Third, there is a lack of adaptive computing power mechanisms. Most existing deep learning models employ static computation graph structures, meaning that regardless of whether the input image is a clear target against a simple background or a blurred target under complex occlusion, a fixed full inference process is executed. In the visual grasping control loop, this "one-size-fits-all" mechanism leads to a huge waste of computational resources. It cannot achieve a higher control frame rate by reducing computing power in simple scenarios, causing system latency fluctuations and affecting the smoothness of closed-loop control.

[0006] Fourth, the grasping strategy lacks geometric awareness and pose optimization. Existing general object detection algorithms typically only output axis-aligned bounding boxes, lacking precise awareness of the object's geometric principal axis direction. When facing non-isotropic objects such as cuboids and cylinders, if the control system simply uses the geometric center as the grasping point without adjusting the gripper angle to be perpendicular to the object's long axis (i.e., grasping along the short side), the object is very likely to slip due to excessive lever arm or insufficient contact area.

[0007] However, existing attention mechanisms (such as SE and Coordinate Attention) generally employ global average pooling to aggregate spatial information. In motion-blurred scenes, pixel values ​​in blurred regions still exist and have high energy. Average pooling operations mix these 'blurry noises' with 'sharp textures,' causing attention weights to fail to distinguish between the background and the blurred target. Furthermore, traditional edge detection operators (such as Sobel) have only a fixed receptive field, making it difficult to adapt to the elongated trailing shadows generated as the robotic arm changes speed from low to high, resulting in poor robustness of feature extraction at different motion speeds. Summary of the Invention

[0008] Purpose of the invention: To address the above-mentioned shortcomings, this invention provides a highly accurate method for detecting small targets that resists motion blur.

[0009] Technical solution: To solve the above problems, this invention employs a small target detection method resistant to motion blur, comprising the following steps:

[0010] Step 1: Obtain the target detection dataset enhanced with motion blur simulation;

[0011] Step 2: Construct an object detection model based on the improved YOLOv8n; the object detection model includes a backbone network, a neck network, and a detection end connected in sequence. The backbone network is used to extract features, and the extracted features are input into the neck network for aggregation. The aggregated features are then identified and labeled at the detection end; the second C2f module of the backbone network is replaced with a local edge-guided LEG module, and the intermediate features spliced ​​in the bottom-up fusion path of the neck network are replaced with the output features spliced ​​by the LEG module through horizontal connection;

[0012] The LEG module splits the input features into semantic branches and edge branches. The edge branches extract gradients from the features to obtain gradient maps. Based on the gradient maps, the pooling weights are calculated using a normalization function. The features of the semantic branches are modulated according to the weights, and the modulated features are output.

[0013] Step 3: Train the object detection model based on the acquired dataset to obtain the trained object detection model;

[0014] Step 4: Input the real-time acquired image to be detected into the trained target detection model, and output the bounding box of the target.

[0015] Furthermore, the image representation enhanced by motion blur simulation is as follows:

[0016] ;

[0017] in, This represents the generated simulated blurred image; Represents the original, clear image; Represents the linear motion fuzzy kernel; This represents the convolution operator; This represents the Gaussian noise matrix.

[0018] Furthermore, it also includes multi-scale transformation enhancement of the object detection dataset by scaling the images in the dataset using a random scaling factor.

[0019] Furthermore, multi-scale expanded difference is used to extract gradients from the features, specifically: for the basic difference operator Apply different expansion rates Perform differential convolution to obtain the gradient map. :

[0020] ;

[0021] in, As input features, , It is a constant. This represents convolution.

[0022] Furthermore, horizontal and vertical difference operators are used to extract horizontal and vertical gradients from the features, resulting in horizontal and vertical gradient maps. Weights are calculated based on the gradient maps using a normalization function, specifically the Softmax function. The weights include horizontal and vertical weights. Horizontal features are reconstructed using the horizontal weights, and vertical features are reconstructed using the vertical weights. Attention coefficients are generated from the reconstructed horizontal and vertical features, and these attention coefficients are used to modulate the features of the semantic branches.

[0023] Furthermore, the ratio of total gradient energy is calculated using the horizontal and vertical gradient maps. This ratio is then used to generate a correction coefficient to adjust the attention coefficient. The adjusted attention coefficient is then used to modulate the features of the semantic branch, resulting in the following modulated features:

[0024] ;

[0025] in, For the corrected attention coefficient, This is an element-wise multiplication operation;

[0026] ;

[0027] in, For activation function, Indicates feature splicing, For horizontal reconstruction features, For vertical reconstruction features, For multilayer perceptrons, This represents the ratio of total gradient energy.

[0028] ;

[0029] in, This is a horizontal gradient plot. This is a vertical gradient plot. To prevent constants with a denominator of zero.

[0030] Furthermore, the detection end includes a prior detection head and a main detection head. The input of the prior detection head is the P3 layer fusion feature output by the neck network. The prior detection head includes a layer... A convolutional layer and a sigmoid activation function generate a confidence score based on the detection results of the prior detection head. If the confidence score meets the requirements, the detection result of the prior detection head is output; if the confidence score does not meet the requirements, the main detection head is activated. The input of the main detection head is the fused features of layers P3, P4, and P5 output by the neck network. The main detection head adopts a decoupled head structure, processing through parallel classification and regression branches. The classification branch outputs the class probability of the target, and the regression branch outputs the bounding box position, size, and angle.

[0031] Furthermore, when training the target detection model, no confidence score is performed on the detection end. Simultaneously, the main detection head and the prior detection head are trained. A self-distillation loss function is introduced for the detection end. The output of the main detection head is used as the teacher signal, and the output of the prior detection head is used as the student signal. The KL divergence loss between the output probability distributions of the main detection head and the prior detection head is calculated so that the prior detection head can imitate the feature expression of the main detection head as much as possible.

[0032] Furthermore, all C2f modules of the neck network are replaced with GS modules, and all convolutional layers of the neck network are replaced with Ghost convolutions. The Ghost convolution includes a channel shuffling module. The GS module includes a convolutional layer, a linear transformation module, a concatenation module, and a channel shuffling module. The convolutional layer is used to perform convolution operations on the input of the GS module, and the number of convolutional kernels in the convolutional layer is half that of the ordinary convolutional layer. The linear transformation module is used to perform a cheap operation on the output of the convolutional layer, which is to perform a linear transformation on the feature map of each channel one by one. The concatenation module is used to concatenate the output of the convolutional layer and the output of the linear transformation module. The channel shuffling module is used to perform a channel shuffling operation on the output of the concatenation module, and the output of the channel shuffling module is the final output of the CS-Ghost module.

[0033] Furthermore, based on the output target bounding box, the center coordinates, long side, short side, and rotation angle of the target are obtained. The aspect ratio is calculated based on the long and short sides of the target, and the gripping angle of the robotic arm is controlled based on the aspect ratio.

[0034] Beneficial Effects: Compared with existing technologies, the significant advantages of this invention are: it fully considers the characteristics of drastic changes in motion blur and large differences in scene complexity during the dynamic grasping process of the robotic arm; it optimizes and improves the YOLOv8n model by introducing the LEG module, which constructs gradient-based weights so that the weights in regions where gradients vanish due to motion blur approach 0, thus filtering out the blurry features in these regions and making the model more robust to blurring; the LEG module directly feeds the extracted "pure high-frequency gradients" to the deep network through lateral connections for cross-layer detail recovery, solving the problem of severe feature loss in small targets. Furthermore, it introduces GS-PANet and Dynamic-Head modules into the YOLOv8n model, utilizing a dynamic cascaded routing strategy based on fuzzy confidence to perform real-time hierarchical scheduling of inference paths; this makes the model lightweight and solves the problem of existing models' inability to adaptively adjust inference computing power; it also introduces a geometrically adaptive grasping strategy, significantly improving the success rate and robustness of physical grasping. This invention effectively solves the problems of missed detections, low accuracy in small target localization, and the difficulty in balancing real-time performance and accuracy in visual grasping processes caused by motion blur. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall control process of a robotic arm using the target detection method of the present invention.

[0036] Figure 2 This is a schematic diagram of the network structure of the target detection model in this invention.

[0037] Figure 3 This is a schematic diagram of the network structure of the local edge guidance module in this invention.

[0038] Figure 4 This is a schematic diagram of the Ghost-Shuffle path aggregation network in this invention.

[0039] Figure 5 This is a schematic diagram of the inference routing logic of the dynamic cascaded detection head in this invention.

[0040] Figure 6 This is a comparison chart of the detection performance of the method of the present invention and the traditional YOLOv8n model in blurred scenes. Detailed Implementation

[0041] like Figure 1 and Figure 2 As shown in the figure, a small target detection method with resistance to motion blur in this embodiment includes the following steps:

[0042] S1: Acquire image data captured by the end-effector camera of the robotic arm, construct a dataset containing target detection in multiple scene conditions, and perform specific anti-blurring and multi-scale enhancement processing on the dataset;

[0043] S2: Construct an object detection model based on an improved YOLOv8n. The object detection model includes a backbone network module, a neck network module, and a detection head module connected in sequence; the second C2f module of the backbone network is replaced with a local edge-guided LEG module, and the intermediate features spliced ​​in the bottom-up fusion path of the neck network are replaced with the output features spliced ​​by the LEG module through horizontal connections;

[0044] The LEG module splits the input features into a semantic branch and an edge branch. The edge branch extracts gradients from the features to obtain a gradient map. The gradient map is used as a weight to generate sharpness confidence. The semantic branch is modulated according to the weights, and the modulated features are output.

[0045] S3: Input the training set and annotation files from the acquired dataset into the target detection model, and perform multi-stage training using an adaptive distillation temperature adjustment strategy and a dynamic loss weighting strategy to optimize the model parameters and obtain a trained target detection model.

[0046] S4: Input the real-time acquired image of the target to be captured into the trained target detection model, perform adaptive inference through the dynamic cascaded detection head, and output the target's category, confidence level, bounding box position and geometric feature information;

[0047] S5: Generates an adaptive grasping strategy based on the geometric feature information output by the target detection model, and calculates the visual grasping control law by combining position deviation and area deviation to drive the robotic arm to perform the grasping task.

[0048] Step S1 in this embodiment specifically includes:

[0049] S11: Collect real-time video streams of the robotic arm at different movement speeds (0-0.5m / s) and under different lighting conditions, filter key frames using the frame difference method, and manually label the target category, bounding box, and rotation angle;

[0050] To ensure the dataset covers the complex working conditions of actual industrial sites, this embodiment employs a multi-dimensional acquisition scheme. The experimental platform uses an industrial depth camera mounted on the end effector of a six-DOF robotic arm, recording at three typical approximation speeds of 0.1 m / s, 0.3 m / s, and 0.5 m / s. Lighting conditions include strong light (direct light), weak light (ambient occlusion), and dynamic shadows. To avoid a large number of static or similar redundant frames in the dataset, a frame difference method based on pixel changes is used for filtering: calculating the current frame... Compared to the previous keyframe The sum of the absolute values ​​of grayscale differences When this value exceeds a preset threshold, the current frame is determined to have a significant pose change and is retained. LabelMe and other tools are used for annotation, not only annotating standard horizontal bounding boxes (HBB) but also rotating bounding boxes (OBB) specifically for the grasping task. The recording format is as follows: ,in The coordinates are centered, and w and h are the width and height, This represents the tilt angle of the object's principal axis relative to the horizontal axis of the image, providing a ground truth value for subsequent grasping pose planning.

[0051] S12: Motion blur simulation enhancement is performed on the dataset. A blur kernel with a kernel size of k is used to perform convolution operation on the clear image to simulate blur degradation under high-speed motion, forcing the model to learn anti-blur features.

[0052] During the final approach phase of visual grasping, the angular and linear velocities of the camera relative to the object increase sharply, leading to severe motion blur. To enable the model to adapt to this visual degradation during training, this invention applies directional motion blur synthesis to the sharp image. A linear motion blur kernel is defined. Its size (Kernel Size) Follows uniform distribution motion angle obey Blurry image The generation process can be mathematically modeled as a clear image. Convolution with a blur kernel, and superimposed with Gaussian noise. , expressed as:

[0053]

[0054] In the formula, This represents the generated simulated blurred image; Represents the original, clear image; Represents the linear motion fuzzy kernel; This represents the convolution operator; This represents the Gaussian noise matrix used to simulate sensor thermal noise. This enhancement strategy not only increases the diversity of samples, but more importantly, it changes the frequency domain distribution of the image, suppressing high-frequency texture information. This forces subsequent network models (especially the LEG module) to find and utilize the strong gradient edge features that remain in the blurred state, rather than relying on clear surface textures, thus significantly improving the robustness of the model in dynamic environments.

[0055] S13: Perform multi-scale transformation enhancement on the dataset by scaling the images with random scaling factors to increase the model's adaptability to targets at different distances.

[0056] Considering that the size of the target in the image changes drastically with decreasing distance during the robotic arm's grasping process (from a tiny target at a distance to filling the field of view at close range), this embodiment introduces a randomized multi-scale training strategy. A random scaling factor S is set, with a value ranging from [0.5, 1.5]. When S < 1, the image is scaled down to simulate distant or small target scenarios, forcing the model to focus on minute features; when S > 1, the image is scaled up and randomly cropped to simulate large targets at close range or partially occluded scenarios. Furthermore, Mosaic data augmentation is employed, stitching four images transformed with different methods into a single image for training. This significantly enriches the background complexity and target distribution characteristics, effectively preventing model overfitting.

[0057] S14: Divide the enhanced dataset into training, validation, and test sets in an 8:1:1 ratio and generate the corresponding configuration files.

[0058] The final dataset is rigorously partitioned and a YAML configuration file conforming to the YOLO format standard is generated, defining the training path, category names and quantities to ensure the standardization of the model training process.

[0059] like Figure 2 As shown, step S2 in this embodiment specifically includes:

[0060] S21: Build an improved YOLOv8n backbone network.

[0061] In the P3 layer of the backbone network (shallow high-resolution features, Stride=8), a Local Edge Guiding Module (LEG) is introduced. This means replacing the second C2f module of the backbone network with the LEG module, and replacing the intermediate features spliced ​​in the bottom-up fusion path of the neck network with the output features spliced ​​by the LEG module through horizontal connections.

[0062] The LEG module produces two outputs: one is the enhanced semantic features of layer P3, which are then passed down to deeper layers of the backbone network (P4, P5); the other is the uncompressed, clean edge features. The data is directly transmitted to the subsequent GS-PANet via a LateralConnection for cross-layer recovery details.

[0063] The LEG module splits the input features into a "semantic branch" and a "marginal branch". The marginal branch uses multi-scale diffraction to extract gradient maps and generates sharpness weights through DGCA units to modulate the semantic features.

[0064] S22: Construct a Ghost-Shuffle path aggregation network (GS-PANet) that accepts cross-layer injection.

[0065] This network, acting as the Neck, is responsible for aggregating multi-scale features from layers P3, P4, and P5 of the backbone network. In the bottom-up fusion path, the received intermediate features are replaced by the received edge features transmitted across layers from the LEG module. ).Will After downsampling and alignment, the features are concatenated with those of the current layer. This design allows GS-PANet to utilize the Ghost module for lightweight feature reconstruction (generating phantom features) while simultaneously using injected edge information to physically compensate for the loss of high-frequency information caused by linear transformation, outputting a fused feature map that combines semantic depth and edge precision.

[0066] S23: Construct a dynamic-head cascaded detection head based on confidence gating.

[0067] This module receives the P3, P4, and P5 three-layer fused features from the output of GS-PANet.

[0068] Cascaded Processing Flow: Prior Judgment: The P3 layer fused features are first fed into a lightweight Nano-OBB Head detection head. This detection head quickly calculates the maximum class confidence (Score) for the current frame. Routing Distribution: The system compares this Score with a preset threshold. Perform a comparison. If... (In low-ambiguity / simple scenarios), the Nano-Head output is directly used as the final result, blocking subsequent data flow and ending the inference process. If (In highly blurred / complex scenes), the data stream is activated and passed down. P4 and P5 features are fed into the high-precision Main-OBB Head and combined with P3 features for full inference. Output: The final output contains rotated bounding box information (x, y, w, h, θ), which is then passed to the visual servoing control module to calculate the Jacobian matrix.

[0069] like Figure 3 As shown, the operation of the Local Edge Guidance Module (LEG) in this embodiment specifically includes:

[0070] S211: Split the input feature map into semantic branches and edge branches;

[0071] Let the input feature map be .pass Convolution halves the number of channels and generates semantic branches separately. and edge branches .

[0072] S212: In the edge branches, multi-scale dilated difference is used to extract gradient priors under variable-length motion blur.

[0073] Considering that changes in the robotic arm's motion speed lead to variations in the blur kernel size, a single-scale Sobel operator struggles to simultaneously capture minute jitter and high-speed motion blur. This embodiment constructs a dual-path extended differential extractor:

[0074] Let the input features be Define the basic differential convolution kernel. (As with Sobel initialization), the horizontal difference operator is: The vertical difference operator is Applying the dilation rate respectively... and Differential convolution, dilation rate Used to capture long-distance motion blur, calculate multi-scale gradient magnitude maps, including horizontal and vertical gradient maps:

[0075]

[0076]

[0077] In the formula, These represent the horizontal and vertical gradient maps, respectively, which contain multi-scale edge information. The output of this step is... It incorporates edge responses at different scales, serving as the "sharpness discrimination criterion" for subsequent DGCA units. Dilated convolution is introduced to match the "long trailing shadow" blur generated by high-speed motion, outperforming the conventional Sobel.

[0078] S213: Constructing a gradient signal-to-noise ratio weighted anisotropic correction unit (DGCA)

[0079] In this embodiment, DGCA no longer uses conventional global average pooling, but proposes a nonlinear aggregation mechanism based on gradient signal-to-noise ratio (GSNR). Its core logic is to use multi-scale difference operators to perceive the degree of motion blur and dynamically determine the pooling weights according to the "sharpness" of the features, thereby physically filtering out motion blur noise during the feature aggregation stage.

[0080] Gradient-Weighted Sharpness Pooling (GWSP)

[0081] Traditional coordinate attention (CA) uses average pooling: This can lead to fuzzy noise (low-frequency, high-energy) contamination of the features. This embodiment proposes utilizing gradient maps. As the "sharpness confidence level", weighted moment aggregation is performed. For the first... The passage at height Horizontal aggregation features The calculation formula is:

[0082] (Gradient-based Softmax weights)

[0083]

[0084] In the formula, This represents the temperature coefficient. Physical meaning: This formula no longer calculates the "average feature," but rather the "weighted sum of the sharpest features." If a certain region experiences gradient changes due to motion blur... Disappearance, its weight The value will approach 0, and the fuzzy features of that region will not be included. This mathematically eliminates the propagation of fuzzy noise, constituting a fundamental difference from standard CA. The calculation for the width direction is similar and will not be elaborated upon here.

[0085] Horizontal reconstructed features are obtained after feature reconstruction. and vertical reconstruction features Subsequently, unlike CA which directly generates attention coefficients, this embodiment introduces an anisotropic correction term. The ratio of the total gradient energy in the X and Y directions is calculated. :

[0086]

[0087] like This indicates severe ambiguity (gradient loss) in the X direction. Using this ratio, a correction coefficient is generated to perform nonlinear suppression on the generated attention map.

[0088]

[0089] Final output .

[0090] All C2f modules in the neck network are replaced with GS modules, and all convolutional layers in the neck network are replaced with Ghost convolutions. The Ghost convolutions include a channel shuffling module, forming a Ghost-Shuffle path aggregation network. The GS module includes a Ghost module and a channel shuffling module. The Ghost module includes convolutional layers, a linear transformation module, and a concatenation module, as shown below. Figure 4 As shown, the operation of the Ghost-Shuffle path aggregation network (GS-PANet) in this embodiment specifically includes:

[0091] S221: The Ghost module generates phantom feature maps through linear transformation, significantly reducing computational cost. The Ghost module assumes that there is a large amount of redundancy (Ghosts) in the feature maps. It first uses a small number of convolutional kernels (e.g., half the number of kernels in a regular convolutional layer). Generate intrinsic feature maps from 1, and then perform inexpensive operations on these feature maps, applying simple linear transformations (such as...). Depthwise convolution generates another Each ghost feature map. This two-step method, compared to directly generating... Each feature map has a theoretical speedup of nearly 2 times.

[0092] S222: Perform Channel Shuffle to break down the barrier between intrinsic and phantom features;

[0093] Because the first and second steps of the Ghost module are independent, there is a lack of information exchange between the output channels. The channel shuffling operation, through the process of reshape->transpose->flatten, evenly shuffles and interweaves the intrinsic feature channels and phantom feature channels, allowing the next layer of the network to utilize information from both types of features simultaneously, significantly improving the expressive power of the lightweight network.

[0094] S223: Perform cross-layer edge injection: After downsampling and aligning the high-resolution edge features output by the LEG module, they are spliced ​​into the bottom-up fusion path of the deep layer of the neck network to restore the geometric edge information of small targets.

[0095] In the bottom-up (PAN) path of GS-PANet, after the feature maps are upsampled and fused from P3 and then convolved to generate P4, an additional lateral connection is introduced: the output of the LEG module... The feature map (containing the most original and clearest edge information) is extracted and downsampled to the same size as P4 through max pooling (Stride=2). Then, it is concatenated with the feature map of the current P4 layer. This mechanism effectively compensates for the high-frequency details lost by deep networks due to multiple downsampling and Ghost module transformations, which is crucial for locating small parts commonly encountered in grasping tasks.

[0096] The Ghost module generates feature maps through linear operations, which manifests as attenuation of high-frequency information in the frequency domain. Using the Ghost module alone can lead to blurred edges on small objects. By injecting edges across layers, the "pure high-frequency gradients" extracted by the LEG module are directly fed into the deeper network, compensating for the spectral loss caused by the Ghost module. This is not a simple feature fusion, but rather a "frequency domain complementarity between low-frequency semantics (Ghost) and high-frequency edges (LEG)."

[0097] like Figure 5 As shown, the specific structure of the dynamic-head in this embodiment includes:

[0098] S231: Lightweight design of the prior detection head (Nano-OBB Head); the Nano-OBB Head is directly mounted on the P3 layer of the neck network (resolution). It contains only one layer inside. The design employs a convolutional layer and a sigmoid activation function. The aim is to quickly extract saliency information from the current frame with minimal computational cost (FLOPs less than 5% of the total model size) to generate confidence scores. This serves as the basis for subsequent routing decisions.

[0099] S232: Decoupling and Rotation Regression Reconstruction of the Main-OBB Head; The Main-OBB Head adopts a decoupled head structure, and for each layer of input features (P3, P4, P5), it is processed through two parallel convolutional branches respectively:

[0100] Classification Branch (Cls Branch): Outputs the probability of the target's class;

[0101] Regression Branch: In traditional YOLO output Based on this, an angle prediction channel was added at the end of the regression branch, making its output dimension become ,Right now This angle Using the long-side definition method, the value range is normalized to... This directly serves the subsequent grasping pose planning.

[0102] Step S3 in this embodiment specifically includes:

[0103] S31: Initialization configuration: Load the pre-trained weights as initial parameters, and use the K-means clustering algorithm to regenerate Anchors (anchor boxes) according to the target size of this dataset.

[0104] Since the targets grasped by robotic arms are typically small in size and have large aspect ratio differences (such as long screws), the preset anchor boxes in the COCO dataset are not suitable. This invention performs K-means clustering analysis on the ground truth boxes in the training set to regenerate 9 sets of anchor sizes adapted to this task, thereby accelerating the regression convergence speed.

[0105] S32: Multi-stage training and dynamic loss weighting: Iterative training is performed using a stochastic gradient descent (SGD) optimizer. Dynamic loss weights are introduced: classification loss is given higher weights in the early stages of training to accelerate convergence, while CIoU bounding box regression loss is weighted more heavily in the later stages to improve localization accuracy; the total loss function is defined as:

[0106]

[0107] In the formula, This represents the total loss value; This represents the bounding box regression loss, which includes angle regression. It adds an angle regression loss term to the CIoU loss to supervise the model's rotation angle of the target. Learning; This represents the binary cross-entropy classification loss; Indicates the distribution focus loss; and They represent the number of training rounds respectively. The weighting coefficients of the regression loss and the classification loss; The fixed weighting coefficient represents the distribution focus loss.

[0108] Introducing a feature that varies with the number of training epochs. Changing dynamic weighting factors:

[0109] In the early stages ( ), set larger This enables the network to quickly learn to distinguish between foreground and background, avoiding getting trapped in local optima;

[0110] In the later period ( ), linearly increasing Furthermore, a CIoU loss is introduced, which simultaneously considers the overlapping area, center point distance, and aspect ratio, enabling the predicted bounding box to closely approximate the real bounding box and meet the requirements of precise grasping.

[0111] Self-Distillation Strategy: For Dynamic-Head structures, a self-distillation loss function is introduced. The output of the deep main OBB head is used as the "teacher signal," and the output of the shallow prior head (Nano-OBB head) is used as the "student signal." The KL divergence loss between their feature maps is calculated, forcing the shallow head to mimic the feature representation of the deep head as much as possible under limited parameters, thereby improving the accuracy of early stopping detection.

[0112] In multi-exit networks, shallow Nano-OBB Heads, due to their shallow layer count and limited receptive field, often lack sufficient feature representation capabilities and are prone to misclassification (misclassifying complex samples as simple samples and prematurely outputting them). This embodiment utilizes the concept of knowledge distillation, not truncating gradients during the training phase, but instead calculating the Kullback-Leibler (KL) divergence between the output probability distributions of the master detector head and the prior detector head:

[0113]

[0114] In the formula, This indicates loss due to self-distillation; Indicates a category index; This indicates the output of the main detection head (Main-OBBHead) regarding the first... The predicted probability distribution of the class; This indicates that the output of the prior detection head (Nano-OBB Head) is about the first... The predicted probability distribution of the class.

[0115] Before computation, the output feature map of the main-OBB head is upsampled by bilinear interpolation to make its spatial resolution consistent with that of the prior head (Nano-OBB head), thereby ensuring that the two perform pixel-level knowledge transfer on the same spatial scale.

[0116] By minimizing this divergence, the shallow Nano-OBB Head is forced to mimic the decision-making logic of the deep Main-OBB Head as much as possible, and learns to hesitate on "difficult samples" (outputting low confidence), thereby significantly improving the reliability of the Nano-OBB Head's confidence during the inference phase and ensuring the safety of the early stopping mechanism.

[0117] Step S4 in this embodiment specifically includes:

[0118] S41: Input preprocessing: Adjust the image to be detected to the model input size and perform normalization processing;

[0119] The raw images captured by the camera are scaled proportionally using the Letterbox algorithm. The pixel values ​​are then filled with gray borders and normalized from [0,255] to [0,1] to accelerate model inference.

[0120] S42: Prior Inference: After the image data flows through the P3 layer of the backbone network and the neck network, the prior detection head (Nano-OBB Head) immediately outputs the maximum class confidence of all candidate boxes in the current frame;

[0121] The data stream first passes through the improved Backbone (including the LEG module) to reach the P3 node of the Neck. At this point, the Nano-OBBHead immediately intervenes, performing a single convolutional prediction on the P3 feature map and outputting the maximum confidence score of all potential targets in the current frame. In the formula, This represents the highest composite score among all predicted bounding boxes in the current frame; Conf represents the confidence that a target exists within the predicted bounding box. This represents the conditional probability that the target belongs to a specific category, given the existence of the target. The product of these two probabilities is the final confidence level of the detection box for that specific category.

[0122] S43: Routing Decision: Set an early stopping threshold. If the maximum category confidence is greater than the threshold, it is determined to be a "low ambiguity / significant target condition" (i.e., the target moves slowly or is in a hovering state), in which case the Nano Head is sufficient for accurate positioning; if the confidence is low, it is determined to be a "high dynamic / strong ambiguity condition" (i.e., the robotic arm is in a rapid approach phase), in which case the Main Head is activated. Technical effect: This mechanism essentially constructs a "speed-precision dynamic game player," enabling the system to perceive the current motion state and automatically adjust the perception strategy.

[0123] Set early stop threshold (For example, 0.85). This step is crucial for achieving "adaptive inference." Considering that visual grasping tasks typically prioritize the salient target with the highest confidence in the field of view (i.e., the target to be grasped):

[0124] Scenario 1 (Simple scenario): When At this point, the system deems the primary target to be very clear and salient. It then directly adopts the bounding box of the Nano-OBB Head as the final result and forcibly terminates GPU kernel calls to subsequent P4, P5 layers, and the Main-OBBHead. This operation skips approximately 60% of the network depth, significantly reducing inference time and boosting the frame rate.

[0125] Scenario 2 (Complex Scenarios): When At this point, the system determines that motion blur or occlusion exists, and the Nano-OBBHead is not very certain about its accuracy. The data stream continues to flow downwards, undergoing deep aggregation by GS-PANet, and finally the more accurate Main-OBB Head provides the result.

[0126] S44: Post-processing: Non-maximum suppression (NMS) is applied to the output predicted bounding boxes to filter out redundant boxes with high overlap, and the final detection result is output.

[0127] Regardless of the detection head from which the output originates, the NMS operation is performed at the end. The IoU between candidate boxes is calculated, and when the IoU is greater than 0.45, boxes with low confidence are discarded to ensure that only one best detection box is retained for each target.

[0128] Step S5 in this embodiment specifically includes:

[0129] S51: Feature Analysis: Analyze the target bounding box output by the model to obtain the target's center coordinates, long side, short side, and rotation angle; the final tensor output by the model contains... .in Let w be the pixel coordinates of the center point, and h be the length and width of the OBB, respectively. The angle between the longer side and the horizontal axis.

[0130] S52: Shape Anisotropy Analysis: Calculate the aspect ratio and set a shape threshold (e.g., 1.2); calculate the aspect ratio. In the formula, Ratio represents the aspect ratio of the target. and These represent the width and height of the rotated bounding box output by the model, respectively; The length of the longer side of the target geometry; This represents the length of the shorter side of the target geometry. This metric reflects the shape characteristics of the object. (Setting a threshold) .like This indicates that the object exhibits a distinctly elongated shape (non-isotropic); if This indicates that the object is approximately square or circular (approximately isotropic).

[0131] S53: If the aspect ratio is greater than the shape threshold, the target is determined to be a long strip or an anisotropic object. The target rotation angle of the gripper at the end of the robotic arm is calculated to ensure that the gripper closing direction is perpendicular to the long axis of the object (i.e., gripping the short side). If the aspect ratio is less than or equal to the shape threshold, the target is determined to be an approximately isotropic object. The current gripper angle or any angle is maintained for gripping.

[0132] This is a gripping strategy based on physical stability. For long, narrow objects (such as wrenches or pens), the optimal gripping point is usually located at the center, and the grippers should close perpendicular to the long axis to achieve the minimum lever arm and maximum contact stability. Therefore, the target rotation angle is set as follows: For round objects (such as bottle caps), the angle has little impact on gripping stability, and the current angle can be maintained to reduce the range of motion of the robotic arm.

[0133] S54: Closed-loop control: Substitute the planned grasping pose and target center coordinates into the image-based visual grasping (IBVS) speed control law to generate a robotic arm speed control signal, driving the robotic arm to approach the target.

[0134] Constructing feature error vectors , where the current feature Expected features This corresponds to the state when the object is located at the center of the image and has reached the predetermined grasping depth.

[0135] At the same time, the target center point is obtained using a depth camera at the end of the robotic arm. Depth value at .

[0136] Using the image Jacobian matrix Mapping the error in the image plane to the velocity in Cartesian space:

[0137]

[0138] In the formula, The velocity vector of the end effector of the robotic arm; This is the gain coefficient for visual grasping control; For the image Jacobian matrix The Moore-Penrose pseudoinverse matrix; This is the error vector between the current feature and the desired feature. Here, the depth value is... And the image feature coordinates are substituted into the image Jacobian matrix. In the analytical expression, the matrix values ​​are updated in real time. This control law generates 6-DoF velocity commands, driving the robotic arm to adjust its posture in real time until the image error converges to an acceptable range, after which it executes the closing gripper action. This closed-loop process can effectively compensate for the kinematic errors of the robotic arm itself and the hand-eye calibration residuals.

[0139] Based on the above scheme, in order to verify the actual effect of the method in this embodiment, this embodiment conducted a performance comparison test between the target detection model of this embodiment and the original YOLOv8n model on the same embedded computing platform. The results were obtained respectively. Figure 6 The comparison images shown are of detection results in blurred scenes. Figure 6 (a) in the figure represents several images to be detected. Figure 6 (b) in the figure represents the detection result of the detection method of the present invention. Figure 6 (c) in the figure represents the detection results of the original YOLOv8n model. A specific qualitative analysis is as follows: When faced with high-speed moving objects, the original YOLOv8n model, due to its inability to handle anisotropic ambiguity, exhibits significant missed detections and positioning drift, and easily misidentifies the blurred background as the target. In contrast, the model of this invention, benefiting from the DGCA unit's perception of the ambiguity direction, not only accurately captures the boundary contours of blurred targets, eliminating false detections and missed detections, but also demonstrates a faster real-time recognition response speed in multi-target concurrent scenarios, proving the algorithm's strong robustness under dynamic conditions.

[0140] Table 1 Comparison of Test Results

[0141]

[0142] To more comprehensively evaluate the independent contributions and synergistic effects of each improved module (LEG, GS-PANet, Dynamic-Head) in this invention, rigorous ablation experiments were designed in this embodiment. Evaluation metrics included mean accuracy (mAP@0.5) and inference speed (FPS). Test results are shown in Table 1.

[0143] The following conclusions can be drawn from the experimental data in Table 1: (1) The anti-fuzzing effectiveness of DGCA: After introducing only the LEG module containing DGCA on the basic model, the mAP increased from 68.5% to 71.2%. This shows that the DGCA unit extracts the gradient prior of the variable-length trailing shadow through multi-scale expansion difference and effectively suppresses the fuzzing noise through gradient weighted sharpness pooling, verifying its theoretical assumption of "physically filtering out fuzziness". (2) Verification of the synergistic mechanism of "lightweight-high frequency compensation": Although using GS-PANet alone greatly improves the inference speed to 39FPS, the linear transformation characteristics of the Ghost module lead to the loss of high-frequency information, and the mAP drops to 67.8%. However, after introducing the LEG module, the mAP rebounded to 74.5%. This result strongly proves that there is a strong coupling complementary relationship between the LEG module and GS-PANet: the cross-layer edge injection mechanism of LEG physically compensates for the spectral loss caused by the Ghost module. (3) The balanced advantage of dynamic inference: After introducing Dynamic-Head, the complete model finally achieved a maximum accuracy of 75.9% and a high frame rate of 34 FPS. This shows that the adaptive routing strategy based on fuzzy confidence successfully saved computing power in the "low fuzzy condition" and ensured accuracy in the "high dynamic condition", achieving the best balance between speed and accuracy.

Claims

1. A method for detecting small targets resistant to motion blur, characterized in that, Includes the following steps: Step 1: Obtain the dataset for target detection enhanced by motion blur simulation; Step 2: Construct an object detection model based on the improved YOLOv8n; the object detection model includes a backbone network, a neck network, and a detection end connected in sequence. The backbone network is used to extract features, and the extracted features are input into the neck network for aggregation. The aggregated features are then identified and labeled at the detection end; the second C2f module of the backbone network is replaced with a local edge-guided LEG module, and the intermediate features spliced ​​in the bottom-up fusion path of the neck network are replaced with the output features spliced ​​by the LEG module through horizontal connection; The LEG module splits the input features into semantic branches and edge branches. The edge branches extract gradients from the features to obtain gradient maps. Based on the gradient maps, the pooling weights are calculated using a normalization function. The features of the semantic branches are modulated according to the weights, and the modulated features are output. The ratio of total gradient energy is calculated using the horizontal and vertical gradient maps. This ratio is then used to generate correction coefficients to adjust the attention coefficients. These corrected attention coefficients are then applied to modulate the features of the semantic branch, resulting in the following modulated features: ； in, For input features, For the corrected attention coefficient, This is an element-wise multiplication operation; ； in, For activation function, Indicates feature splicing, For horizontal reconstruction features, For vertical reconstruction features, For multilayer perceptrons, This represents the ratio of total gradient energy. ； in, This is a horizontal gradient plot. This is a vertical gradient plot. To prevent constants with a denominator of zero; The detection end includes a priori detection head and a main detection head. The input of the priori detection head is the P3 layer fusion feature output by the neck network. The priori detection head includes a layer A convolutional layer and a sigmoid activation function generate a confidence score based on the detection results of the prior detection head. If the confidence score meets the requirements, the detection result of the prior detection head is output; if the confidence score does not meet the requirements, the main detection head is activated. The input of the main detection head is the fused features of layers P3, P4, and P5 output by the neck network. The main detection head adopts a decoupled head structure, which processes data through parallel classification and regression branches. The classification branch outputs the class probability of the target, and the regression branch outputs the bounding box position, size, and angle. Step 3: Train the object detection model based on the acquired dataset to obtain the trained object detection model; Step 4: Input the real-time acquired image to be detected into the trained target detection model, and output the bounding box of the target.

2. The method for detecting small targets against motion blur according to claim 1, characterized in that, The image enhanced by motion blur simulation is represented as follows: ； in, This represents the generated simulated blurred image; Represents the original, clear image; Represents the linear motion fuzzy kernel; This represents the convolution operator; This represents the Gaussian noise matrix.

3. The method for detecting small targets against motion blur according to claim 1 or 2, characterized in that, It also includes multi-scale transformation enhancement of the object detection dataset, scaling the images in the dataset using a random scaling factor.

4. The method for detecting small targets against motion blur according to claim 1, characterized in that, Multi-scale expanded difference is used to extract gradients from features, specifically: for the basic difference operator Apply different expansion rates Perform differential convolution to obtain the gradient map. : ； in, For input features, , It is a constant. This represents convolution.

5. The method for detecting small targets against motion blur according to claim 4, characterized in that, Horizontal and vertical gradients are extracted from the features using horizontal and vertical difference operators to obtain horizontal and vertical gradient maps. Weights are calculated from the gradient maps using a normalization function, specifically the Softmax function. The weights include horizontal and vertical weights. Horizontal features are reconstructed using the horizontal weights, and vertical features are reconstructed using the vertical weights. Attention coefficients are generated from the reconstructed horizontal and vertical features, and these attention coefficients are used to modulate the features of the semantic branches.

6. The method for detecting small targets against motion blur according to claim 1, characterized in that, When training the target detection model, no confidence score is performed on the detection end. The main detection head and the prior detection head are trained simultaneously. A self-distillation loss function is introduced for the detection end. The output of the main detection head is used as the teacher signal and the output of the prior detection head is used as the student signal. The KL divergence loss between the output probability distributions of the main detection head and the prior detection head is calculated so that the prior detection head can imitate the feature expression of the main detection head as much as possible.

7. The method for detecting small targets against motion blur according to claim 1, characterized in that, All C2f modules in the neck network are replaced with GS modules, and all convolutional layers in the neck network are replaced with Ghost convolutions. The Ghost convolution includes a channel shuffling module. The GS module includes convolutional layers, a linear transformation module, a concatenation module, and a channel shuffling module. The convolutional layers perform convolution operations on the input of the GS modules, and the number of convolutional kernels in the convolutional layers is half that of ordinary convolutional layers. The linear transformation module performs a cost-saving operation on the output of the convolutional layers, which involves linearly transforming the feature map of each channel one by one. The concatenation module concatenates the output of the convolutional layers and the output of the linear transformation module. The channel shuffling module is used to perform channel shuffling on the output of the splicing module. The output of the channel shuffling module is the final output of the CS-Ghost module.

8. The method for detecting small targets against motion blur according to claim 1, characterized in that, Based on the bounding box of the target, obtain the center coordinates, long side, short side and rotation angle of the target. Calculate the aspect ratio based on the long and short sides of the target, and control the gripping angle of the robotic arm based on the aspect ratio.

Images