Lightweight multi-scale feature fusion variable edge efficient visual detection method and computing device

Abstract

The invention provides a lightweight multi-scale feature fusion-based variable edge efficient visual detection method and computing equipment. The method comprises the steps of obtaining a to-be-detected image; extracting multi-layer feature maps of different scales through the backbone network; performing cross-level fusion on the multi-level feature maps of different scales through a neck network to obtain a fused multi-scale feature map; outputting a target detection result from the fused multi-scale feature map through a detection head; wherein the backbone network comprises a plurality of alternately arranged edge sensing down-sampling convolution modules and variable Sobel convolution inversion modules; the edge sensing down-sampling convolution module extracts compression features and fine-grained features through a double-branch structure, and outputs a down-sampling feature map to the variable Sobel convolution inversion module to obtain an edge enhancement feature map. According to the technical scheme of the invention, the problems of weak edge extraction capability, serious detail loss and high calculation overhead in a complex scene in the prior art can be solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a lightweight, multi-scale feature fusion-based, high-efficiency visual detection method and computing device for variable edges. Background Technology

[0002] Real-time object detection is one of the core tasks in the field of computer vision. In recent years, the YOLO series of models have been widely used due to their balance between speed and accuracy. However, object detection still faces severe challenges in complex natural environments (such as underwater, traffic monitoring, and drone aerial photography).

[0003] The existing technologies mainly have the following problems: the parameters of traditional convolution kernels are fixed, which makes it difficult to adapt to the situation of blurred target edges and variable directions in complex scenes, resulting in insufficient feature extraction; existing multi-scale feature fusion modules (such as SPP and SPPF) rely heavily on pooling operations, which can easily cause the loss of high-frequency detail information and have high computational redundancy; traditional downsampling operations often involve significant loss of semantic information when compressing feature map size.

[0004] Therefore, a technical solution is needed that can balance edge perception, multi-scale feature preservation, and computational efficiency in detection methods. Summary of the Invention

[0005] This invention aims to propose a lightweight, multi-scale feature fusion-based, high-efficiency visual detection method and computing device for variable edges. By introducing a dynamic edge perception mechanism and an efficient convolutional structure, it solves the problems of weak edge extraction capability, severe loss of details, and high computational overhead in existing technologies under complex scenes.

[0006] According to one aspect of the present invention, a lightweight, multi-scale feature fusion-based, high-efficiency visual detection method for variable edges is provided, comprising: Acquire the image to be detected; Multi-layer feature maps of different scales are extracted using the backbone network; The multi-layer feature maps of different scales are fused across layers through the neck network to obtain the fused multi-scale feature map. The fused multi-scale feature map is then used by the detection head to output the target detection result. in, The backbone network includes multiple alternating edge-aware downsampling convolutional modules and variable Sobel convolution inversion modules; The edge-aware downsampling convolution module extracts compressed features and fine-grained features through a dual-branch structure, and outputs the downsampling feature map to the variable Sobel convolution inversion module. The variable Sobel convolution inversion module dynamically adjusts the direction and size of the convolution kernel through variable Sobel convolution, and combines it with the inverted residual structure to obtain an edge enhancement feature map.

[0007] According to some embodiments, the variable Sobel convolution inversion module includes a variable Sobel convolution unit and an inversion unit, wherein: The variable Sobel convolutional unit is used to generate Sobel operators at different angles through a rotation matrix, and to perform multi-directional edge detection on the input feature map to obtain multi-directional edge response features; The inverted unit is used to expand the feature channels, perform depthwise separable convolution, compress them back to the original dimension, and perform residual connection with the input features to obtain semantically enhanced features. The multi-directional edge response features and the semantic enhancement features are normalized and summed to obtain the output features.

[0008] According to some embodiments, in the variable Sobel convolution unit, the smoothing operator coefficients S m Simulated Gaussian smoothing is used to remove noise through the difference operator coefficients D. m Calculate the gradient using the smoothing operator coefficients S. m With difference operator coefficient D m Construct the weight matrix by performing an outer product combination, where the weight matrix is ​​located in the zero-degree direction of the horizontal axis: , , , , Where L is the kernel size and m is the index of the window along the horizontal axis. It is a defined function. This represents the local pixel grayscale value of the image at window coordinates (m, n), where M represents the original matrix. This is the output gradient response.

[0009] According to some embodiments, the backbone network includes a spatial pyramid shared convolutional module at its end. The spatial pyramid shared convolutional module aggregates global context information, constructs a spatial pyramid structure using dilated convolutions with shared weights, and extracts multi-scale contextual features.

[0010] According to some embodiments, the spatial pyramid shares a convolutional module: The first 1×1 convolution is used to compress the input channel; Three parallel dilated convolution branches share the convolution kernel weights and have different dilation rates and padding strategies, with the dilation rates set to 1, 3, and 5, respectively. The splicing module splices the outputs of the three parallel dilated convolution branches with the compressed original features along the channel dimension. The second 1×1 convolutional fusion is used to output multi-scale perceptual features.

[0011] According to some embodiments, the edge-aware downsampling convolutional module includes a main path and an auxiliary path, wherein... The main path compresses the channel through a 1×1 convolution, and then uses a depthwise separable convolution with a stride of 2 for spatial downsampling; The auxiliary path directly processes the original input using a standard convolution with a stride of 2 to preserve fine-grained features; The output features of the main path and the output features of the auxiliary path are concatenated to form the downsampled feature map x. out : , in, This represents the tensor concatenation operation. The feature map is from the main path. The feature map comes from the auxiliary path.

[0012] According to some embodiments, the backbone network, the neck network, and the detection head are built based on the YOLO architecture; The detection head uses the edge-aware downsampling convolution module to extract positional features and combines them with the classification branch to output the category and position of the predicted bounding box.

[0013] According to another aspect of the present invention, a computing device is provided, comprising: Processor; and A memory that stores a computer program, which, when executed by the processor, implements the method as described in any of the preceding methods.

[0014] According to another aspect of the present invention, a drone, an intelligent driving vehicle, or a monitoring device is provided, including the aforementioned computing device.

[0015] According to an embodiment of the present invention, the backbone network extracts multi-layer feature maps of different scales through multiple alternating edge-aware downsampling convolutional modules and variable Sobel convolution inversion modules, and outputs high-fidelity multi-scale features layer by layer. The neck network performs cross-layer fusion on this basis to avoid semantic drift caused by multiple upsampling / downsampling. By replacing traditional pooling or single-path convolution with a dual-branch downsampling structure, the computational load is reduced while retaining key spatial details. This is particularly beneficial for the unified optimization of small target detection (relying on high-resolution features) and large target localization (relying on high-level semantics).

[0016] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit the invention. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0018] Figure 1 A flowchart of a lightweight, multi-scale feature fusion-based, efficient visual detection method for variable edges is shown according to an example embodiment.

[0019] Figure 2 The diagram illustrates the overall architecture of a variable edge-aware detection network based on lightweight multi-scale feature fusion, according to an example embodiment.

[0020] Figure 3 This diagram illustrates the composition of a variable edge-aware detection model based on lightweight multi-scale feature fusion, according to an example embodiment.

[0021] Figure 4 A schematic diagram of the structure of a neck network according to an example embodiment is shown.

[0022] Figure 5 A schematic diagram of the structure of a dual-channel position self-attention module (C2PSA) according to an example embodiment is shown.

[0023] Figure 6 A schematic diagram of the structure of the Variable Sobel Convolution Inversion Module (VSCIB) according to an example embodiment is shown.

[0024] Figure 7 A schematic diagram of the structure of the Spatial Pyramid Shared Convolutional Module (SPSC) according to an example embodiment is shown.

[0025] Figure 8 The diagram illustrates an edge-aware downsampling convolutional module (EDC) according to an example embodiment and its application structure in a detection head.

[0026] Figure 9 A block diagram of a computing device according to an exemplary embodiment is shown. Detailed Implementation

[0027] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that the invention will be thorough and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0028] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0029] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0031] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of the present invention. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0032] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing the present invention, and therefore cannot be used to limit the scope of protection of the present invention.

[0033] Real-time object detection, as one of the core tasks in computer vision, is widely used in key scenarios such as intelligent transportation, unmanned systems, security monitoring, underwater detection, and industrial automation. In recent years, single-stage detectors, represented by the YOLO series, have become the preferred architecture for industrial deployments due to their balance between high inference speed and good accuracy. However, in complex natural environments (such as low-light underwater scenes, aerial footage from high-speed moving drones, and densely obscured traffic intersections), existing methods still face significant challenges.

[0034] The feature extraction capabilities of existing technologies are limited by fixed convolutional kernels. Traditional convolutional neural networks (CNNs) use static, isotropic convolutional kernels for feature extraction. This design performs well under ideal imaging conditions, but in complex scenes with blurred target edges, weakened textures, or variable orientations (such as underwater suspended particle interference, rainy / foggy weather, and motion blur), it is difficult to effectively capture local structural information and orientation-sensitive features. This leads to insufficient model response to small targets and low-contrast targets, severely impacting detection recall. Current mainstream YOLO architectures generally use SPP (Spatial Pyramid Pooling) or SPPF (Fast SPP) modules to achieve multi-scale contextual modeling. However, these modules heavily rely on max pooling operations, which, while expanding the receptive field, inevitably filter out high-frequency detail information (such as edges, corners, and textures) and introduce redundant computation. Especially in scenarios requiring precise localization (such as license plate recognition and underwater bio-counting), this information loss directly reduces localization accuracy. To reduce computational complexity, detection networks typically compress feature map sizes step-by-step using strided convolutions or pooling layers. However, this process often comes at the cost of sacrificing spatial resolution and local semantics, resulting in a lack of fine-grained geometric cues in deep features. For targets with drastic scale changes or densely packed arrangements (such as pedestrian clusters seen from above by a drone), traditional downsampling strategies are prone to causing target confusion or missed detections.

[0035] To address these issues, this application proposes a lightweight, multi-scale feature fusion-based, high-efficiency visual detection method and computing device for variable edges. By introducing a dynamic edge perception mechanism and an efficient convolutional structure, it solves the problems of weak edge extraction capability, severe detail loss, and high computational cost in existing technologies in complex scenes. It can synergistically optimize edge perception, multi-scale fusion, and downsampling efficiency, significantly improving the model's robustness and generalization ability in the real world without increasing computational burden.

[0036] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application.

[0037] Figure 1 A flowchart of a lightweight, multi-scale feature fusion-based, efficient visual detection method for variable edges is shown according to an example embodiment.

[0038] See Figure 1 In S101, the image to be detected is acquired.

[0039] According to the example implementation, an image to be detected is obtained, and an original RGB image is used as the input of a variable edge efficient visual detection network to construct an end-to-end target detection process.

[0040] In S103, multi-layer feature maps of different scales are extracted through the backbone network.

[0041] According to the example embodiment, multi-layer feature maps of different scales are extracted through a backbone network. The backbone network includes multiple alternating edge-aware downsampling convolutional modules (EDC) and variable Sobel inverted convolutional modules (VSCIB). The edge-aware downsampling convolutional modules (EDC) extract compressed features and fine-grained features through a dual-branch structure, and output the downsampling feature map to the variable Sobel inverted convolutional module, as referred to below. Figure 8 As described in detail, the variable Sobel convolution inversion module dynamically adjusts the direction and size of the convolution kernel through variable Sobel convolution (VSC) and combines it with the inverted residual structure (CIB) to obtain an edge enhancement feature map.

[0042] According to some embodiments, the backbone network includes a Spatial Pyramid Shared Convolution (SPSC) module at its end. This SPSC module aggregates global context information, constructs a spatial pyramid structure using dilated convolutions with shared weights, and extracts multi-scale contextual features. See [link to SPSC implementation details]. Figure 7 The system includes a first 1×1 convolution for compressing the input channels, three parallel dilated convolution branches sharing kernel weights, each with different dilation rates and padding strategies, set to 1, 3, and 5 respectively. A concatenation module concatenates the outputs of the three parallel dilated convolution branches with the compressed original features along the channel dimension, and a second 1×1 convolution fusion module outputs multi-scale perceptual features.

[0043] According to some embodiments, the Edge-aware Detection Convolution (EDC) module includes a main path and an auxiliary path. The main path compresses channels using a 1×1 convolution and then performs spatial downsampling using a depthwise separable convolution with a stride of 2. The auxiliary path directly processes the original input using a standard convolution with a stride of 2 to preserve fine-grained features. The output features of the main path and the output features of the auxiliary path are concatenated to form the downsampled feature map x. out : , in, This represents the tensor concatenation operation. The feature map is from the main path. The feature map comes from the auxiliary path.

[0044] According to some embodiments, the variable Sobel convolution unit is used to generate Sobel operators at different angles through a rotation matrix to perform multi-directional edge detection on the input feature map and obtain multi-directional edge response features; the inverted unit is used to expand the feature channels, perform depthwise separable convolution, compress them back to the original dimension, and perform residual connection with the input features to obtain semantically enhanced features; the multi-directional edge response features and the semantically enhanced features are normalized and summed to obtain the output features.

[0045] In the variable Sobel convolution unit, the smoothing operator coefficients S m Simulated Gaussian smoothing is used to remove noise through the difference operator coefficients D. m Calculate the gradient using the smoothing operator coefficients S. m With difference operator coefficient D m Construct the weight matrix by performing an outer product combination, where the weight matrix is ​​located in the zero-degree direction of the horizontal axis: , , , , Where L is the kernel size and m is the index of the window along the horizontal axis. It is a defined function. This represents the local pixel grayscale value of the image at window coordinates (m, n), where M represents the original matrix. This is the output gradient response.

[0046] In S105, the multi-layer feature maps of different scales are fused across layers through the neck network to obtain the fused multi-scale feature maps.

[0047] According to the example embodiment, the multi-layer feature maps of different scales are fused across layers through the neck network to obtain the fused multi-scale feature map. See [link to example embodiment]. Figure 2 and Figure 3 .

[0048] In some embodiments, after the backbone network completes feature extraction, it uses three key feature layers at different scales (typically from the outputs of the 8x, 16x, and 32x downsampling stages) as input to the neck network. For example, the neck network employs an enhanced PAFPN structure to deeply fuse these features through two opposing paths.

[0049] According to some embodiments, the first path of the neck network (a top-down FPN path) aims to transfer the rich semantic information from the deep-processed feature maps in the high-level feature maps to the shallow feature maps to help identify small targets. The first path starts with the feature map from the deepest layer (32x downsampling) of the backbone network, enlarges its size through an Up (upsampling) operation, and performs a C (feature merging) with the feature map from a shallower layer (16x downsampling) of the backbone network. The merged feature then passes through the C2f module (see...). Figure 4 Deep processing is performed to achieve efficient feature recombination and expression.

[0050] This process is repeated: the 16x-scale fused features are upsampled again and merged with a shallower 8x-scale feature map, followed by C2f (Cross-Connection Feature Fusion) processing. After the top-down fusion, the network immediately initiates a second path (a bottom-up path). The goal of this second path is to transfer more precise localization information from the shallow feature maps back to the deeper layers, improving the accuracy of target localization. The second path starts with the shallowest (8x-scale) feature map after semantic fusion. After processing by the C2f module, its size is reduced through an EDC module (which contains convolutions with a stride of 2 for downsampling), and then fused with the 16x-scale feature map from the first path, thus supplementing the shallow localization details into the deeper features. This process is repeated again, with the 16x-scale enhanced features downsampled again through EDC or a similar structure and fused with a 32x-scale feature map.

[0051] After sufficient information exchange and enhancement through the top-down and bottom-up paths, the neck network will eventually output a new set of deeply fused multi-scale feature maps (corresponding to the s, m, and l outputs in the figure).

[0052] In S107, the fused multi-scale feature map is output as a target detection result through the detection head.

[0053] According to the example embodiment, the backbone network, the neck network, and the detection head are built based on the YOLO architecture. The detection head uses the edge-aware downsampling convolution module to extract positional features and combines the classification branch to output the category and position of the predicted bounding box.

[0054] According to the example implementation, the fused multi-scale features are input into three detection head branches, which correspond to small target detection head, medium target detection head and large target detection head, respectively.

[0055] For example, a small object detection head used at 80×80 resolution first refines features through an EDC module, where the channel compression rate is set to 0.3C to retain more details. Then, spatial information is enhanced through a 3×3 depthwise separable convolution. Finally, a 1×1 convolution outputs a prediction tensor with the following shape: [batch, anchors×(4+1+num_classes), 80, 80], Where batch represents the number of samples in a batch, anchors is the preset number of bounding boxes, 4 represents the bounding box coordinates (tx, ty, tw, th), 1 represents the target confidence, num_classes represents the class probability, and 80 represents the resolution.

[0056] According to some embodiments, the medium target detection head is used for 40×40 resolution, applying the same structure as the small target detection head, but the channel compression rate of the EDC module is adjusted to 0.5C, and the output prediction tensor shape is: "[batch, anchors×(4+1+num_classes), 40, 40]". The large target detection head is used for 20×20 resolution, the channel compression rate of the EDC module is set to 0.7C, and the output prediction tensor shape is: "[batch, anchors×(4+1+num_classes), 20, 20]".

[0057] For the prediction results of each detector head, edge-sensitive region recalibration is performed, and edge-aware weights are calculated: W(x,y)=σ(α·E(x,y)+β), Where E(x,y) is the edge map generated from the original image, σ is the sigmoid function, α=0.8, β=0.2.

[0058] Multiplying the weighted map with the bounding box coordinate prediction enhances the localization accuracy of edge regions.

[0059] The post-processing mainly includes non-maximum suppression (NMS) to filter redundant detection boxes, and the final detection results are selected based on the set confidence threshold.

[0060] The raw predictions output by the detection head are post-processed to generate the final results, including confidence filtering, retaining predicted bounding boxes with a target confidence greater than 0.3, and independently applying a confidence threshold for each category. For example, the threshold for small targets is set to 0.25, for medium targets to 0.3, and for large targets to 0.35. The bounding box coordinates are decoded, converting the offsets (tx, ty, tw, th) output by the network into actual coordinates (x, y, w, h). x=σ(tx)·2-0.5+grid_x, y=σ(ty)·2-0.5+grid_y, w=pw·e tw h = ph·e th , Where (grid_x,grid_y) are the grid coordinates, and (pw,ph) are the anchor frame dimensions. According to some embodiments, edge-aware non-maximum suppression (NMS) is performed by first calculating the intersection ratio between the predicted bounding box and the edge map, and then assigning edge weights to each predicted bounding box. During NMS, the IoU threshold is dynamically adjusted based on the edge weights: when the edge weight > 0.7, the IoU threshold = 0.45; when the edge weight ≤ 0.7, the IoU threshold = 0.65, prioritizing the retention of detection boxes with high edge weights.

[0061] The final result is filtered, the detection results of the three scales are merged, and outlier boxes smaller than 3×3 pixels or larger than 85% of the image size are removed. The output includes a list of detection results containing the target category, confidence score, bounding box coordinates, and an optional edge-sensitive segmentation mask.

[0062] Figure 2 The diagram shows the input block diagram of a lightweight, multi-scale feature fusion-based, high-efficiency visual detection network for variable edges, according to an example embodiment.

[0063] According to the example embodiment, the original image received by the system is a standard color image, and the dimensions of the input tensor are represented as: [H,W,3], where: H represents the height, W represents the width, and 3 represents the three color channels of red (R), green (G), and blue (B), with a spatial size of H×W and three color channels (RGB).

[0064] First, the images undergo standardization preprocessing, including pixel value normalization to the range of [-1, 1], adaptive scaling to maintain aspect ratio, and adjusting the shorter side to 640 pixels. Data augmentation includes random horizontal flipping (50% probability), Mosaic four-image stitching enhancement (75% probability), and color dithering (brightness, contrast, and saturation randomly ±20%). After a series of standardization preprocessing and data augmentation operations, the image input data ensures that the model has good generalization ability and robustness during the training phase, and is finally fed into the model's backbone network for feature extraction.

[0065] According to some embodiments, the original image is fed into a lightweight, multi-scale feature fusion-based, high-efficiency visual detection network model with variable edges in the form of an image tensor. The backbone network is based on the CSPDarknet architecture and begins to extract multi-scale features layer by layer. Subsequently, the feature maps are passed sequentially to the neck network (PAFPN) and the detection head. Through a loss function and an optimizer, the detection results containing the category, confidence score, and bounding box are finally generated.

[0066] Figure 3 This diagram illustrates the composition of a variable edge-aware detection model based on lightweight multi-scale feature fusion, according to an example embodiment.

[0067] See Figure 3 According to the example embodiment, the input data of the backbone network first passes through two EDC modules in succession. The core objective of the EDC modules is to minimize the loss of edge information caused by downsampling while effectively reducing the feature map resolution and expanding the receptive field.

[0068] The EDC internal structure comprises a dual-branch architecture. The main path consists of a 1×1 CBS convolution (channels compressed to 0.5C) followed by a 3×3 depthwise separable convolution (stride=2). The auxiliary path is a 3×3 standard convolution (stride=2). The outputs of the two paths are merged element-wise to preserve more edge details, as shown in the following reference. Figure 8 As described in detail.

[0069] In this embodiment, the VSCIB module utilizes the built-in variable Sobel operator to adaptively enhance the ability to extract edge and texture features of different directions and scales, thereby enabling it to accurately capture the contour information of small objects.

[0070] According to some embodiments, the feature map enters the first VSCIB module, where the downsampling rate is 4x. In the backbone network, the data flow alternates between the EDC module and the VSCIB module to continuously enhance edge perception capabilities at different scales. First, the downsampling rate is increased to 8x through an EDC module. Then, the data flow passes through another VSCIB module, capturing a larger range of edge structures at an 8x downsampling scale. This process is repeated, increasing the downsampling rate to 16x through the EDC module, and the VSCIB module further enhances edge perception of large targets at a 16x downsampling scale. Finally, the downsampling rate is increased to 32x through an EDC module and connected to a VSCIB module to obtain edge features within the maximum receptive field.

[0071] At the end of the backbone network, to aggregate multi-scale contextual information and focus on key regions, the feature maps pass through two high-level modules: SPSC and C2PSA (Channel-wise + Positional + Position-aware Self-Attention). The C2PSA dual-channel positional self-attention module (see...) Figure 5 Internally, it integrates the PSABlock (polarization self-attention) mechanism, which allows the model to adaptively learn attention weights in the channel and spatial dimensions, thereby focusing more on feature regions that are crucial to the detection task and suppressing interference from irrelevant information.

[0072] Figure 6 A schematic diagram of the structure of the Variable Sobel Convolution Inversion Module (VSCIB) according to an example embodiment is shown.

[0073] The VSCIB module is the core unit, composed of the VSC and CIB sub-modules in parallel. It fuses edge and semantic features through 1×1 convolutions. The VSC sub-module dynamically generates Sobel convolution kernels with a kernel size of L=5 in four directions (0°, 45°, 90°, and 135°) to extract multi-directional edge information, and smooths it by element-wise summation and averaging (+ / n). The CIB sub-module uses a residual structure of "3×3 depthwise separable convolution → 1×1 convolution (expansion) → 3×3 depthwise separable convolution → 1×1 convolution (compression) → 3×3 depthwise separable convolution" to extract deep features while maintaining lightweight design. CBS (Conv BN SiLU, Convolution-Batch Normalization-SiLU module) integrates convolution, normalization, and activation functions to ensure efficient non-linear representation and feature flow.

[0074] In this embodiment, the VSCIB module utilizes the built-in variable Sobel operator to adaptively enhance the ability to extract edge and texture features of different directions and scales, thereby enabling it to accurately capture the contour information of small objects.

[0075] Figure 7 A schematic diagram of the structure of the Spatial Pyramid Shared Convolutional Module (SPSC) according to an example embodiment is shown.

[0076] See Figure 7 According to some embodiments, the spatial pyramid shared convolutional module includes a first 1×1 convolution for compressing the input channels, three parallel dilated convolutional branches sharing kernel weights, each with different dilation rates and padding strategies, the dilation rates being set to 1, 3, and 5 respectively. A concatenation module concatenates the outputs of the three parallel dilated convolutional branches with the compressed original features along the channel dimension, and a second 1×1 convolution is used for fusion to output multi-scale perceptual features.

[0077] According to the example embodiment, the SPSC module first compresses the number of channels to 50% through a 1×1 CBS convolution, and then performs three dilated convolutions with shared weights in parallel (dilation rates of 1, 3, and 5, respectively). Each branch uses the same 3×3 convolution kernel but uses a different padding strategy. Finally, the outputs of each branch are concatenated with the original features and the number of channels is restored through a 1×1 convolution.

[0078] The SPSC module in this embodiment effectively captures multi-scale spatial context information by processing features of different receptive fields in parallel, thereby enhancing the model's ability to recognize targets of different sizes.

[0079] Figure 8This diagram illustrates the edge-aware downsampling convolutional module (EDC) according to an example embodiment and its application structure in the detection head (EDC_Detect).

[0080] According to some embodiments, see Figure 8 The EDC module showcases a dual-path feature extraction and fusion unit, comprising a main path and an auxiliary path, designed to enhance edge detail preservation. The input tensor has dimensions [b, c0, h0, w0], where b is the batch size, c0 is the number of input channels, and h0 and w0 are the spatial height and width of the input, respectively. In the main path, a 1×1 CBS convolution first compresses the channels to c1 / 2 (half the original number of channels), while maintaining the output size [b, c1 / 2, h0, w0]. A 3×3 depthwise separable convolution (DWConv) with a stride of 2 is then used for processing. This not only reduces computational complexity but also achieves spatial downsampling, halving the spatial size to obtain [b, c1 / 2, h1, w1], where h1 = h0 / 2 and w1 = w0 / 2. The auxiliary path contains a standard 3×3 convolution (Conv+BN) with a stride of 2, directly spatially downsampling the input and adjusting the number of channels to c1 / 2. Its output dimension is also [b, c1 / 2, h1, w1]. Finally, the outputs of the main path and the auxiliary path are fused element-wise to produce the final output [b, c1, h1, w1]. The outputs of the two paths are added at the same positions because they have the same shape, thus achieving multi-scale feature fusion and preserving more edge details.

[0081] According to some embodiments, the EDC_Detect module is a dual-path detection head structure used to simultaneously predict bounding box regression and class confidence. The input feature dimensions are [b, c_in, h, w]. Here, DWCBS refers to a combined module of "Depthwise Convolution + Batch Normalization + Swish activation function": First, depthwise convolution (DWConv) is used to perform independent spatial filtering on each input channel, significantly reducing computational cost. Then, batch normalization (BatchNorm) is used to stabilize the training process. Finally, Swish (or a similar non-linear activation function) introduces non-linear expressive power. This design significantly improves computational efficiency while maintaining strong feature extraction capabilities.

[0082] In the EDC_Detect module, the first path (regression path) first passes through an edge-aware convolution (EDC), then through CBS (i.e., standard convolution + BN + Swish) and a 1×1 convolution, finally outputting bounding box offsets (tx, ty, tw, th) with dimensions [b, 4, h, w]. The second path (classification path) sequentially uses DWCBS, 1×1 EDC, then DWCBS again, CBS, and a 1×1 convolution, progressively enhancing semantic expressiveness while preserving edge details, finally outputting a class confidence map with dimensions [b, num_classes, h, w]. The two paths share input but are structurally decoupled: the regression path is concise and efficient, focusing on localization accuracy; the classification path is deeper and uses DWCBS and EDC multiple times, strengthening sensitivity to edges and textures, and improving the ability to distinguish small targets or boundary regions. The overall design balances lightweight design and performance, making it suitable for target detection tasks with clear edges and varying scales. It also provides high-quality raw predictions for subsequent post-processing steps such as confidence filtering, coordinate decoding, and edge-aware NMS.

[0083] According to some embodiments, each scale of the feature map from the neck network takes into account rich contextual semantics and high-resolution detail information, which are then input into the detection head (EDC_Detect), laying a solid foundation for accurate detection.

[0084] According to embodiments of the present invention, the edge-aware optimized YOLO model is adapted to the Raspberry Pi hardware environment through INT8 quantization and operator fusion technology, converting the 32-bit floating-point model into an 8-bit integer format while preserving key computational precision for core modules such as VSCIB and SPSC. The model is converted to TensorFlow Lite format, integrates the XNNPACK inference engine, and optimizes memory access patterns for the ARM architecture to reduce data transfer overhead.

[0085] The system of this invention implements dynamic resource management on a Raspberry Pi 4B, automatically adjusting the input resolution (480×480 to 640×640) and inference frequency based on CPU temperature and load. Preprocessing and post-processing tasks are assigned to the GPU, while the main detection process is executed in parallel on a quad-core CPU. The deployment package includes lightweight API services that support receiving image input via HTTP or GPIO interfaces and outputting structured detection results, with memory usage kept below 350MB, ensuring coexistence with other edge applications.

[0086] In actual operation, the system achieved an inference speed of 14.3 FPS (640×640 resolution) on a Raspberry Pi 4B, with a stable power consumption of 3.5W, meeting the requirements of real-time edge detection. It maintained 54.1% mAP50 accuracy on the standard test set, a decrease of only 2.2% compared to the original model. The system supports continuous operation 24 / 7, features temperature protection and anomaly recovery mechanisms, and has been successfully applied in resource-constrained scenarios such as intelligent security and industrial inspection, achieving seamless deployment from algorithm to edge devices.

[0087] According to some embodiments, some test data are shown in Tables 1 and 2.

[0088] Table 1 Core Technical Indicators (MSCOCO)

[0089] As shown in Table 1, the improved model proposed in this invention improves the mAP50 by 1.5% and the mAP50-95 by 1.0% on the MS COCO dataset compared with the benchmark model YOLOv11n. It also significantly improves the edge perception accuracy by 7.2%, the inference speed by 7.7%, and reduces the number of parameters and GFLOPs by 7.7% and 10.8%, respectively.

[0090] Table 2 Performance Indicators for Different Application Scenarios

[0091] In specific application scenarios, the false detection rate for underwater target detection was reduced by 18.6%, the edge detection accuracy for road defect detection reached 90.2%, and the mAP50 for safety helmet detection reached 92.9%. This demonstrates that the method of this invention can effectively enhance the model's ability to capture complex edges and multi-scale targets while significantly reducing computational costs, thus significantly improving the model's detection performance and generalization adaptability.

[0092] In summary, compared to the original YOLOv11n benchmark model, the Edge Aware-YOLO model of this invention achieves significant performance improvements. The overall detection accuracy mAP50 increases from 54.8% to 56.3%, an improvement of 1.5 percentage points; the more stringent evaluation metric mAP50-95 increases from 39.5% to 40.5%, an improvement of 1.0 percentage point, demonstrating the steady growth in the model's overall detection performance. Particularly noteworthy is the significant improvement in edge perception accuracy by 7.2 percentage points, while the number of parameters and computational cost (GFLOPs) decreases by 7.7% and 10.8%, respectively, fully validating the synergistic advantages of the VSCIB, SPSC, and EDC modules in enhancing fuzzy edge detection capabilities and achieving model lightweighting. Furthermore, in specific scenarios such as underwater environments, road defects, and security monitoring, the model's false detection rate is significantly reduced, and the inference speed remains at a real-time level of 48.7 FPS, fully meeting the actual deployment requirements of UAVs and edge computing devices.

[0093] Figure 9 A block diagram of a computing device according to an exemplary embodiment of the present invention is shown.

[0094] like Figure 9 As shown, the computing device 30 includes a processor 12 and a memory 14. The computing device 30 may also include a bus 22, a network interface card 16, and an I / O interface 18. The processor 12, memory 14, network interface card 16, and I / O interface 18 can communicate with each other via the bus 22. The computing device of this invention can be applied to drones, autonomous vehicles, or monitoring equipment.

[0095] Processor 12 may include one or more general-purpose CPUs (Central Processing Units), microprocessors, or application-specific integrated circuits, for executing relevant program instructions. According to some embodiments, computing device 30 may also include a high-performance display adapter (GPU) 20 for accelerating processor 12.

[0096] Memory 14 may include a machine system readable medium in the form of volatile memory, such as random access memory (RAM), read-only memory (ROM), and / or cache memory. Memory 14 is used to store one or more programs containing instructions, as well as data. Processor 12 may read the instructions stored in memory 14 to perform the methods described above according to embodiments of the present invention.

[0097] The computing device 30 can also communicate with one or more networks via the network interface card 16. The network interface card 16 can be a DPU smart network card.

[0098] Bus 22 can include address bus, data bus, control bus, etc. Bus 22 provides a path for exchanging information between components.

[0099] It should be noted that, in specific implementations, the computing device 30 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the device described above may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0100] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of medium or device suitable for storing instructions and / or data.

[0101] This invention also provides a computer program product comprising a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0102] Those skilled in the art will clearly understand that the technical solutions of this application can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function, where the hardware may be, for example, a field-programmable gate array (FPGA), integrated circuit, etc.

[0103] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0104] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0106] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

[0109] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0110] The exemplary embodiments of this application have been specifically shown and described above. It should be understood that this application is not limited to the detailed structures, arrangements, or implementation methods described herein; rather, this application is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended provisions.

Claims

1. A lightweight, multi-scale feature fusion-based, high-efficiency visual detection method for variable edges, characterized in that, The method includes: Acquire the image to be detected; Multi-layer feature maps of different scales are extracted using the backbone network; The multi-layer feature maps of different scales are fused across layers through the neck network to obtain the fused multi-scale feature map. The fused multi-scale feature map is then used by the detection head to output the target detection result. in, The backbone network includes multiple alternating edge-aware downsampling convolutional modules and variable Sobel convolution inversion modules; The edge-aware downsampling convolution module extracts compressed features and fine-grained features through a dual-branch structure, and outputs the downsampling feature map to the variable Sobel convolution inversion module; The variable Sobel convolution inversion module dynamically adjusts the direction and size of the convolution kernel through variable Sobel convolution, and combines it with the inverted residual structure to obtain an edge enhancement feature map.

2. The method according to claim 1, characterized in that, The variable Sobel convolution inversion module includes variable Sobel convolution units and inversion units, wherein: The variable Sobel convolutional unit is used to generate Sobel operators at different angles through a rotation matrix, and to perform multi-directional edge detection on the input feature map to obtain multi-directional edge response features; The inverted unit is used to expand the feature channels, perform depthwise separable convolution, compress them back to the original dimension, and perform residual connection with the input features to obtain semantically enhanced features. The multi-directional edge response features and the semantic enhancement features are normalized and summed to obtain the output features.

3. The method according to claim 2, characterized in that, In the variable Sobel convolution unit, the smoothing operator coefficients S m Simulated Gaussian smoothing is used to remove noise through the difference operator coefficients D. m Calculate the gradient using the smoothing operator coefficients S. m With difference operator coefficient D m Construct the weight matrix by performing an outer product combination, where the weight matrix is ​​located in the zero-degree direction of the horizontal axis: ， ， ， ， Where L is the kernel size and m is the index of the window along the horizontal axis. It is a defined function. This represents the local pixel grayscale value of the image at window coordinates (m, n), where M represents the original matrix. This is the output gradient response.

4. The method according to claim 1, characterized in that, The backbone network ends with a spatial pyramid shared convolutional module, which aggregates global context information and uses dilated convolutions with shared weights to construct a spatial pyramid structure and extract multi-scale context features.

5. The method according to claim 4, characterized in that, The spatial pyramid shares a convolutional module: The first 1×1 convolution is used to compress the input channel; Three parallel dilated convolution branches share the convolution kernel weights and have different dilation rates and padding strategies, with the dilation rates set to 1, 3, and 5, respectively. The splicing module splices the outputs of the three parallel dilated convolution branches with the compressed original features along the channel dimension. The second 1×1 convolutional fusion is used to output multi-scale perceptual features.

6. The method according to claim 1, characterized in that, The edge-aware downsampling convolutional module includes a main path and an auxiliary path, wherein... The main path compresses the channel through a 1×1 convolution, and then uses a depthwise separable convolution with a stride of 2 for spatial downsampling; The auxiliary path directly processes the original input using a standard convolution with a stride of 2 to preserve fine-grained features; The output features of the main path and the output features of the auxiliary path are concatenated to form the downsampled feature map x. out : ， in, This represents the tensor concatenation operation. The feature map comes from the main path. The feature map comes from the auxiliary path.

7. The method according to claim 1, characterized in that, The backbone network, the neck network, and the detection head are built based on the YOLO architecture; The detection head uses the edge-aware downsampling convolution module to extract positional features and combines them with the classification branch to output the category and position of the predicted bounding box.

8. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

9. A computing device, characterized in that, include: processor; as well as A memory storing a computer program that, when executed by the processor, implements the method as described in any one of claims 1-8.

10. A drone, intelligent driving vehicle, or monitoring device, characterized in that, Includes the computing device according to claim 9.

Images