Multi-scale visible light target detection method and system

Abstract

The invention discloses a multi-scale visible light target detection method and system, and relates to the technical field of computer vision, and the detection method comprises the following steps: generating a dynamic filtering kernel direction and scale parameter based on the content of an input image, predicting the filtering kernel parameter through a lightweight sub-network, and carrying out the detection of a multi-scale visible light target. The method comprises the steps of generating a dynamic filtering kernel with the self-adaptive direction and scale, enabling the dynamic filtering kernel to act on an input image, strengthening the feature response of a key region in combination with a channel-space double-attention mechanism, and generating a multi-scale feature map, and has the advantages that the dynamic filtering kernel can be flexibly adjusted according to the target form and background complexity; according to the method, the feature response of a key region is effectively enhanced, noise interference is suppressed, the problem of detail loss or noise sensitivity caused by a traditional fixed filtering kernel is solved, meanwhile, an attention mechanism is further focused on a target salient region, the feature expression robustness under complex illumination and shielding scenes is improved, and the detection stability of a multi-scale target is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a multi-scale visible light target detection method and system. Background Art

[0002] Visible light target detection, as one of the core tasks of computer vision, is widely used in security monitoring, autonomous driving, industrial inspection and other fields. However, in real scenes, the significant differences in target scale, sparse feature information of small targets, and complex and changeable ambient lighting and occlusion pose severe challenges to the robustness of detection algorithms. Traditional methods usually rely on fixed-parameter filter kernels when extracting multi-scale features, which makes it difficult to adaptively adjust the direction and scale according to the input content, resulting in insufficient response of key features or increased background noise interference. In addition, although existing feature pyramid networks integrate multi-scale information, the efficiency of combining shallow details with deep semantic features is limited. Cross-level information transmission is prone to redundancy or loss, affecting target positioning accuracy in complex scenes.

[0003] At the same time, static detection threshold design is difficult to adapt to dynamic environmental changes and is prone to missed detections or false detections. Due to the low resolution and weak feature expression of small targets, conventional methods often suffer from information loss during the sampling process or the high computational overhead of super-resolution reconstruction, making it difficult to balance detection accuracy and real-time performance. Although existing technologies attempt to optimize the above problems through multi-stage networks or complex models, they generally have defects such as high computing resource consumption and insufficient generalization capabilities. Therefore, there is an urgent need for a solution that can dynamically generate filter kernel parameters, efficiently fuse cross-scale features, adaptively adjust detection thresholds, and achieve lightweight enhancement for small targets, so as to comprehensively improve the performance and practicality of visible light target detection in complex scenarios. Summary of the Invention

[0004] The object of the present invention is to provide a multi-scale visible light target detection method and system.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a multi-scale visible light target detection method, the detection method comprising the following steps:

[0006] Step 1: Generate dynamic filter kernel direction and scale parameters based on the input image content, predict the filter kernel parameters through a lightweight sub-network, and generate a dynamic filter kernel with adaptive direction and scale;

[0007] Step 2: Apply the dynamic filter kernel to the input image, combine the channel-spatial dual attention mechanism to enhance the feature response of key areas, and generate a multi-scale feature map;

[0008] Step 3: A lightweight multi-scale feature pyramid network is used to fuse shallow details with deep semantic features, and a feature backtracking module is used to stitch feature maps of different levels across scales.

[0009] Step 4: Dynamically generate detection threshold parameters based on the meta-learning model, and adjust the filter kernel scale and feature fusion weight in real time in combination with the environment perception module;

[0010] Step 5: Perform super-resolution reconstruction on the small target area, jointly optimize the detection loss and super-resolution reconstruction loss, and output the target detection result.

[0011] As a further solution of the present invention: in the step of generating the dynamic filter kernel, the lightweight sub-network includes a 1×1 convolutional layer and a fully connected layer, outputs a filter kernel direction angle θ∈[0°,360°) and a scale σ∈[0.5,2.0], and the generation formula of the dynamic filter kernel is:

[0012] K dynamic =Softmax(W θ ·F in +b θ )

[0013] Among them, W θ and b θ is a learnable parameter, F in is the input feature map.

[0014] As a further solution of the present invention: the backbone network of the lightweight multi-scale feature pyramid network adopts a Ghost module, the Ghost ratio is 2:1, and the calculation formula of its output feature map is:

[0015] F ghost =Concat(F base ,DWConv(F base ))

[0016] Among them, Concat refers to the feature map splicing operation, F base is the traditional convolution output, and DWConv is the depth-wise separable convolution.

[0017] As a further solution of the present invention: the meta-learning model is pre-trained using the MAML algorithm, the input feature is a 256×256×32-dimensional tensor, and the output dynamic threshold T dynamic The generation formula is:

[0018] T dynamic =σ(W meta ·F feature +b meta )

[0019] Among them, σ is the Sigmoid function, W meta and b meta Meta-learning parameter, T dynamic ∈[0.1,0.9],Ffeature Input features to the meta-learning model.

[0020] As a further solution of the present invention, the normalized Gaussian Wasserstein distance (NWD) is used to calculate the positioning loss for the small target area, and the formula is:

[0021]

[0022] Where C is the normalization constant, exp(·) represents the natural exponential function, and b pred represents the bounding box predicted by the model, b gt Represents the true bounding box, NWD is defined as:

[0023]

[0024] Among them, μ pred 、μ gt The center coordinates of the predicted box and the real box, σ pred , σ gt The standard deviation of the Gaussian distribution between the predicted box and the true box.

[0025] The present invention also provides a multi-scale visible light target detection system, which includes the following modules:

[0026] Dynamic filter kernel generation module: used to predict the filter kernel direction and scale parameters according to the input image content and generate an adaptive dynamic filter kernel;

[0027] Attention-guided feature extraction module: Integrates a channel-spatial dual attention mechanism to enhance feature responses in key areas and suppress noise;

[0028] Lightweight feature pyramid module: Based on the backbone network optimized by neural architecture search, it integrates shallow and deep features across scales through the feature backtracking module;

[0029] Adaptive threshold adjustment module: uses a meta-learning model to dynamically generate detection thresholds and optimizes parameters in real time based on environmental sensor data;

[0030] Small target enhancement module: Embeds a lightweight super-resolution network to jointly optimize detection and super-resolution tasks to improve the accuracy of small target detection.

[0031] As a further solution of the present invention: the dynamic filter core generation module supports multi-core parallel computing, and its filtering operation formula is:

[0032]

[0033] Where k is the radius of the filter kernel, and the time required to implement single image filtering on the GPU is ≤3ms.

[0034] As a further solution of the present invention: in the skip connection path of the lightweight feature pyramid module, the shallow feature map F low With the deep feature map F high The fusion formula is:

[0035] F high =Conv 1×1 (Concat(Deconv(F high ),F low ))

[0036] Deconv(·) represents the deconvolution operation, where the number of channels is compressed to 256 dimensions.

[0037] As a further solution of the present invention: the super-resolution network of the small object enhancement module includes 4 residual blocks, and the residual is calculated as:

[0038] F sr =F in +Conv 3×3 (ReLU(Conv 3×3 (F in )))

[0039] The joint loss function is defined as:

[0040] L total =0.9·L det +0.1·||F sr -F hr ||1.

[0041] As a further solution of the present invention: when the system is deployed on an edge device, it supports INT8 quantization, and its inference speed meets the following requirements:

[0042]

[0043] Where α = 1.2 × 10 -6 , β = 0.5, ≥ 30 FPS on the NVIDIA Jetson Xavier platform.

[0044] By adopting the above technical solution, compared with the prior art, the beneficial effects of the present invention are:

[0045] 1. This invention dynamically generates the direction and scale parameters of the filter kernel through a lightweight sub-network, and combines it with a channel-spatial dual attention mechanism to achieve adaptive feature enhancement of the input image content. The dynamic filter kernel can be flexibly adjusted according to the target morphology and background complexity, effectively enhancing the feature response of key areas and suppressing noise interference, solving the problems of detail loss or noise sensitivity caused by traditional fixed filter kernels. At the same time, the attention mechanism further focuses on the salient areas of the target, improving the robustness of feature expression in complex lighting and occlusion scenarios, and significantly improving the detection stability of multi-scale targets.

[0046] 2. This paper constructs a lightweight multi-scale feature pyramid network based on the Ghost module, and uses the feature backtracking module to cross-level splicing of shallow details and deep semantic features to achieve efficient feature fusion and information transfer. This design reduces redundant calculations in traditional feature pyramid networks, while retaining multi-scale details through skip connections, enhancing the ability to distinguish small targets from complex backgrounds. The cross-scale fusion mechanism effectively balances positioning accuracy and computational efficiency, reducing model complexity while improving the detection accuracy of multi-scale targets.

[0047] 3. The present invention introduces a meta-learning model to dynamically generate detection threshold parameters, and combines it with the environmental perception module to optimize the filter kernel scale and feature weights in real time, so that the detection system can adapt to dynamic factors such as illumination changes and target density, reducing missed detection and false detection rates. In addition, the lightweight super-resolution network jointly optimizes detection and reconstruction tasks, and performs feature enhancement on small target areas, thereby improving resolution and feature expression capabilities while avoiding high computational overhead, significantly improving the detection accuracy and real-time performance of small targets, and meeting the deployment requirements of edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of the multi-scale visible light target detection method of the present invention. DETAILED DESCRIPTION

[0049] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0050] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0051] Please see the attached Figure 1 The present invention provides a multi-scale visible light target detection method, the detection method comprising the following steps:

[0052] Step 1: Generate dynamic filter kernel direction and scale parameters based on the input image content, predict the filter kernel parameters through a lightweight sub-network, and generate a dynamic filter kernel with adaptive direction and scale;

[0053] Step 2: Apply the dynamic filter kernel to the input image, combine the channel-spatial dual attention mechanism to enhance the feature response of key areas, and generate a multi-scale feature map;

[0054] Step 3: A lightweight multi-scale feature pyramid network is used to fuse shallow details with deep semantic features, and a feature backtracking module is used to stitch feature maps of different levels across scales.

[0055] Step 4: Dynamically generate detection threshold parameters based on the meta-learning model, and adjust the filter kernel scale and feature fusion weight in real time in combination with the environment perception module;

[0056] Step 5: Perform super-resolution reconstruction on the small target area, jointly optimize the detection loss and super-resolution reconstruction loss, and output the target detection result.

[0057] In one embodiment of the present invention, in the dynamic filter kernel generation step, the lightweight sub-network includes a 1×1 convolutional layer and a fully connected layer, and outputs a filter kernel direction angle θ∈[0°,360°) and a scale σ∈[0.5,2.0]. The dynamic filter kernel generation formula is:

[0058] K dynamic =Softmax(W θ ·F in +b θ )

[0059] Among them, W θ and b θ is a learnable parameter, F in is the input feature map.

[0060] In one embodiment of the present invention, the backbone network of the lightweight multi-scale feature pyramid network adopts the Ghost module, the Ghost ratio is 2:1, and the calculation formula of its output feature map is:

[0061] F ghost =Concat(F base ,DWConv(F base ))

[0062] Among them, Convcat refers to the feature map concatenation operation, F base is the traditional convolution output, and DWConv is the depth-wise separable convolution.

[0063] In one embodiment of the present invention, the meta-learning model is pre-trained using the MAML algorithm, the input feature is a 256×256×32 dimensional tensor, and the output dynamic threshold Tdynamic The generation formula is:

[0064] T dynamic =σ(W meta ·F feature +b meta )

[0065] Among them, σ is the Sigmoid function, W meta and b meta Meta-learning parameter, T dynamic ∈[0.1,0.9],F feature Input features to the meta-learning model.

[0066] In one embodiment of the present invention, the normalized Gaussian Wasserstein distance (NWD) is used to calculate the positioning loss for the small target area, and the formula is:

[0067]

[0068] Where C is the normalization constant, exp(·) represents the natural exponential function, and b pred represents the bounding box predicted by the model, b gt Represents the true bounding box, NWD is defined as:

[0069]

[0070] Among them, μ pred 、μ gt The center coordinates of the predicted box and the real box, σ pred , σ gt The standard deviation of the Gaussian distribution between the predicted box and the true box.

[0071] The present invention also provides a multi-scale visible light target detection system, which includes the following modules:

[0072] Dynamic filter kernel generation module: used to predict the filter kernel direction and scale parameters according to the input image content and generate an adaptive dynamic filter kernel;

[0073] Attention-guided feature extraction module: Integrates a channel-spatial dual attention mechanism to enhance feature responses in key areas and suppress noise;

[0074] Lightweight feature pyramid module: Based on the backbone network optimized by neural architecture search, it integrates shallow and deep features across scales through the feature backtracking module;

[0075] Adaptive threshold adjustment module: uses a meta-learning model to dynamically generate detection thresholds and optimizes parameters in real time based on environmental sensor data;

[0076] Small target enhancement module: Embeds a lightweight super-resolution network to jointly optimize detection and super-resolution tasks to improve the accuracy of small target detection.

[0077] In one embodiment of the present invention, the dynamic filter core generation module supports multi-core parallel computing, and its filtering operation formula is:

[0078]

[0079] Where k is the radius of the filter kernel, and the time required to implement single image filtering on the GPU is ≤3ms.

[0080] In one embodiment of the present invention: in the skip connection path of the lightweight feature pyramid module, the shallow feature map F low With the deep feature map F high The fusion formula is:

[0081] F high =Conv 1×1 (Concat(Deconv(F high ),F low ))

[0082] Deconv(·) represents the deconvolution operation, where the number of channels is compressed to 256 dimensions.

[0083] In one embodiment of the present invention, the super-resolution network of the small object enhancement module includes four residual blocks, and the residual is calculated as:

[0084] F sr =F in +Conv 3×3 (ReLU(Conv 3×3 (F in )))

[0085] The joint loss function is defined as:

[0086] L total =0.9·L det +0.1·||F sr -F hr ||1.

[0087] In one embodiment of the present invention, when the system is deployed on an edge device, it supports INT8 quantization, and its inference speed meets the following requirements:

[0088]

[0089] Where α = 1.2 × 10 -6 , β = 0.5, ≥ 30 FPS on the NVIDIA Jetson Xavier platform.

[0090] Example 1: Multi-scale target detection in security monitoring scenarios

[0091] Scenario Description

[0092] Input data: 1920×1080 resolution real-time surveillance video frames, containing multi-scale targets such as pedestrians, vehicles, and suspicious objects.

[0093] Core challenges: interference from noise from swaying leaves, low resolution of small targets (such as suspicious objects), and frequent lighting changes.

[0094] Technical Implementation

[0095] 1. Dynamic filter kernel generation

[0096] Lightweight sub-network prediction parameters: direction angle θ = 45°, scale σ = 1.2.

[0097] Filter kernel formula:

[0098] K dynamic =Softmax(W θ ·F in +b θ )

[0099] 2. Feature fusion and enhancement

[0100] Ghost module output: F ghost =Concat(F base ,DWConv(F base ))(Ratio 2:1).

[0101] Cross-scale stitching:

[0102] F high =Conv 1×1 (Concat(Deconv(F high ),F low ))

[0103] (Shallow F low : 128×128×64, deep F high :32×32×256)

[0104] 3. Adaptive Optimization

[0105] Meta-learning model generation threshold: T dynamic =0.7, and dynamically adjust the weight based on the light sensor.

[0106] Super-resolution network: 4 residual blocks, joint loss: L total =0.9·L det +0.1·||F sr -Fhr ||1.

[0107] Effect verification

[0108] Accuracy improvement: Small target detection accuracy +15%, missed detection rate 3.2%.

[0109] Real-time performance: The NVIDIA Jetson Xavier platform has an inference speed of 28 FPS.

[0110] Example 2: Real-time target detection in autonomous driving

[0111] Scenario Description

[0112] Input data: 1280×720 resolution road image, objects include vehicles, pedestrians, and traffic signs.

[0113] Core challenges: Rain and fog interference, large scale differences of distant targets, and high real-time requirements.

[0114] Technical Implementation

[0115] 1. Multi-core parallel filtering

[0116] Dynamic parameters: θ = [30°, 90°], σ = [1.0, 1.5], kernel radius k = 3.

[0117] Filtering operation:

[0118]

[0119] GPU-side single frame time consumption: 2.8ms.

[0120] 2. Lightweight Feature Pyramid

[0121] Ghost module output: F ghost Dimensions: 64×64×128.

[0122] Neural architecture search optimizes the backbone network and suppresses background interference.

[0123] 3. Dynamic threshold adjustment

[0124] Meta-learning model output: T dynamic = 0.6, and the scale parameters are optimized by fusing lidar data.

[0125] Effect verification

[0126] Improved accuracy: Traffic sign detection accuracy reaches 92.7%.

[0127] Deployment performance: After INT8 quantization, the edge device inference speed is 35FPS, fully meeting real-time requirements.

[0128] Example 3: Small Target Defect Recognition in Industrial Inspection

[0129] Scenario Description

[0130] Input data: 512×512 resolution macro image of a PCB board, with the target being a solder joint crack (size <10×10 pixels).

[0131] Core challenges: extremely small defect size, complex texture, and strict false detection rate control.

[0132] Technical Implementation

[0133] 1. Small target enhancement

[0134] Super-resolution network: 4x upsampling, residual calculation:

[0135] Positioning loss function:

[0136] 2. Dynamic filtering and fusion

[0137] Filter kernel scale: σ = 0.8, enhances texture details.

[0138] Feature pyramid cross-level fusion: shallow F low (256×256×32) and deep F high : (64×64×256) splicing, channel compression to 256 dimensions.

[0139] Effect verification

[0140] Improved accuracy: small target positioning error -18%, defect detection rate 98.3%.

[0141] Robustness: False detection rate <1.5%, inference speed ≥30FPS after INT8 quantization on edge devices.

[0142] Although the present invention is disclosed above with reference to preferred embodiments, this is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, any modifications, equivalent variations, and modifications made to the above embodiments in accordance with the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection defined by the claims of the present invention.

Claims

1. A multi-scale visible light target detection method, characterized by: The detection method comprises the following steps: Step 1: Generate dynamic filter kernel direction and scale parameters based on the input image content, predict the filter kernel parameters through a lightweight sub-network, and generate a dynamic filter kernel with adaptive direction and scale; Step 2: Apply the dynamic filter kernel to the input image, combine the channel-spatial dual attention mechanism to enhance the feature response of key areas, and generate a multi-scale feature map; Step 3: A lightweight multi-scale feature pyramid network is used to fuse shallow details with deep semantic features, and a feature backtracking module is used to stitch feature maps of different levels across scales. Step 4: Dynamically generate detection threshold parameters based on the meta-learning model, and adjust the filter kernel scale and feature fusion weight in real time in combination with the environment perception module; Step 5: Perform super-resolution reconstruction on the small target area, jointly optimize the detection loss and super-resolution reconstruction loss, and output the target detection result.

2. The multi-scale visible light target detection method according to claim 1, characterized in that: In the dynamic filter kernel generation step, the lightweight sub-network includes a 1×1 convolutional layer and a fully connected layer, and outputs a filter kernel direction angle θ∈[0°,360°) and a scale σ∈[0.5,2.0]. The generation formula of the dynamic filter kernel is: K dynamic =Softmax(W θ ·F in +b θ ) Among them, W θ and b θ is a learnable parameter, F in is the input feature map.

3. The multi-scale visible light target detection method according to claim 1, wherein: The backbone network of the lightweight multi-scale feature pyramid network adopts the Ghost module with a Ghost ratio of 2:

1. The calculation formula of its output feature map is: F ghost =Concat(F base ,DWConv(F base )) Among them, Convcat refers to the feature map concatenation operation, F base is the traditional convolution output, and DWConv is the depth-wise separable convolution.

4. The multi-scale visible light target detection method according to claim 1, wherein: The meta-learning model is pre-trained using the MAML algorithm, with the input feature being a 256×256×32-dimensional tensor and the output being a dynamic threshold T. dynamic The generation formula is: T dynamic =σ(W meta ·F feature +b meta ) Among them, σ is the Sigmoid function, W meta and b meta Meta-learning parameter, T dynamic ∈[0.1,0.9],F feature Input features to the meta-learning model.

5. The multi-scale visible light target detection method according to claim 1, characterized in that: For small target areas, the normalized Gaussian Wasserstein distance is used to calculate the positioning loss, and the formula is: Where C is the normalization constant, exp(·) represents the natural exponential function, and b pred represents the bounding box predicted by the model, b gt Represents the true bounding box, NWD is defined as: Among them, μ pred 、μ gt The center coordinates of the predicted box and the real box, σ pred , σ gt The standard deviation of the Gaussian distribution between the predicted box and the true box.

6. A multi-scale visible light target detection system applicable to any one of claims 1-5, characterized in that: The detection system includes the following modules: Dynamic filter kernel generation module: used to predict the filter kernel direction and scale parameters according to the input image content and generate an adaptive dynamic filter kernel; Attention-guided feature extraction module: Integrates a channel-spatial dual attention mechanism to enhance feature responses in key areas and suppress noise; Lightweight feature pyramid module: Based on the backbone network optimized by neural architecture search, it integrates shallow and deep features across scales through the feature backtracking module; Adaptive threshold adjustment module: uses a meta-learning model to dynamically generate detection thresholds and optimizes parameters in real time based on environmental sensor data; Small target enhancement module: Embeds a lightweight super-resolution network to jointly optimize detection and super-resolution tasks to improve the accuracy of small target detection.

7. The multi-scale visible light target detection system according to claim 6, characterized in that: The dynamic filter core generation module supports multi-core parallel computing, and its filtering operation formula is: Where k is the radius of the filter kernel, and the time required to implement single image filtering on the GPU is ≤3ms.

8. The multi-scale visible light target detection system according to claim 6, characterized in that: In the skip connection path of the lightweight feature pyramid module, the shallow feature map F low With the deep feature map F high The fusion formula is: F high =Conv 1×1 (Concat(Deconv(F high ),F low )) Deconv(·) represents the deconvolution operation, where the number of channels is compressed to 256 dimensions.

9. The multi-scale visible light target detection system according to claim 6, characterized in that: The super-resolution network of the small object enhancement module contains 4 residual blocks, and its residual is calculated as: F sr =F in +Conv 3×3 (ReLU(Conv 3×3 (F in ))) The joint loss function is defined as: L total =0.9 L det +0.1·||F sr -F hr ||1.

10. The multi-scale visible light target detection system according to claim 6, characterized in that: When deployed on edge devices, the system supports INT8 quantization and its inference speed meets the following requirements: Where α = 1.2 × 10 -6 , β = 0.5, ≥ 30 FPS on the NVIDIA Jetson Xavier platform.

Images