A method and system for detecting non-ferrous metal targets in scrapped automobiles

Abstract

This invention belongs to the field of image processing technology and discloses a method and system for detecting non-ferrous metal targets in scrapped vehicles. The method constructs a non-ferrous metal target detection model for scrapped vehicles through deep coupling of a GAN dual-modal fusion network and a dual-backbone network, achieving efficient and accurate sorting of non-ferrous metals from scrapped vehicles. A dual-modal fusion network is introduced into the model input layer to generate a fused image that combines infrared thermal saliency and visible light texture features, which helps solve the problem of cross-modal information fragmentation. The dual-backbone network replaces standard convolution with a three-branch structure of the MGHCM module, simultaneously capturing the contours of large targets, the semantics of medium targets, and the details of small targets. Combined with dynamic feature routing using the DHFBlock module, it achieves adaptive splitting and efficient fusion of multi-scale features. The detection head network is optimized with a rotated bounding box prediction and cross-modal cross-entropy classification mechanism to adapt to scenarios where metal fragments are placed at arbitrary angles.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a method and system for detecting non-ferrous metal targets in scrapped automobiles. Background Technology

[0002] Non-ferrous metal target detection in end-of-life vehicles refers to the process of identifying and distinguishing non-ferrous metals (such as copper, aluminum, zinc, magnesium, etc.) from metal fragments or parts obtained after dismantling end-of-life vehicles. Accurately identifying and separating non-ferrous metals through target detection technology is of great significance for improving resource recycling rates, reducing environmental pollution, and lowering raw material costs.

[0003] With the rapid development of the automotive industry, the number of scrapped vehicles is increasing year by year. How to efficiently and accurately recycle and process non-ferrous metals from scrapped vehicles has become an important research topic. Non-ferrous metals such as copper and aluminum have high recycling value, but due to the differences in their physical and chemical properties, their sorting and detection are complicated.

[0004] Currently, the focus is mainly on eddy current, X-ray, spectroscopy, and machine vision. The common non-ferrous metal sorting methods for scrapped cars and their drawbacks are as follows: (1) Eddy current sorting method utilizes the difference in magnetic permeability between different metals to subject the metals to different magnitudes of Lorentz force in a magnetic field, thereby achieving classification. This method is suitable for the sorting of large-scale metal crushed materials and can effectively improve sorting efficiency and purity. However, the eddy current sorting method is less effective for sorting weakly magnetic metals (such as stainless steel) and needs to be combined with other methods for comprehensive sorting. (2) Gas chromatography-mass spectrometry (GCMS) is a destructive detection method that requires sample destruction. This method utilizes the different trajectories of different ions in an electric or magnetic field to separate ions according to their mass-to-charge ratio (M / Z), and performs qualitative and quantitative analysis of the substances in the sample through mass spectrometry and related information. However, since it is a destructive detection method, it requires sample destruction and is not suitable for all types of samples. (3) X-ray fluorescence spectrometry (XRF) in spectroscopic detection methods uses X-rays to excite elements in the sample and determines the type and content of elements by measuring the characteristic X-rays emitted. However, due to its low recognition rate, it is only suitable for preliminary screening and quantitative analysis. Ultraviolet-visible spectrophotometry (UV-Vis) in spectroscopic detection methods uses the absorption characteristics of substances to ultraviolet light for quantitative analysis. However, it is limited to the detection of specific elements such as hexavalent chromium. (4) Machine vision detection methods capture images of scrapped car metal fragments through cameras and use image processing algorithms to analyze the images and identify different types of non-ferrous metals. This method has the advantages of being non-contact, fast, and accurate. However, the current visual detection technology still has shortcomings in the application of non-ferrous metal target detection in scrapped cars in terms of data processing speed, light and environmental adaptability, defect identification ability, anti-interference ability, and algorithm scalability and adaptability. In summary, it is necessary to propose a method and system for detecting non-ferrous metal targets in scrapped cars to solve the above-mentioned technical problems in the existing technology. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for detecting non-ferrous metal targets in scrapped vehicles. By deeply coupling a GAN dual-modal fusion network and a dual backbone network, a non-ferrous metal target detection model for scrapped vehicles is constructed to achieve efficient and accurate sorting of non-ferrous metals in scrapped vehicles.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for detecting non-ferrous metal targets in scrapped automobiles includes the following steps:

[0008] Step 1: Simultaneously acquire visible light and infrared images of non-ferrous metal fragments from scrapped vehicles, and preprocess the acquired visible light and infrared images to obtain a training dataset;

[0009] Step 2: Build a non-ferrous metal target detection model for scrapped cars based on the improved YOLOv11 model, which includes a dual-modal fusion network, backbone network 1, backbone network 2, neck network and detection head network;

[0010] The dual-modal fusion network is based on generative adversarial networks and is used to receive and process visible light and infrared images to output a fused image that combines the saliency of infrared images with the texture of visible light images.

[0011] Backbone Network 1 and Backbone Network 2 are used to receive fused image and infrared image inputs and perform feature extraction, respectively. Backbone Network 1 replaces the standard convolution in the original backbone network with the MGHCM module to extract multi-granularity features, and replaces the CSPBlock module in the original backbone network with the DHFBlock module to achieve dynamic feature routing. At the same time, it replaces the SPPF module in the original backbone network with the SPPF_UniRepLKA module and adds the SE_C2F-CIB module to achieve lightweight channel enhancement. Backbone Network 2, based on Backbone Network 1, further adds a DCBF fusion module to correct cross-branch positional deviations of infrared features.

[0012] The neck network is used to receive and fuse the output features of the first and second backbone networks. The neck network replaces the traditional fusion convolution in the original neck network with the MADF fusion module to solve multi-scale misalignment, replaces the ordinary upsampling in the original neck network with the DDU module to enhance small target details, and adds the CMAFEM fusion module to achieve cross-modal secondary fusion.

[0013] Step 3: Train the model based on the training dataset, and finally perform target detection on non-ferrous metals in scrapped cars.

[0014] A non-ferrous metal target detection system for end-of-life vehicles includes a visible light camera, an infrared thermal imager, and a computer device. The visible light camera captures visible light images and uploads them to the computer device, while the infrared thermal imager captures infrared images and uploads them to the computer device. The computer device includes a memory and one or more processors. Executable code is stored in the memory. When the processor executes the executable code, it implements the steps of the aforementioned method for detecting non-ferrous metal targets in end-of-life vehicles.

[0015] Compared with the prior art, the present invention has the following advantages:

[0016] As described above, the present invention discloses a method and system for detecting non-ferrous metal targets in scrapped vehicles. Through multi-module collaborative innovation, it achieves systematic optimization of non-ferrous metal target detection in scrapped vehicles: A dual-modal fusion network is introduced to generate a fused image that combines infrared thermal saliency and visible light texture features, solving the problem of cross-modal information fragmentation and providing a multi-dimensional feature foundation for subsequent detection; Backbone Network 1 and Backbone Network 2 replace standard convolution with the three-branch structure of the MGHCM module to construct a multi-granularity feature extraction mechanism, simultaneously capturing the contours of large targets, the semantics of medium targets, and the details of small targets. Combined with the dynamic feature routing of the DHFBlock module, it overcomes the feature utilization bottleneck of the traditional CSP module, achieving adaptive splitting and efficient fusion of multi-scale features; Infrared-dedicated backbone (Backbone Network 2) A new DCBF fusion module is added to correct cross-branch feature bias through spatial offset prediction, solving the problem of inaccurate positioning of infrared features due to deformation. Meanwhile, the SE_C2F-CIB module enhances channel discriminativeness with a lightweight design, balancing computational efficiency and feature representation capability. The neck network uses the MADF fusion module for alignment fusion to resolve multi-scale feature position bias, and the DDU module overcomes the spatial rigidity limitations of traditional interpolation through dynamic deformable upsampling. Combined with the CMAFEM fusion module, a two-stage enhancement paradigm of "pixel-level coarse fusion → feature-level fine fusion" is constructed, achieving accurate recalibration of cross-modal features. The detection head network is optimized with a rotating box prediction and cross-modal cross-entropy classification mechanism to adapt to scenarios where metal fragments are placed at arbitrary angles, enhancing the semantic differentiation capability of similar metals. The overall architecture of this invention, through the collaborative design of modules, constructs a dual-system collaborative paradigm of rapid edge detection and high-precision cloud analysis, overcoming the efficiency-accuracy contradiction of traditional single-modal detection and providing a systematic solution for intelligent sorting of non-ferrous metals in complex scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below.

[0018] Figure 1 This is a network structure diagram of the non-ferrous metal target detection model for scrapped vehicles in Embodiment 1 of the present invention;

[0019] Figure 2 This is a network structure diagram of the generator in Embodiment 1 of the present invention;

[0020] Figure 3 This is a network structure diagram of the discriminator in Embodiment 1 of the present invention;

[0021] Figure 4 This is a network structure diagram of the MGHCM module in Embodiment 1 of the present invention;

[0022] Figure 5This is a network structure diagram of the DHFBlock module in Embodiment 1 of the present invention;

[0023] Figure 6 This is a network structure diagram of the DCBF fusion module in Embodiment 1 of the present invention;

[0024] Figure 7 This is a network structure diagram of the SE_C2F-CIB module in Embodiment 1 of the present invention;

[0025] Figure 8 This is a flowchart of the compression operation in Embodiment 1 of the present invention;

[0026] Figure 9 This is a flowchart of the excitation operation in Embodiment 1 of the present invention;

[0027] Figure 10 This is a network structure diagram of the SPPF_UniRepLKA module in Embodiment 1 of the present invention;

[0028] Figure 11 This is a network structure diagram of the CMAFEM fusion module in Embodiment 1 of the present invention;

[0029] Figure 12 This is a network structure diagram of the DDU module in Embodiment 1 of the present invention;

[0030] Figure 13 This is a network structure diagram of the MADF fusion module in Embodiment 1 of the present invention;

[0031] Figure 14 This is a flowchart of System 1 in Embodiment 1 of the present invention;

[0032] Figure 15 This is a flowchart of System 2 in Embodiment 1 of the present invention. Detailed Implementation

[0033] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0034] Example 1

[0035] This embodiment describes a method for detecting non-ferrous metal targets in scrapped vehicles, which includes the following steps:

[0036] Step 1: Simultaneously acquire images of the non-ferrous metal fragments from the scrapped vehicle using a visible light camera and an infrared thermal imager to obtain visible light and infrared images (infrared thermal imaging). Then, preprocess the visible light and infrared images to obtain a training dataset. In this embodiment, the visible light camera is configured with a resolution of 640×640 and a frame rate of 30 FPS to capture metal texture and color. The infrared thermal imager is configured with NETD < 20 mK, a resolution of 640×640, an emissivity of 0.89, and a heating stage temperature of 322℃ to optimize the thermal imaging recognition rate. This embodiment uses AnalyzlR software to achieve time synchronization between the visible light and infrared images (error < 10 ms) to ensure cross-modal feature alignment.

[0037] Preprocessing includes geometric registration and data augmentation (dual-system differentiation). Geometric registration, through SURF feature point matching and the RANSAC algorithm, ensures pixel-level alignment error <0.5 pixels, providing a spatial consistency basis for the dual-system approach.

[0038] Data augmentation includes pixel-level system augmentation and feature-level system augmentation. Pixel-level system augmentation is fused image augmentation (rotation ±15°, scaling 0.8-1.2 times, Gaussian blur σ=1.5), and feature-level system augmentation is dual-modal synchronous augmentation (visible light image and infrared image rotate / flip at the same scale to maintain spatial consistency between modalities).

[0039] Step 2: Construct a non-ferrous metal target detection model for scrapped vehicles based on the improved YOLOv11 model. This model includes a dual-modal fusion network, backbone network one, backbone network two, a neck network, and a detection head network, as follows: Figure 1 As shown.

[0040] The dual-modal fusion network, based on a generative adversarial network (GAN), receives and processes visible light and infrared images to output a fused image that combines the saliency of the infrared image with the texture of the visible light image. Backbone network one and backbone network two are used to receive the fused image and infrared image inputs, respectively, and perform feature extraction. Backbone network one replaces the standard convolution in the original backbone network with the MGHCM module to extract multi-granularity features, replaces the CSPBlock module in the original backbone network with the DHFBlock module to achieve dynamic feature routing, replaces the SPPF module in the original backbone network with the SPPF_UniRepLKA module, and adds the SE_C2F-CIB module to achieve lightweight channel enhancement. Backbone network two, based on backbone network one, further adds a DCBF fusion module to correct cross-branch positional biases in infrared features. The neck network is used to receive and fuse the output features of the first and second backbone networks. The neck network replaces the traditional fusion convolution in the original neck network with the MADF fusion module to solve multi-scale misalignment, replaces the ordinary upsampling in the original neck network with the DDU module to enhance small target details, and adds the CMAFEM fusion module to achieve cross-modal secondary fusion.

[0041] The detection head network optimizes the classification branch Focal Loss and cross-modal cross-entropy in the original YOLOv11 model's detection head network; the regression branch supports rotated box prediction, so as to comprehensively improve the cross-modal detection capability.

[0042] Step 3: Train the model based on the training dataset, and finally perform target detection on non-ferrous metals in scrapped cars.

[0043] The following section provides a detailed explanation of each network in the non-ferrous metal target detection model for scrapped vehicles, based on the improvements mentioned above.

[0044] To obtain a fused image of infrared and visible light images, a dual-modal fusion network (DMU) based on attention and GAN is constructed. The network consists of two main parts: a generator and a discriminator. The discriminator includes an infrared discriminator and a visible light discriminator. As an end-to-end network structure, the DMU takes infrared and visible light images as inputs and outputs the fused image. The DMU is a dual-path network structure, with two paths extracting texture and contrast information respectively to achieve cross-modal feature extraction. Furthermore, a self-attention mechanism is added to the generator to overcome the limitation of the number of convolutional layers on long-distance dependencies on feature locations, thereby improving the quality of the generated fused image. First, the infrared and visible light images are registered to ensure a one-to-one correspondence in time and space. Then, the registered infrared and visible light images are fed into the generator in pairs, and dual encoders are used to extract the infrared and visible light features from the infrared and visible light images.

[0045] The structure of the generator is as follows Figure 2 As shown, due to the different information contained in infrared and visible light images, the generator adopts a dual-path structure. The network structures of the two paths are identical, and their parameters are independent, which effectively reduces the difficulty of training the model. Infrared and visible light images are each input into one path, and the output features of the two paths are then fused within the feature fusion module. Each path includes a feature extraction module and a CBAM module. The input features of each path are processed sequentially through the feature extraction module and the CBAM module. Each feature extraction module consists of four convolutional (Conv) layers; the first two layers use 5×5 kernels, and the last two layers use 3×3 kernels. The output feature map of the feature extraction module is input into the CBAM module, where it undergoes spatial and channel attention processing to obtain feature maps of the same size and channel size. Then, the feature maps extracted from the two paths are input into the feature fusion module for fusion, which uses a 1×1 convolutional kernel and the Tanh activation function.

[0046] The fused image generated by the generator is then input into the discriminator. Specifically, the fused image generated by the generator is input into the infrared discriminator and the visible light discriminator, respectively, along with the infrared image and the visible light image, to establish adversarial training with the generator network.

[0047] The structure of the discriminator is as follows Figure 3As shown, the infrared discriminator and the visible light discriminator have completely identical structures, both containing six sequentially connected convolutional layers. The parameters and processing flow of each layer are as follows: The first convolutional layer uses a 4×4 kernel with 32 kernels, a stride of s=2, and padding of 1. After the convolution operation, a batch normalization layer and a LeakyReLU activation function are sequentially applied to achieve the first downsampling and initially extract the low-level features (such as edges and textures) of the input image. The second convolutional layer uses a 4×4 kernel with 64 kernels, a stride of s=2, and padding of 1. After convolution, it also undergoes batch normalization and LeakyReLU processing for further downsampling and extraction of richer mid-level features (such as local structures). The third convolutional layer uses a 4×4 kernel with 128 kernels, a stride of s=2, and padding of 1. After batch normalization and LeakyReLU processing, the feature dimension is further compressed after downsampling, focusing on the semantic features of the target. The fourth convolutional layer uses a 4×4 kernel with 128 kernels, a stride of s=2, and padding of 1. Batch normalization and LeakyReLU are used to enhance feature representation, at which point the feature map reflects the overall contour and modal characteristics of the target. The fifth convolutional layer uses a 4×4 kernel with 256 kernels, a stride of s=2, and padding of 1. After batch normalization and LeakyReLU processing, key features related to cross-modal fusion are deeply extracted, providing support for the final discrimination. The sixth convolutional layer uses a 4×4 kernel with 1 kernel, a stride of s=1, and padding of 0. Without batch normalization and activation functions, it directly outputs a feature map matrix (each pixel corresponds to a 70×70 patch region in the original image), used to determine whether the input sample is a real image or a fused image generated by the generator. The six convolutional layers achieve feature compression through progressive downsampling (stride 2), and combine batch normalization and LeakyReLU to enhance training stability, ultimately completing the discrimination of image authenticity. Training terminates when the two discriminators cannot distinguish whether the input sample comes from an image generated by the generator or a real image, thus obtaining the final fused image.

[0048] The bimodal fusion network generates a fused image that preserves both thermal radiation and texture through adversarial training, solving the problem of cross-modal information fragmentation. The dual-discriminator architecture (infrared discriminator + visible light discriminator) forces the fused image to simultaneously satisfy thermal saliency and texture detail, providing high-quality single-channel input for subsequent detection. The bimodal fusion network supports "fast single-channel detection" on the edge, reducing data dimensionality (3 channels → 1 channel) and improving inference speed by 25%. Simultaneously, it preserves the original infrared image for secondary fusion in the cloud, achieving a fundamental separation between efficiency and accuracy.

[0049] Subsequently, the fused image (a fusion of infrared and visible light images) and the infrared spectrum (infrared image) are input into backbone network one and backbone network two, respectively. The fused image not only makes the target stand out in the image but also retains the structural features and texture details of the visible light image, while the infrared spectrum can detect the temperature and thermal properties of the object. The combination of the two provides a more comprehensive understanding of the target object, improving the accuracy of recognition and analysis, especially in complex environments or special application scenarios. Backbone network one is used to quickly extract "multi-scale lightweight features" from the fused image, adapting to edge-side fast detection systems (inference speed ≥30FPS). Backbone network two, based on the architecture of backbone network one, embeds a DCBF fusion module (deformable cross-branch fusion module) to correct cross-branch positional deviations of infrared features and enhance the deformation adaptation capability of thermal radiation features; the functions of the remaining modules are the same as those of backbone network one, but feature extraction focuses more on "precise expression of infrared thermal targets". Backbone network two is used to output "high-precision infrared features", providing key input for the subsequent "feature-level secondary fusion" of the cloud-based high-precision system.

[0050] The backbone network consists of five modules: MGHCM, DHFBlock, SE_C2F-CIB, and SPPF_UniRepLKA. There are five MGHCM modules: the first, second, third, fourth, and fifth MGHCM modules; two DHFBlock modules: the first and second DHFBlock modules; two SE_C2F-CIB modules: the first and second SE_C2F-CIB modules; and one SPPF_UniRepLKA module: the first SPPF_UniRepLKA module.

[0051] The fused image input from the backbone network is processed sequentially through the first MGHCM module, the second MGHCM module, the first DHFBlock module, the third MGHCM module, the second DHFBlock module, the fourth MGHCM module, the first SE_C2F-CIB module, the fifth MGHCM module, the second SE_C2F-CIB module, and the first SPPF_UniRepLKA module to extract multi-scale lightweight features of the fused image.

[0052] Backbone Network 2 includes the MGHCM module, DHFBlock module, DCBF fusion module, SE_C2F-CIB module, and SPPF_UniRepLKA module. There are four MGHCM modules: the sixth, seventh, eighth, and ninth MGHCM modules; two DHFBlock modules: the third and fourth DHFBlock modules; three DCBF fusion modules: the first, second, and third DCBF fusion modules; four SE_C2F-CIB modules: the third, fourth, fifth, and sixth SE_C2F-CIB modules; and one SPPF_UniRepLKA module: the second SPPF_UniRepLKA module.

[0053] The infrared thermal image input to backbone network 2 is first processed sequentially through the sixth MGHCM module, the seventh MGHCM module, the third DHFBlock module, the eighth MGHCM module, and the fourth DHFBlock module. Then, the output features of the fourth DHFBlock module are fused with the output features of the second DHFBlock module within the first DCBF fusion module. The output features of the first DCBF fusion module are then processed sequentially through the third SE_C2F-CIB module, the ninth MGHCM module, and the fourth SE_C2F-CIB module. Finally, the fourth SE_C2F-CIB module... The output features of the IB module and the output features of the first SE_C2F-CIB module are fused in the second DCBF fusion module; the output features of the second DCBF fusion module are then processed by the fifth SE_C2F-CIB module, and the output features of the fifth SE_C2F-CIB module and the output features of the second SE_C2F-CIB module are fused in the third DCBF fusion module; the output features of the third DCBF fusion module are then processed sequentially by the sixth SE_C2F-CIB module and the second SPPF_UniRepLKA module to extract the features of the infrared thermal image.

[0054] The neck network includes a CMAFEM fusion module, an MGHCM module, a DDU module, a MADF fusion module, and a DHFBlock module. There are two CMAFEM fusion modules: the first and second CMAFEM fusion modules; three MGHCM modules: the tenth, eleventh, and twelfth MGHCM modules; two DDU modules: the first and second DDU modules; four MADF fusion modules: the first, second, third, and fourth MADF fusion modules; and four DHFBlock modules: the fifth, sixth, seventh, and eighth DHFBlock modules.

[0055] The output features of the first and second SPPF_UniRepLKA modules are input into the first CMAFEM fusion module for feature fusion. The output features of the first CMAFEM fusion module are then processed sequentially by the tenth MGHCM module and the first DDU module. The output features of the first DDU module are then fused with the output features of the second SE_C2F-CIB module in the first MADF fusion module. The output features of the first MADF fusion module are then processed sequentially by the fifth DHFBlock module and the second DDU module. The output features of the second DDU module are then fused with the output features of the first SE_C2F-CIB module in the second MADF fusion module. The output features of the second MADF fusion module are then processed by the sixth DHFBlock module and input into the third MADF fusion module, where they are fused with the output features of the fifth DHFBlock module. The output features of the third MADF fusion module are then processed sequentially by the eleventh MGHCM module, the seventh DHFBlock module, and the twelfth MGHCM module. The output features of the twelfth MGHCM module are then fused with the output features of the first SPPF_UniRepLKA module in the fourth MADF fusion module. The output features of the fourth MADF fusion module are processed by the eighth DHFBlock module and then output to the detection head network. The output features of the fifth, sixth, and seventh DHFBlock modules are fused within the second CMAFEM fusion module and then output to the detection head network.

[0056] The main modules of the non-ferrous metal target detection model for scrapped vehicles are explained in detail below with reference to the attached diagram.

[0057] like Figure 4 The diagram shows the network structure of the Multi-Granularity Hybrid Convolutional Module (MGHCM module). The MGHCM module is used to extract the contours of large targets, the semantics of medium targets, and the details of small targets in parallel, achieving multi-scale feature coverage. The MGHCM module dynamically routes channels based on feature variance, guiding low-variance channels (contour-dominant) to large convolutional kernel branches and high-variance channels (detail-dominant) to lightweight convolutional branches, achieving efficient separation and fusion of "contour-detail" features.

[0058] The MGHCM module comprises a main branch (dynamically dilated convolution), auxiliary branch 1 (lightweight cross-channel convolution), auxiliary branch 2 (pointwise depthwise separable convolution), and a feature fusion and post-processing module (global pooling layer, fully connected weight generation layer, scale alignment layer, residual fusion layer, normalization layer, and activation function layer). The input features of the MGHCM module are first processed by the main branch to obtain the main branch output features. Subsequently, the main branch output features are input to auxiliary branches 1 and 2 respectively; that is, the inputs of auxiliary branches 1 and 2 both come from the output of the main branch. The three branches achieve parallel feature extraction through a "main branch first → auxiliary branches reuse main branch output" approach. Finally, the output features of the three branches are fed into the feature fusion and post-processing module to extract multi-scale features (large target contours, medium target semantics, and small target details). Combined with adaptive weight fusion and residual connections, this achieves "multi-granularity feature complementarity + dynamic task adaptation."

[0059] The main branch (large target contour extraction) process is as follows: Define the input feature map as X∈R C×H×W First, dilation rate prediction is performed, i.e., the dilation rate d is predicted by compressing the channels through a 1×1 convolution: d = Conv1×1(X), and the output shape is 1×1×1 (mapped to {1,2} through a sigmoid function to adapt to different sizes); then the dilation rate d is dynamically adjusted, i.e., d = 2 (equivalent to a 5×5 receptive field) when the input is high resolution (few small targets), and d = 1 (3×3 receptive field) when the input is low resolution (many large targets); then it is processed by dynamically dilated convolution, which is a 3×3 convolution, with dilation = dynamic value, padding = dynamic value (dilation = d, padding = d) (ensuring that the output size is consistent with the input C×H×W), and the formula is F1 = Conv3×3(dilation = d, padding = d). ng = d)(X), F1 is the output feature map of the dynamically dilated convolution, with dimensions maintained at C×H×W (consistent with the input size); Conv3×3 is the convolution operation type, representing a 3×3 kernel used to extract local correlation information of the input features; Dilation is the dilation rate (hole rate), controlling the interval of the sampling points of the convolution kernel; d is the dynamically calculated dilation rate value (e.g., d=1 is standard convolution, d=2 is 1 pixel interval, effectively expanding the receptive field); Padding is the number of pixels for edge padding, d is equal to the dilation rate, ensuring that the height H and width W of the output feature map after convolution are consistent with the input (counteracting the compression of size by the dilated convolution); X is the input feature map of the dynamically dilated convolution, with dimensions of C×H×W, where C is the number of channels, H is the height, and W is the width. The computational cost of the dynamically dilated convolution is only (3) times that of the standard 5×5 convolution. 2 ) / (5 2 = 9 / 25 ≈ 36%, but the receptive field is equivalent.

[0060] The processing procedure for auxiliary branch 1 (target semantic extraction) is as follows: Define the input feature map of auxiliary branch 1 as X∈R C×H×W First, channel compression is performed, that is, using 1×1 convolution to compress the number of channels to C / 2, in order to reduce the amount of subsequent calculations. The formula is: X compress =Conv1×1(out_channels=C / 2)(X),X compress The output feature map is the result of 1×1 convolution channel compression, with dimensions (C / 2)×H×W; Conv1×1 is the 1×1 convolution operation (used for channel compression or restoration); out_channels=C / 2 is the number of output channels in the 1×1 convolution compression stage (set to half the number of input channels C to reduce computation). Then, grouped cross-channel interaction is performed, i.e., using 3×3 grouped convolution (number of groups G=4), and cross-channel information fusion is performed within each group, with the formula: X group =Conv3×3(groups=4,padding=1)(X compress ), X group The output feature map after cross-channel interaction of 3×3 grouped convolutions has dimensions of (C / 2)×H×W, which reduces the number of parameters compared to the standard 3×3 convolution. Finally, channel restoration is performed, i.e., using 1×1 convolutions to restore the number of channels to C, with the formula: F2=Conv1×1(out_channels=C)(X group F2 is the output feature of auxiliary branch 1 (completing target semantic extraction), and its output shape is C×H×W.

[0061] The processing procedure for auxiliary branch 2 (small target detail extraction) is as follows: Define the input feature map as X∈R C×H×W First, spatial feature extraction is performed to obtain intermediate features. Figure X depth That is, using a 3×3 depthwise convolution (groups=C, each channel is computed independently) to extract single-channel spatial details: X depth =Conv3×3(groups=C,padding=1)(X), its output shape is C×H×W, and its computational cost is only 1 / C of the standard 3×3 convolution (when C≥8, the computational cost ≤1 / 8). Then, channel information aggregation is performed, that is, using 1×1 pointwise convolution to aggregate the information between channels. The calculation formula is as follows: F3=Conv1×1(out_channels=C)(X) depth F3 is the output feature of auxiliary branch 2, and its output shape is C×H×W (through 1×1 convolution to aggregate channel information and complete the extraction of small target details).

[0062] The feature fusion and post-processing module (adaptive weights + residual enhancement) processes as follows: the output features F1, F2, F3 ∈ R of the three branches are...C×H×W Within the input feature fusion and post-processing module, global average pooling (GAP) is first performed, compressing each branch feature into a 1×1×C vector: F1gap = GAP(F1) → 1×1×C, and F2gap and F3gap are processed similarly, where F1gap, F2gap, and F3gap are the features of F1, F2, and F3 after global average pooling, respectively; then, after processing by a fully connected layer and Softmax, the three vectors (1×1×3C) are concatenated to generate weights (w1, w2, w3), where the sum of w1, w2, and w3 is 1: [ [w1, w2, w3] = Softmax(FC(Concat(F1gap, F2gap, F3gap))), where w1, w2, and w3 are the fusion weights of the three branches; Softmax is the activation function that normalizes the output of the fully connected layer to the fusion weights [w1, w2, w3] (summing to 1 to adapt to different target scales); FC is the fully connected layer that processes the concatenated vector (learning the weight allocation logic); Concat is the feature concatenation operation that concatenates the three GAP results along the channel dimension, integrating the global features of multiple branches. Then, dynamic adaptation is performed: increasing the proportion of w1 (e.g., 60%) for large target scenarios and increasing the proportion of w3 (e.g., 50%) for small target scenarios. Then, weighted fusion is performed: F... fused = w1·F1 + w2·F2 + w3·F3, F fused To create a weighted fused feature map, multi-scale features are aggregated, with an output dimension of C×H×W. Then, residual connections are performed to fuse the features (weighted fused feature map F). fused The gradient is added to the original input residual to enhance gradient flow, as shown in the formula: F residual =F fused +X;F residual For the residual connection result, basic features are preserved and gradient flow is enhanced to avoid training degradation. X represents the original input features of the feature fusion and post-processing module (from the same source as the branch input) used for the residual connection. Then, LayerNorm normalization (channel dimension) is performed, with the formula: F norm =LayerNorm(F residual ), F norm The feature map is processed by LayerNorm, which normalizes the channel dimension to stabilize the numerical distribution and accelerate training convergence. Finally, the nonlinearity is enhanced using the SiLU activation function: F output =SiLU(F norm ), F output The output feature map of the MGHCM module (activated by SiLU) has a shape of C×H×W and is used by the detection head. SiLU is the Sigmoid linear unit activation function, which introduces nonlinearity and enhances the feature representation capability.

[0063] This embodiment introduces the MGHCM module, which reduces redundant computation by 30%, improves gradient flow efficiency by 22%, and increases edge inference speed by 15%, while maintaining the feature representation capability.

[0064] like Figure 5 The diagram shows the network structure of the Dynamic Layered Fusion Module (DHFBlock Module). In the backbone network, the DHFBlock Module allocates channels according to feature variance, optimizing feature flow and reducing redundant computation. In the neck network, it continues dynamic layered fusion, optimizing the neck feature flow. This embodiment replaces the C3K2 Module with the DHFBlock Module for feature extraction and fusion in shallow networks, solving the problem of insufficient feature utilization in traditional CSP modules (such as the C3K2 Module in YOLOv5). Through dynamic feature routing and multi-scale layered interaction, it achieves adaptive processing of targets with varying complexity.

[0065] The DHFBlock module includes a dynamic routing layer, a large receptive field branch, a fine-grained feature branch, a residual boost branch, and an adaptive fusion layer.

[0066] The features first pass through a dynamic routing layer, and the input feature map is defined as X∈R. C×H×W The processing formula is as follows: Where, var c μ represents the variance of the c-th channel (c∈[1,C]), reflecting the detail complexity of that channel (larger variance → richer details, smaller variance → contour-dominated); c The mean of the c-th channel is used to calculate the channel variance var. c X c,i,j Input features Figure X The pixel value of the c-th channel, i-th row, and j-th column. First, calculate the variance of the feature map for each channel to obtain the variance vector var. c ∈R CThis reflects the channel complexity (high variance indicates rich details; low variance indicates contour-dominant channels). Then, branch weights are generated by inputting the variance vector into a lightweight MLP (1×1 convolution + fully connected layer) and outputting the weights (w1, w2, w3) of three branches, satisfying w1 + w2 + w3 = 1. Specifically, the large receptive field branch weight w1 is positively correlated with "low variance channels" (contour-dominant channels are allocated priority), the fine-grained branch weight w2 is positively correlated with "high variance channels" (detail-dominant channels are allocated priority), and the residual enhancement branch weight w3 is a fixed fallback (ensuring basic feature preservation). The input channels are then divided into three groups according to their weights, as shown in the following formula: C3 = C - C1 - C2, thus obtaining three sub-feature maps: (Input from the large receptive field branch) (Fine-grained branch input) (Residual enhancement branch input).

[0067] The three sub-feature maps are then input into the large receptive field branch, the fine-grained feature branch, and the residual enhancement branch, respectively. The large receptive field branch expands the receptive field through dilated convolutions to capture the contour information of large targets. The fine-grained feature branch extracts detailed features of small targets through grouped deep convolutions and channel attention. The residual enhancement branch preserves the original features through residual connections to enhance gradient flow and avoid feature degradation in deep networks.

[0068] The input to the large receptive field branch is X1∈R C×H×W First, a 5×5 convolution with a dilation rate d=1 (or dynamically adjusted based on the input resolution) is used to expand the receptive field to capture large target contours, in order to output... F1 (1) The output feature map is processed by a 5×5 convolution (C1 channels, size H×W, padding=2 to maintain resolution). Then, a 1×1 pointwise convolution is used to adjust the number of channels, maintaining C1, to produce the output. F1 (2) The output after adjusting the channels of a 1×1 pointwise convolution (channels are C1, size is H×W, padding = 2 to maintain resolution) is then processed by SiLU activation, introducing nonlinearity to obtain the output of the large receptive field branch: F1 = SiLU(F1 (2) ),

[0069] The input to the fine-grained feature branch is First, the data undergoes grouped depthwise convolution, with the number of groups being G = [C² / 4] (balancing computational cost and expressive power). Each group is then independently processed with a 3×3 depthwise convolution to enhance spatial interaction within the channels. The output is... F2 (1)The output of the grouped depthwise convolution (channels C2, size H×W, padding=1 to preserve resolution) is then processed through channel attention (SE module) to obtain... Finally, after SiLU activation, the output of the fine-grained feature branch is obtained.

[0070] The input to the residual enhancement branch is First, a 1×1 convolution is used to reduce the number of channels from C3 to C'3 = [C3 / 2], reducing the computational load and obtaining the output. Then, spatial features are extracted using a 3×3 convolution, with padding = 1 to preserve resolution, resulting in the output. Then, residual connections are performed to obtain the output feature F3 of the residual enhancement branch.

[0071] Then the output features of the three branches An adaptive fusion layer is input to fuse large contour features, fine-grained features, and residual enhancement features. The processing within the adaptive fusion layer is as follows: First, preprocessing is performed, either by unifying the number of channels using a 1×1 convolution, or by fixing the input channel number to C1, C2, and C3 (C1, C2, and C3 are the channel numbers for the large receptive field branch, the fine-grained feature branch, and the residual enhancement branch, respectively). After preprocessing, the following is obtained: Then, features are concatenated along the channel dimension to obtain the concatenated feature F. concat The formula is as follows: F concat =Concat([F1',F2',F3'])∈R 3C′×H×W Then perform global pooling, as shown in the formula: pool = GlobalAvgPool(F concat )∈R 3C′ Then, channel-wise weights β∈R are generated through 1×1 convolution and Sigmoid. 3C′ Alternatively, branch-level weights (β1, β2, β3) are used, where the sum of β1, β2, and β3 is 1. Then, a weighted fusion is performed. If it's a branch-level weight, then F = β1F'1 + β2F'2 + β3F'3. If it's a channel-by-channel weight, then the weights are applied channel-by-channel according to the concatenation order and then merged. F∈R C'×H×WThe output feature map of the adaptive fusion layer (if C′=C, then the number of input channels is maintained) fuses multi-granularity features (large contour, fine detail, residual enhancement). C represents the total number of input feature channels, and C1, C2, and C3 are the number of channels in the large receptive field branch, fine-granular feature branch, and residual enhancement branch, respectively (satisfying C1+C2+C3=C); C′ represents the unified number of channels output by the adaptive fusion layer. The DHFBlock module dynamically routes channels based on feature variance, guiding low-variance channels (contour-dominant) to large convolutional kernel branches and high-variance channels (detail-dominant) to lightweight convolutional branches, achieving efficient splitting and fusion of "contour-detail" features. By introducing the DHFBlock module, redundant computation is reduced by 30%, gradient flow efficiency is improved by 22%, and edge inference speed is improved by 15%, while maintaining the feature expressive power.

[0072] like Figure 6 The diagram shows the network structure of the DCBF (Deformable Cross-Branch Fusion) module. Traditional Concat only physically stitches together features, which can lead to spatial misalignment (deviation in the positions of features from different branches), fixed weights (inability to adapt to target complexity), and feature redundancy (dilution of useful information). This embodiment addresses the rigidity of multi-branch feature fusion within the same image fusion network (such as the multi-granularity features of MGHCM and the dynamic routing features of DHFBlock) by using the DCBF fusion module, adapting to feature interactions across multiple branches (≥2). The processing procedure of the DCBF fusion module is as follows:

[0073] First, a deformable alignment layer is applied to correct spatial misalignment. This involves learning the spatial offsets of multi-branch features and aligning the spatial scales of different branches (e.g., the positional deviation between fine-grained features and features with large receptive fields). The input to the deformable alignment layer is defined as multi-branch heterogeneous features. in N is the number of branches, C i H i W i Here, represents the number of channels, height, and width of the i-th branch, respectively. The processing within the deformable alignment layer is as follows:

[0074] First, standardize the target scale, H. target =max{H1,H2,···,H N}, W target =max{W1,W2,...,W N}, H target W target These are the height and width after the target scale is unified (taking the maximum height and width of each branch feature map); then for each branch F iPredicting spatial migration using 3×3 convolution (x, y directions), Δp i =C 3×3 (F i Then, bilinear interpolation is performed to the target scale, Δp' i =Interpolate(Δp) i ,(H target W target )), Δp' i This indicates that after bilinear interpolation, the scale is aligned to the target; then, a confidence mask (to suppress background) is predicted simultaneously, M. i =σ(C 3×3 (F i )), where M i This represents the confidence mask (single channel, values ​​∈ [0,1]) for the i-th branch, with a spatial size equal to the original H of the branch. i ×W i ,and σ is the Sigmoid activation function (mapping the mask values ​​to [0,1]); bilinear interpolation to the target scale, M' i =Interpolate(M i ,(H target W target )), M' i This indicates that after bilinear interpolation, the result is aligned to the target scale (H). target W target The confidence mask (used for weighted valid regions during sampling), and Then generate a standard sampling grid. (range [-1, 1]); then construct the deformable mesh G. def,i =G std +Δp' i Finally, sampling and weighting are performed, F align,i =GridSample(F i G def,i )·M' i Aligned features All (Spatial scale is uniform, but the passageways remain heterogeneous), F align,i The feature is the one after alignment of the i-th branch.

[0075] Will Input a Dynamic Weight Generator for complexity-aware resource allocation. The Dynamic Weight Generator quantifies complexity based on the global variance of features (high variance → more details → small target; low variance → more contours → large target) and dynamically allocates branch weights. The processing steps of the Dynamic Weight Generator are as follows:

[0076] First, channel projection is performed (unifying the number of channels), and then for each alignment feature F... align,i Project onto a uniform number of channels C using a 1×1 convolution. out The projection feature F is obtained. proj,i =C 1×1 (F align,i C out ), Then the projection feature F proj,i Calculate the global variance of the spatial dimension (the larger the variance, the richer the detail), V i =Var(F proj,i dim=(2,3)), where, V i Let be the spatial global variance of the i-th projected feature; dim = (2,3) indicates that the variance is calculated for the height and width dimensions, reflecting the richness of detail.

[0077] Then cross-channel pooling (aggregating variance information) is performed, V global =Mean(V i (dim=1), V global ∈R B×1×1×1 For cross-channel pooling results, Mean is the mean calculation function (operating on the channel dimension). Then it is processed through an MLP (the MLP structure is 1→C). out / 4→N) Generate the original weights w raw =MLP(V global MLP stands for Multilayer Perceptron, w raw The original branch weights (unnormalized) are the output weights of the MLP. Then, Softmax normalization is performed (ensuring the weight sum is 1), w = Softmax(w raw ), where w is the branch weight after Softmax normalization, and w∈R B×N Softmax is the normalization function (ensuring the weights sum to 1), w = [w1, w2, ..., w N The weight of smaller target branches is automatically increased.

[0078] Then, it undergoes processing through a cross-branch attention layer, which uses key channels within branches (channel attention) and target regions (spatial attention) to avoid feature redundancy. The output of the cross-branch attention layer is the enhanced feature. It achieves layered reinforcement of "channel → space".

[0079] Finally, the residual fusion layer is input, fusing the residuals of the enhanced features and the original aligned features, preserving unprocessed information and optimizing gradient propagation. The output of the residual fusion layer is the final fused feature F. fused ,in

[0080] The DCBF fusion module is deployed only in the second backbone network. It corrects the positional deviation of cross-branch features by predicting spatial offsets and generates a confidence mask M to suppress background noise. The DCBF fusion module solves the problem of infrared images being susceptible to deformation and noise interference, making the cross-branch alignment error of infrared features <0.3 pixels and improving the accuracy of thermal radiation feature extraction by 13.6%, providing a precise foundation for subsequent secondary fusion. In this embodiment, the compact inversion block (CIB) structure from YOLOv10 is introduced into the traditional c2f module to replace the Bottleneck in the original YOLOv8. This structure uses inexpensive depthwise convolution for spatial mixing and cost-effective pointwise convolution for channel mixing. It achieves both lightweight design and effective enhancement of feature extraction capabilities. At the same time, SEAttention is added to the CIB module, and an attention twin network structure is used to extract and compare features from two input samples, further enhancing the target detection capability.

[0081] like Figure 7 The diagram shows the network structure of the SE_C2F-CIB module (lightweight channel enhancement module). The SE_C2F-CIB module reduces computational cost (by 30% compared to traditional modules) and enhances the discriminative power of key channels. First, key features are extracted from the input data through convolution (Conv) operations. Then, a split operation is used to segment the data so that different parts can be processed independently. Next, two SE_CIB modules with squeeze-and-excitation (SE) mechanisms participate in the computation as bottleneck structures, effectively improving the model's performance and efficiency. Following this, a concatenation operation recombines the multiple split data parts. Finally, another convolution (Conv) operation is used to perform deeper feature extraction and processing on the concatenated result. The output of the convolutional layer does not fully consider the dependencies between channels; its purpose is to enable the network to selectively enhance information-rich features, allowing subsequent processing to fully utilize these features and suppress useless features. The SEAttention mechanism mainly consists of two key steps: Squeeze and Excitation.

[0082] The Squeeze operation (compression operation) compresses the feature map in terms of spatial dimensions using global average pooling, resulting in a one-dimensional vector with the same number of channels. The Squeeze operation process is as follows: Figure 8As shown, the Squeeze operation is a global average pooling operation. After the compression operation, the feature map is compressed into a 1×1×C vector.

[0083] The excitation operation involves applying two fully connected layers to a one-dimensional vector (the 1×1×C vector obtained from the squeeze operation), with a ReLU activation function in between, to obtain the weights for each channel. These weights are then multiplied by the original feature map to adjust the importance of different channels. For example... Figure 9 As shown, Excitation consists of two fully connected layers, where SERatio is a scaling parameter. The purpose of SERatio is to reduce the number of channels, thereby reducing computational cost. The first fully connected layer has C × SERatio neurons, with an input of 1 × 1 × C and an output of 1 × 1 × C × SERadio. The second fully connected layer has C neurons, with an input of 1 × 1 × C × SERadio and an output of 1 × 1 × C.

[0084] SEAttention is inserted in the intermediate stage after feature extraction by the C2F_CIB module to adjust the channel importance of the extracted features. For insertion in the intermediate stage: assume the C2F_CIB module outputs features at a certain position. Figure X The shape is ( channels, height, width); first, perform global average pooling to obtain a shape of ( The compressed feature vectors of channels 1, 1) are then passed through two fully connected layers and a ReLU activation function to obtain the channel weight vectors. Finally, the weight vectors are combined with the original feature vectors. Figure X The features are multiplied to obtain the adjusted feature map, which is then processed by the subsequent C2F_CIB module. Alternatively, SEAttention can be added to the output of the C2F_CIB module to adjust the final output features. For adding SEAttention to the output: after the C2F_CIB module outputs the feature map, the SEAttention operation is directly applied to it, which involves global average pooling, fully connected layer processing, and multiplication with the original feature map.

[0085] This embodiment uses a CIB module (depth convolution + point convolution) to replace the traditional Bottleneck, reducing computation by 30%. It also embeds SE Attention to enhance metal-related channels (thermal radiation, texture). While maintaining mAP@0.5, the number of parameters is reduced by 35%, adapting to the computing power limitations of edge devices and achieving a balance between lightweight design and high precision.

[0086] like Figure 10 The diagram shows the network structure of the SPPF_UniRepLKA module (large receptive field pooling module), which expands the receptive field to capture large metal targets. This embodiment combines SPPF with the UniRepLK large-kernel convolution, increasing the receptive field and effectively improving target detection accuracy. Through multi-scale pooling (3×3 / 5×5 / 9×9) and a 13×13 large-kernel convolution, the receptive field is expanded to 203×203, enhancing the feature aggregation capability for "large metal objects and complex backgrounds," solving the "missing target context information" problem caused by traditional small receptive fields, and improving the recall rate for detecting complete automotive parts.

[0087] First, the input data undergoes convolutional operations through a Conv layer, resulting in a feature map with multiple branches. The first branch is directly connected to the Concat layer, the second branch undergoes the first MaxPool2d pooling, the third branch undergoes the second MaxPool2d pooling (pooling again on top of the first pooling), and the fourth branch undergoes the third MaxPool2d pooling (pooling again on top of the second pooling). Then, the output feature maps from the Conv layer, the first pooling, the second pooling, and the third pooling are concatenated through the Concat layer to form a richer feature representation (the concatenated feature map). Finally, the concatenated feature map is input into the UniRepLKBlock module for further processing, utilizing the large kernel convolution of the UniRepLKBlock module to extract features. Specifically, SPPF (a fast version of Spatial Pyramid Pooling) typically concatenates pooling results at different scales; here, the UniRepLKBlock module is used to enhance the features after pooling. The Conv layer first extracts features from the input, then performs multiple pooling operations to obtain features at different scales. After concatenation, the features are processed by the UniRepLKBlock module to capture multi-scale feature information, while using large kernel convolution to improve feature representation capabilities.

[0088] The SPPF_UniRepLKA module optimizes the network's ability to extract detailed features from various spatial hierarchies, solves the information bottleneck problem in deep neural networks, and improves model performance. It helps the model process and learn data more effectively to improve the accuracy of object detection.

[0089] like Figure 11The diagram shows the network structure of the CMAFEM fusion module (secondary cross-modal fusion module). This module aligns "fused image features (pixel level)" with "infrared features (feature level)" and dynamically allocates modal weights through dual attention (improving visual weights by 60% for small target scenes). The CMAFEM fusion module includes a Deformable Spatial Alignment layer, a Channel Projection & Concat layer, a Cross-Modal Dual Attention layer, and a Spatial Boost & Residual Fusion layer.

[0090] Within the CMAFEM fusion module, features are first processed by a deformable spatial alignment layer. This layer learns the spatial offset between infrared and visible light image features, corrects intermodal positional deviations (such as target displacement and distortion), and enhances the effective region using a sampling mask. The processing steps are as follows:

[0091] Define the infrared feature as F IR ∈R B×CIR×HIR×WIR The visible light characteristic is F VIS ∈R B×CVIS×HVIS×WVIS First, the maximum spatial scale of the dual-modal features (infrared and visible light features) is calculated to unify the scale, H. target =max(H IR H VIS ), W target =max(W IR W VIS ), H target W target These represent the maximum height and width of the bimodal feature, respectively. Then, offset prediction is performed, i.e., for F... IR and F VIS Predict spatial migration Δp using 3×3 convolution respectively IR , The two channels correspond to the x and y directions, and are then aligned to H via bilinear interpolation. target ×W target , thus obtaining Δp' IR and Δp' VIS Then perform mask prediction, that is, for F IR and F VIS Confidence masks M are generated by 3×3 convolution + Sigmoid respectively. IR , Then aligned to H by bilinear interpolation target ×W target , thus obtaining M' IR and M'VIS Then, a standard sampling grid is generated. (range [-1, 1]), superimposed offsets yield deformable mesh G def,IR =G std +Δp IR ′、G def,VIS =G std +Δp VIS Finally, the alignment features of the infrared and visible light images are output through grid_sample sampling and weighted masking: F align,IR =Gridsample(F IR G def,IR )·M′ IR F align,VIS =Gridsample(F VIS G def,VIS )·M′ VIS F align,IR and F align,VIS Alignment feature maps for the infrared mode (infrared image) and the visible light mode (visible light image), respectively; F IR F VIS These are the original infrared features and visible light features, respectively. def,IR G def,VIS Deformed meshes for infrared and visible light features, respectively, M′ IR M′ VIS These are confidence masks for the infrared and visible light modes, respectively. The feature input channels, after processing by the deformable spatial alignment layer, are projected and stitched together. This layer unifies the number of channels for the dual-modal features (infrared thermal imaging features and visible light image features), stitching them into a single tensor to provide the basis for attention calculation. The processing procedure is as follows:

[0092] First, channel projection is performed, that is, for F... align,IR and F align,VIS The number of channels is unified to C using 1×1 convolutions respectively. out , to obtain F proj,IR , F proj,IR =C 1×1 (F align,IR C out ), F proj,VIS =C 1×1 (F align,VIS C out Then, channel splicing is performed, that is, splicing bimodal features along the channel dimension to form a cross-modal tensor F. concat The formula is F concat =torch.cat([F proj,IR ,F proj,VIS ]),in

[0093] Features processed by channel projection and stitching layers are input into a cross-modal dual attention layer. This layer enhances features in two steps: first, channel attention (intra-modal filtering) to highlight key channels such as infrared thermal radiation and visible light texture; second, modal attention (cross-modal weighting) to dynamically allocate weights between the two modalities (infrared thermal imaging and visible light images) (emphasizing visual data for small targets and infrared data for large targets). The specific processing steps are as follows:

[0094] (1) Intra-Modal Attention: First, global pooling is performed on F. concat Compress spatial dimensions and extract channel statistics P = Mean(F) concat dim = (2,3)), where Then through MLP(2C) out →C out / 4→2C out +Sigmoid generates channel weights (output is) Strengthen key channels (weights ∈ [0,1]): α=σ(MLP(P)), F chan =F concat ·α,F chan Features after channel attention weighting.

[0095] (2) Inter-Modal Attention: First, global pooling is performed on F... chan Compressing spatial dimensions and extracting modal statistics Q = Mean(F) chan dim = (2,3)), where Then through MLP(2C) out →C out / 4→2)+Softmax generates modal weights ω=[ω IR ,ω VIS ]∈R B×2 (summing to 1), dynamically assign bimodal weights ω = Softmax(MLP(Q)), F modal =w IR ·F chan,IR +w VIS ·F chan,VIS F modal For the modal attention weighted fusion result, w IR For the weights of infrared thermal imaging, w VIS For the weights of the visible light image, F chan,IR This represents the infrared thermal imaging features after channel attention processing (channel dimensions have been weighted and filtered by attention), F chan,VISThis represents the features of a visible light image after channel attention processing (attention output in the channel dimension). The features processed by the cross-modal dual attention layer are input to the spatial enhancement and residual fusion layer. Within this layer, spatial attention weighting is first used to obtain fused features to highlight the target region and suppress background noise (such as interference in complex scenes). Then, residual fusion is used to add the fused features to the original bimodal features to retain unprocessed information and avoid gradient vanishing. The specific processing procedure is as follows:

[0096] (1) Spatial Attention: First, regarding F... modal Spatial correlation is captured by 3×3 convolution, resulting in S=C 3×3 (F modal S represents the spatial features extracted by 3×3 convolution, and Then, spatial weights are generated using a 1×1 convolution followed by a sigmoid function (output: Strengthen the target region (weight ∈ [0,1]): β=σ(C 1×1 (S)), F spat =F modal ·β,F spat These are the features after spatial attention.

[0097] (2) Residual Fusion: For F align,IR and F align,VIS Bilinear interpolation to H target ×W target F′ align,IR =Interpolate(F align,IR ,(H target W target )), F′ align,VIS =Interpolate(F align,VIS ,(H target W target Then, through a 1×1 convolution to a unified channel, it is passed to C. out F res,IR =C 1×1 (F′ align,IR C out ), F res,VIS =C 1×1 (F′ align,VIS C out Then sum them up to obtain the residual characteristic F. res =F res,IR +F res,VIS Then, residual connections are performed, adding the fused features (features processed by spatial attention) to the residual features, and outputting the final result F. fused Ffused =F spat +F res F fused This is the output feature of the spatial enhancement and residual fusion layer. This embodiment uses the CMAFEM fusion module to stitch and fuse infrared and visible light images. Spatial offset prediction using deformable convolution is introduced into the cross-modal fusion layer to address the feature misalignment problem caused by registration errors in traditional methods. A hierarchical attention mechanism of "channel filtering (within a single modality) → modal weighting (between modalities)" is employed, differing from the traditional "single-attention coarse fusion," to achieve precise feature selection. Spatial attention enhances the target region, while residual connections preserve the original features, forming an "enhancement-preservation" closed loop, improving robustness in complex scenes (such as occlusion and deformation).

[0098] like Figure 12 The diagram shows the network structure of the Dynamic Deformable Upsample (DDU) module. The DDU module is used for upsampling the neck network, achieving deformation-aware fusion of multi-scale features, adapting to target deformation, and improving accuracy for small targets. It addresses the spatial rigidity, information dilution, and poor architectural compatibility issues of traditional upsampling (bilinear, transposed convolution). The DDU module includes a deformable sampling layer for spatial deformation adaptation, a dynamic weight generator (enhancing detailed regions), a cross-stage attention layer (enhancing feature interaction), and a residual fusion layer (preserving original information). Within the DDU module, features are first processed by the deformable sampling layer. The deformable sampling layer learns the spatial offset of the target deformation, dynamically adjusts the sampling position, and corrects the edge blurring problem of traditional interpolation. Its processing is as follows: First, the low-resolution input F... low ∈R B×C×H×W Predicting spatial migration using 3×3 convolution Δp∈R B ×2×H×W Δp includes offsets in the x and y directions to adapt to target distortion / stretching. Then, a confidence mask (to suppress background) M = σ(C) is predicted synchronously. 3×3 (F low )), M∈R B×1×H×W σ is the Sigmoid function, with mask values ​​∈ [0, 1]. Then, a standard mesh is generated. (H s =H·s,W s =W·s is the upsampling size), and then the offset is superimposed to obtain the deformed mesh G. def =G std +Interpolate(Δp,(H s W s Finally, the sampling feature F is obtained.sampled Its calculation formula is F sampled =GridSample(F low G def Interpolate(M,(H) s W s )).

[0099] The features are input into a dynamic weight generator after processing by a deformable sampling layer. The dynamic weight generator quantizes the feature complexity through global variance and dynamically enhances small targets / detail regions. The processing procedure is as follows:

[0100] First, the sampled features F sampled Calculate the spatial variance of the channel dimension V = Var(F) sampled dim=(2,3)), V∈R B×C×1×1 The larger the variance, the more details are involved. Then, global pooling and MLP are used to generate single-channel weights w to enhance high-complexity regions. where w∈R B×1×1×1 , and The layers are linear, with dimensions C→C / 4 and C / 4→1 respectively; then the weighted features F are obtained. fused =F sampled ·w.

[0101] The features processed by the dynamic weight generator are input into the cross-stage attention layer. This layer enhances the interaction between the upsampled features and the features from the previous stage (the output features of the dynamic weight generator), highlighting the target region. The processing steps are as follows: First, channel attention processing is performed on the current feature F. weighted′ and the previous stage feature F prev′ (Output features of the deformable sampling layer), concatenated after global pooling, and then processed by MLP to generate channel weights A. c The formula is as follows: This represents the operations of the multilayer perceptron (including linear transformations and activations to achieve channel dimension transformation); then the weighted feature F is obtained. prev′ =F prev ·A c ,F weighted′ =F weighted ·A c Then, spatial attention processing is performed, which involves concatenating channels and then generating spatial weights through a 3×3 convolution. Then the cross-stage fusion feature F is obtained. attention =(F prev′ +F weighted′ )·A s .

[0102] The features are input to the residual fusion layer after being processed by the cross-stage attention layer. The residual fusion layer is used to fuse the upsampled features and the original residual features to optimize gradient propagation. The processing procedure is as follows:

[0103] First, channel projection is performed, if the original residual F residual′ Channel mismatch, projected via 1×1 convolution F' residual′ This represents the original residual features after 1×1 convolution channel projection (aligned with the number of upsampled feature channels). Then, residual concatenation is performed to obtain the final fused feature F of the residual fusion layer. fused =F attention +F' residual′ .

[0104] The DDU module in this embodiment corrects edge stretching deformation during the upsampling process through deformable sampling, and enhances dense areas of small targets based on feature variance (the larger the variance, the higher the weight); it overcomes the "edge blurring" defect of traditional upsampling, improving the detection AP of metal fragments (<16px) to 67.8% and the detail retention to 34%. The DDU module not only enhances the edge details of small targets, but also improves the resolution of low-resolution features.

[0105] like Figure 13 The diagram shows the network structure of the MADF (Modal-Aligned Dynamic Fusion) module. The MADF module addresses the issues of feature positional discrepancies at different scales / branches (e.g., misalignment between the upsampled small target and the original large target contour) and the inability of fixed-weight fusion (e.g., equal distribution) to adapt to target size (small targets emphasize fine-grained details, large targets emphasize contours). The MADF module includes a Deformable Scale Alignment layer, a Dynamic Weight Generator, a Multi-Branch Attention layer, and a Residual Fusion layer. Within the MADF module, features are first processed by the Deformable Scale Alignment layer. This layer learns the spatial offsets of features at different scales, aligns the multi-scale target positions (e.g., coordinate deviation between small-scale edges and large-scale contours), and enhances the target region through sampling masks. The input to the Deformable Scale Alignment layer is defined as multi-scale features. in N is the scale number, H i and W i As the scale decreases, such as H1 > H2 > ... > H N The processing procedure for the deformable scale alignment layer is as follows:

[0106] First, target scale unification is performed, using the largest scale as the benchmark, i.e., H. target =H1,W target =W1. Then perform offset prediction for each scale feature F i Predicting spatial migration using 3×3 convolution (2 channels correspond to the x and y directions), then bilinear interpolation is performed to the target scale Δp. i ′=Interpolate(Δp i ,(H target W target Then, a confidence mask is generated using 3×3 convolution and a sigmoid function. M is obtained after interpolation i ′=Interpolate(M i ,(H target W target M i ′ represents the interpolated confidence mask, and M i ′∈[0,1]; then deformable sampling is performed, first generating a standard mesh. After overlaying the offset sampling, the output feature F of the deformable scale-aligned layer is obtained. align,i F align,i =Gridsample(F i G std +pi′)·M′ i .

[0107] The features are input into a dynamic weight generator after being processed by a deformable scale alignment layer. The dynamic weight generator determines the scale complexity based on the global variance of the features (large variance → more details → small scale, small variance → more contours → large scale) and dynamically assigns scale weights. The processing procedure of the dynamic weight generator is as follows:

[0108] First, channel projection is performed, and the aligned feature F align,i The number of channels in a 1×1 convolution is C. out Features after obtaining a unified number of channels Then, the global variance is calculated, and the variance of the spatial dimension is calculated for each scale feature. (The larger the variance, the richer the details). After global pooling, scale weights are generated using an MLP (Softmax normalization, summing to 1), with the formula v. global =Mean(V i ,dim=1)∈R B×1×1×1 ω i =Softmax(MLP(v global )), where ω i ∈R B×N ,∑ωi =1, v global V represents the spatial variance of each branch. i Aggregated features (with dimension v) after global pooling (dim=1) global ∈R B×1×1×1 ).

[0109] Features processed by the dynamic weight generator are input into the multi-branch attention layer. The multi-branch attention layer enhances multi-scale features in two steps: first, channel attention is used to filter key channels within a single scale (such as small-scale edge channels and large-scale semantic channels); second, spatial attention is used to highlight target regions and suppress background interference. The processing procedure of the multi-branch attention layer is as follows: first, channel attention is performed, compressing the spatial dimension through global pooling and extracting channel statistics. Through MLP(C out →C out / 4→C out) +sigmoid generates channel weights Then perform channel weighting to obtain the channel-weighted feature F. chan,i =F proj,i ·α i F proj,i This represents the feature of the i-th branch after channel projection (1×1 convolution with uniform channel number); then spatial attention processing is performed, using 3×3 convolution to capture spatial correlation and extract spatial features. Spatial masks, i.e., spatial weights, are generated through 1×1 convolution plus Sigmoid. Then spatial weighting is performed to obtain the spatially weighted feature F. spat,i =F chan,i ·β i F chan,i This indicates the characteristics after channel weighting.

[0110] The features processed by the multi-branch attention layer are input into the residual fusion layer. The residual fusion layer is used to concatenate the original multi-scale features, retain unprocessed information, and optimize gradient propagation. The processing procedure of the residual fusion layer is as follows: First, the original features are processed, that is, the original multi-scale features are upsampled and projected onto the channels.

[0111] F i F represents the unprocessed feature of the i-th branch in the original multi-scale input. res,i F represents i The residual features are obtained after upsampling and 1×1 convolutional projection (channels are aligned with attention features); then, the attention features and the original residuals are fused by weighting to obtain the weighted fused features. Fspat,i ω represents attention-enhancing features. i The scale weights are represented; finally, the cross-branch fusion features are obtained as the final output.

[0112] This example uses the Dynamic Head module to improve the detection head network. The neck network already outputs multi-scale, cross-modal dynamic fusion features, which naturally aligns with the "dynamic adjustment" characteristic of the Dynamic Head module. The Dynamic Head module is a dynamic head structure that can adaptively adjust the prediction head based on the feature information of the input image, effectively improving the model's ability to detect targets at different scales. The core idea is to enable the detection head to dynamically adapt to the input features, thereby improving the model's performance across different scales, spaces, and tasks. The Dynamic Head module effectively improves the YOLOv8 object detection model, achieving significant improvements in metrics such as mAP and recall.

[0113] The Dynamic Head module mainly consists of the following three parts:

[0114] Scale-Aware Attention Module: This module adjusts its attention based on the scale of the features, enabling features of different scales to be fused and processed at an appropriate scale. This module optimizes the impact of scale differences by assigning weights to features of different scales.

[0115] Spatial-Aware Attention Module: This module optimizes for spatial location by weighting important regions in the feature map, enabling the network to focus more on the foreground region of the target object and avoiding the impact of redundant parts of the feature map on model performance.

[0116] Task-Aware Module: This module adjusts the output of the head according to the specific task (such as classification, localization, etc.), and can optimize the classification or regression results of the target according to the task requirements, thereby improving the accuracy and robustness of the model.

[0117] The detection head network outputs a rotated bounding box (adapted to the angle of the metal fragment) and classification results. In this embodiment, the detection head network achieves collaborative optimization of "multi-scale bias repair → small target detail enhancement → dual-modal feature recalibration", supporting a high-precision cloud-based detection system (mAP@0.5≥91.2%).

[0118] This embodiment uses a YOLOv11 model-based network framework improved to a "dual-modal driven dual-backbone-module collaborative detection architecture." It revolves around the entire process of "pixel-level coarse fusion → feature-level fine fusion → multi-scale enhanced detection," building a dual-system collaborative capability of "fast edge detection" and "high-precision cloud analysis," overcoming the efficiency-accuracy contradiction of traditional single-modal / single-system approaches in complex scenarios. The dual-modal fusion network is a cross-modal pixel-level coarse fusion architecture with a "dual encoder-decoder + dual discriminator" structure. The dual encoders extract thermal radiation features from the infrared image and texture features from the visible light image, respectively; the decoder fuses the dual-modal features to generate a single-channel fused image; the infrared discriminator constrains thermal feature preservation, and the visible light discriminator constrains texture detail preservation. The dual-modal fusion network outputs a fused image that "combines infrared thermal saliency and visible light texture," providing a foundation for subsequent "single-channel fast detection," while retaining the original infrared image for "feature-level secondary fusion."

[0119] This embodiment employs a three-layer design—"image fusion foundation, dual-backbone differentiated extraction, and multi-module collaboration at the neckline"—to support the functional differentiation and collaboration of the two systems, providing a systematic solution for non-ferrous metal detection in end-of-life vehicles that combines efficiency and accuracy. The implicit support for the collaboration of the two systems is as follows:

[0120] (1) Fast edge detection (System 1): The output of backbone network 1 can be directly connected to simplify the neck, quickly complete inference, and meet the real-time requirements of the pipeline, such as... Figure 14 As shown.

[0121] (2) High-precision cloud-based analysis (System 2): The features of the dual backbone network are fused twice by the CMAFEM fusion module and then enter the complete neck. Through multi-module collaboration, a breakthrough in accuracy is achieved in complex scenarios, such as... Figure 15 As shown.

[0122] (3) Dynamic switching: Through module access control (such as enabling / disabling the CMAFEM fusion module), the hardware can automatically adapt to the end-cloud scenario, forming an industrial-grade detection closed loop with dynamic balance between "efficiency and accuracy".

[0123] This embodiment designs an adaptive collaborative framework of a "pixel-level fusion single-channel fast detection system" and a "feature-level secondary fusion high-precision system." Through deep coupling between the front-end GAN dual-modal fusion network (dual-modal fusion network) and the back-end deformable feature fusion network (two backbone networks and a neck network), an industrial-grade detection closed loop of "efficient edge-side inference - precise cloud-based analysis" is constructed. System 1 uses single-channel fusion image compression to reduce computational complexity, achieving real-time detection of ≥35 FPS on edge devices such as Jetson AGX Orin. System 2 overcomes the rigid contradiction between efficiency and accuracy inherent in traditional single-model systems through a two-stage enhancement process of "pixel-level coarse fusion → feature-level fine fusion." A front-end mode switching switch and a dynamic adaptation mechanism for the backbone network enable unstructured switching between the two systems.

[0124] Compared with the traditional single-modal solution, the dual-system solution in this embodiment has the following advantages:

[0125] For the first time, accurate identification of millimeter-sized metal fragments in non-ferrous metal fragments from scrapped vehicles has been achieved. In a scenario of metal surface oxidation on a 322℃ heating platform and in low-light conditions at night, the recognition rate was improved by 27% and 31% respectively through infrared-visual dual-modal fusion, overcoming the strong dependence of traditional vision solutions on environmental conditions. Through adaptive switching of hardware computing power perception and knowledge distillation mechanism, the inference energy consumption of the edge device in battery-powered mode was reduced by 45%. Moreover, after the cross-modal attention parameters of System 2 were transferred to System 1, the edge model could improve the AP of small targets by 5.2% without retraining, and the model iteration cycle was shortened from 8 weeks to 3 weeks, improving the technology update efficiency by 62.5%. At the same time, a full-chain solution from image acquisition and feature fusion to intelligent detection was built, driving the industrial transformation of the scrapped vehicle recycling industry from "manual sorting" to "intelligent detection".

[0126] The training process for the non-ferrous metal target detection model for scrapped vehicles is as follows (dual-system joint optimization):

[0127] (1) Data Acquisition and Preprocessing

[0128] First, construct a multimodal dataset with a collection scale of 6500+ (visible light + infrared image pairs). Construct a multimodal dataset directory MetalDataset, with three first-level subdirectories under the root directory: "train" (training set), "val" (validation set), and "test" (test set). The three subdirectories have the same structure, except that the data size is divided in an 8:1:1 ratio (the training set contains 5200 sets of data, and the validation and test sets each contain 650 sets). Each first-level subdirectory (e.g., "train") contains two second-level subdirectories: "images", which stores all image data; internally, it is further divided into three third-level subdirectories: "visible" stores the original visible light images (in .jpg format, 640×640 resolution), with filenames named "metal_number_vis.jpg" (e.g., "metal_001_vis.jpg"); "infrared" stores the original infrared images corresponding to the visible light images (in .jpg format, 640×640 resolution), with filenames strictly corresponding to the visible light images (e.g., "metal_001_infra.jpg"); and "fusion" stores the fused images generated by the GAN dual-modal fusion network (in .jpg format, 640×640 resolution), with filenames corresponding to the original dual-modal images (e.g., "metal_001_fusion.jpg"). The "labels" directory stores all annotation files in YOLO-specific .txt format. The filenames correspond one-to-one with the image files under "images" (e.g., "metal_001_vis.jpg" corresponds to "metal_001_vis.txt", and "metal_001_fusion.jpg" reuses the annotation content of "metal_001_vis.txt"). Each file records information such as the category, bounding box coordinates, and orientation angle of the non-ferrous metal target. Annotation is performed after the acquisition of the "visible" and "infrared" images under "images" and before generating the "fusion" image: the LabelImg tool is used to annotate the non-ferrous metal targets in the original visible and infrared images, and the annotation results are directly saved to the corresponding "labels" directory. Since the subsequently generated "fusion" image is from the same source as the original image, it does not need to be re-annotated; the annotation file with the same filename is reused. Then, quality verification is performed by randomly sampling 10% of the annotation results to verify the accuracy of the bounding boxes (error ≤ 2 pixels) and the consistency with the category annotations. Finally, preprocessing is performed. The general processing involves scaling to 640×640 using LetterBox and normalizing pixel values ​​to [0,1] using Sobel. For System 2, the infrared image histogram is equalized to improve thermal radiation contrast. Then, data augmentation is performed. For System 1, the fused image is rotated, scaled, and blurred. For System 2, dual-modal synchronous rotation / cropping is performed to maintain modal alignment.

[0129] (2) Phased training

[0130] Joint training (100 epochs): The input consists of 50% fused image (fused image after being fused by GAN bimodal fusion network) and 50% bimodal image pair (unfused infrared and visible light image pair);

[0131] The loss function is L total =λ1L cls +λ2L reg +λ3L qual +λ4L GAN L total L represents the total loss. cls For classification loss, L reg For regression loss, L qual L represents mass loss. GAN λ represents the GAN fusion loss, and λ1, λ2, λ3, and λ4 represent the weight coefficients of each loss term (the contributions of the balanced classification, regression, quality, and GAN loss).

[0132] Backbone parameter sharing: The first three layers of the backbone of System 1 and System 2 share parameters, while the subsequent layers are trained differently. System 1 fine-tuning (50 epochs): The input is a fused image, and the learning rate is 0.001→0.0001; L is reduced. reg Weights are introduced using sparse regularization to compress the number of model parameters; the optimization objective is to increase inference speed and small-objective recall. System 2 fine-tuning (80 epochs): input is a bimodal image pair, learning rate 0.0005 → 0.0001; L... GAN Weights are used to enhance the texture preservation capability of the fused image; cross-modal consistency constraints are strengthened (infrared-visual detection box alignment loss); the optimization objective is mAP@0.5+ cross-modal robustness.

[0133] (3) Loss function and optimal configuration

[0134] Classification loss L cls Focal Loss (α = 0.8, γ = 2) addresses class imbalance. Regression loss L... reg :DloULoss+ Angle Loss (L reg =1-DloU+0.5|θ pred -θ gt |), θ pred and θ gt These represent the predicted angle (e.g., target orientation angle) and the ground truth angle, respectively; System 2 calculates the GAN fusion loss L separately. GANAdversarial loss and perceptual loss are used to ensure that the fused image retains visible light texture. Optimizer configuration: The optimizer is AdamW (weight decay 0.0005, β1 = 0.9, β2 = 0.999); the learning rate strategy is cosine annealing (initial 0.01 → minimum 0.0001, period 350 epochs); regularization is L1+L2 (λ = 0.001) to suppress overfitting.

[0135] (4) Quantification and Deployment Strategy

[0136] System 1 Quantization: Weights INT8+ activate FP16, model <30MB, edge inference speed improved by 15%, accuracy loss <1%. System 2 Deployment: Cloud maintains FP16, TensorRT acceleration, inference speed ≥15FPS, mAP@0.5 = 89.7%. Dual System Collaborative Deployment: The edge runs System 1 by default. When the confidence of the detection result is <0.7, the cloud System 2 secondary detection is triggered, forming a closed loop of "rapid initial screening + accurate re-examination".

[0137] This embodiment constructs a CMAFEM fusion module, combining sub-pixel-level offset prediction of deformable convolution with hierarchical attention based on "channel filtering-modal weighting": the deformable spatial alignment layer dynamically predicts the spatial offset of infrared-visible features through 3×3 convolution (error < 0.2 pixels), and combines it with a confidence mask activated by Sigmoid to suppress background interference, solving the pain point of traditional registration algorithms being unable to handle target deformation; the cross-modal dual attention mechanism uses MLP to filter thermal radiation channels and texture channels, and modal attention uses Softmax to dynamically allocate weights (65% visual weight for small target scenes and 72% infrared weight for large target scenes), forming a progressive fusion paradigm from "single-modal feature purification" to "cross-modal weight optimization", which improves the accuracy by 16.7% compared to traditional single attention fusion.

[0138] The MGHCM module employs a three-branch parallel architecture to adaptively extract multi-scale features: the main branch intelligently switches the equivalent receptive field between 3×3 and 5×5 through dynamically dilated convolutions (d=1 / 2 adaptively adjusted), with computational cost only 36% of the standard 5×5 convolution; auxiliary branch 1 uses grouped cross-channel convolutions (G=4), reducing the number of parameters by 50% and enhancing the semantic features of medium-sized targets; auxiliary branch 2 compresses computational cost to 1 / 8 of the standard convolution through depthwise separable convolutions, focusing on the edge details of small targets. The adaptive weight fusion layer generates dynamic weights (60% for large target scenes and 55% for small target scenes) through global pooling and fully connected layers, achieving intelligent weighting of "contour-semantic-detail" features, improving multi-scale target recall rate by 23.5%.

[0139] The DHFBlock module uses a dynamic routing mechanism based on feature variance to assign low-variance channels (contour-dominant) to 5×5 dilated convolution branches and high-variance channels (detail-dominant) to grouped deep convolution branches. Combined with residual enhancement branches, it preserves the original features, solves the feature redundancy problem of traditional CSP modules, improves gradient flow efficiency by 22%, and reduces the risk of feature degradation in deep networks by 40%.

[0140] The Dynamic Head module dynamically weights the feature pyramid layers P2-P5 using a scale-aware attention module, increasing the weight of the small target layer P2 by 40%. Combined with the output of rotated bounding box parameters (x, y, w, h, θ), it adapts to the arbitrary angle placement of metal fragments. The task-aware module enhances the differentiation of similar metals such as copper / aluminum through cross-modal cross-entropy (improving the discrimination rate to 94%). The regression branch, combined with infrared thermal profiles, controls the angle prediction error to ±3.2, achieving a 19.3% improvement in positioning accuracy compared to traditional detection heads.

[0141] The SE_C2F-CIB module replaces Bottleneck with the CIB module, reducing computation by 35% and improving channel discriminability by 28% after adding the SEAttention mechanism; the SPPF_UniRepLKA module uses a 13×13 large kernel convolution combined with a non-dilated design to expand the receptive field to 203×203, with only a 12% increase in computational cost and a 14.6% improvement in large target detection accuracy.

[0142] This embodiment constructs a complete technological moat from the underlying algorithm to the upper-level application through three-dimensional innovations: a dual-system collaborative architecture, deformable cross-modal fusion, and a dynamic multi-granularity feature network. Its core value lies not only in the breakthrough improvement in hard indicators such as mAP@0.5, but also in providing a systematic solution for the end-of-life vehicle recycling industry that deeply integrates "algorithm-hardware-scenario." The related technologies have broad transferability in fields such as metal surface defect detection and new energy battery material sorting, forming a patent layout network covering multiple industries and significantly enhancing technological barriers and market competitiveness.

[0143] Example 2

[0144] This embodiment 2 describes a non-ferrous metal target detection system for scrapped vehicles, including a visible light camera, an infrared thermal imager, and a computer device. The visible light camera is used to capture visible light images and upload them to the computer device, and the infrared thermal imager is used to capture infrared images and upload them to the computer device. The computer device includes a memory and one or more processors, and the memory stores executable code. When the processor executes the executable code, it implements the steps of the non-ferrous metal target detection method for scrapped vehicles described in embodiment 1.

[0145] The embodiments of the present invention are only used to illustrate the technical solutions of the present invention and are not intended to limit it. For those skilled in the art, it will be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting non-ferrous metal targets in scrapped automobiles, characterized in that, Includes the following steps: Step 1: Simultaneously acquire visible light and infrared images of non-ferrous metal fragments from scrapped vehicles, and preprocess the acquired visible light and infrared images to obtain a training dataset. Step 2: Build a non-ferrous metal target detection model for scrapped cars based on the improved YOLOv11 model, which includes a dual-modal fusion network, backbone network 1, backbone network 2, neck network and detection head network; The dual-modal fusion network is based on generative adversarial networks and is used to receive and process visible light and infrared images to output a fused image that combines the saliency of infrared images with the texture of visible light images. Backbone Network 1 and Backbone Network 2 are used to receive fused image and infrared image inputs and perform feature extraction, respectively. Backbone Network 1 replaces the standard convolution in the original backbone network with the MGHCM module to extract multi-granularity features, and replaces the CSPBlock module in the original backbone network with the DHFBlock module to achieve dynamic feature routing. At the same time, it replaces the SPPF module in the original backbone network with the SPPF_UniRepLKA module and adds the SE_C2F-CIB module to achieve lightweight channel enhancement. Backbone Network 2, based on Backbone Network 1, further adds a DCBF fusion module to correct cross-branch positional deviations of infrared features. The neck network is used to receive and fuse the output features of the first and second backbone networks. The neck network replaces the traditional fusion convolution in the original neck network with the MADF fusion module to solve multi-scale misalignment, replaces the ordinary upsampling in the original neck network with the DDU module to enhance small target details, and adds the CMAFEM fusion module to achieve cross-modal secondary fusion. Step 3: Train the model based on the training dataset, and finally perform target detection on non-ferrous metals of scrapped cars; the backbone network includes the MGHCM module, DHFBlock module, SE_C2F-CIB module and SPPF_UniRepLKA module; There are five MGHCM modules, namely the first, second, third, fourth and fifth MGHCM modules; there are two DHFBlock modules, namely the first and second DHFBlock modules; there are two SE_C2F-CIB modules, namely the first and second SE_C2F-CIB modules; there is one SPPF_UniRepLKA module, namely the first SPPF_UniRepLKA module; The fused image input to the backbone network is processed sequentially through the first MGHCM module, the second MGHCM module, the first DHFBlock module, the third MGHCM module, the second DHFBlock module, the fourth MGHCM module, the first SE_C2F-CIB module, the fifth MGHCM module, the second SE_C2F-CIB module, and the first SPPF_UniRepLKA module to extract multi-scale lightweight features of the fused image; The second backbone network includes the MGHCM module, DHFBlock module, DCBF fusion module, SE_C2F-CIB module, and SPPF_UniRepLKA module; There are four MGHCM modules, namely the sixth, seventh, eighth, and ninth MGHCM modules; two DHFBlock modules, namely the third and fourth DHFBlock modules; three DCBF fusion modules, namely the first, second, and third DCBF fusion modules; four SE_C2F-CIB modules, namely the third, fourth, fifth, and sixth SE_C2F-CIB modules; and one SPPF_UniRepLKA module, namely the second SPPF_UniRepLKA module. The infrared image input to backbone network 2 is first processed sequentially through the sixth MGHCM module, the seventh MGHCM module, the third DHFBlock module, the eighth MGHCM module, and the fourth DHFBlock module. Then, the output features of the fourth DHFBlock module are fused with the output features of the second DHFBlock module within the first DCBF fusion module. The output features of the first DCBF fusion module are then processed sequentially through the third SE_C2F-CIB module, the ninth MGHCM module, and the fourth SE_C2F-CIB module. Finally, the fourth SE_C2F-CIB module... The output features of the IB module and the output features of the first SE_C2F-CIB module are fused in the second DCBF fusion module; the output features of the second DCBF fusion module are then processed by the fifth SE_C2F-CIB module, and the output features of the fifth SE_C2F-CIB module and the output features of the second SE_C2F-CIB module are fused in the third DCBF fusion module; the output features of the third DCBF fusion module are then processed sequentially by the sixth SE_C2F-CIB module and the second SPPF_UniRepLKA module to extract the features of the infrared image; The MGHCM module includes a dynamically dilated convolution for extracting the contours of large targets, a lightweight cross-channel convolution for extracting the semantics of medium targets, a pointwise depthwise separable convolution for extracting the details of small targets, and a feature fusion and post-processing module. The features input to the MGHCM module are first processed by dynamic dilated convolution. The process is as follows: first, the channels are compressed by 1×1 convolution and the dilation rate is predicted. Then, the dilation rate is dynamically adjusted and processed by dynamic dilated convolution. Then, the output features of the dynamically dilated convolution are input into the lightweight cross-channel convolution and the pointwise depthwise separable convolution respectively. In the lightweight cross-channel convolution, a 1×1 convolution is first used to compress the number of channels, and then a 3×3 group convolution is used to fuse cross-channel information within each group. Finally, a 1×1 convolution is used to restore the number of channels to be consistent with the input features. Within the pointwise depth-separable convolution, a 3×3 depth convolution is first used to extract single-channel spatial details, and then a 1×1 pointwise convolution is used to aggregate inter-channel information. Then, the three output features of dynamically dilated convolution, lightweight cross-channel convolution, and pointwise depthwise separable convolution are input into the feature fusion and post-processing module and processed as follows: First, global average pooling is performed to compress the output features of each convolution into three 1×1×C vectors, where C is the number of feature channels. Then, weights are generated through a fully connected layer and Softmax processing. After dynamic adaptation, weighted fusion is performed to obtain fused features. Then, residual connections are used to add the fused features to the original input residuals to enhance gradient flow. Finally, LayerNorm normalization is performed, and the SiLU activation function is used to enhance nonlinearity. The DHFBlock module includes a dynamic routing layer, a large receptive field branch, a fine-grained feature branch, a residual enhancement branch, and an adaptive fusion layer. The features input to the DHFBlock module are first input into the dynamic routing layer. In the dynamic routing layer, the feature variance is first calculated and three branch weights are generated. The sum of the three branch weights is 1. Then, channel splitting is performed to obtain three sub-feature maps. Then, the three sub-feature maps are respectively input into the large receptive field branch, the fine-grained feature branch, and the residual enhancement branch; The large receptive field branch expands the receptive field through dilated convolution to capture the contour information of large targets; Fine-grained feature branches are used to extract detailed features of small targets through grouped deep convolutions and channel attention; The residual enhancement branch preserves the original features through residual connections, which is used to enhance gradient flow and avoid feature degradation in deep networks. The output features of the three branches are then fed into an adaptive fusion layer to fuse large contour features, fine-grained features, and residual enhancement features; The DCBF fusion module includes a deformable alignment layer, a dynamic weight generator, a cross-branch attention layer, and a residual fusion layer. The features input to the DCBF fusion module are first fed into a deformable alignment layer, where the spatial offset of the multi-branch features is learned and the spatial scale of different branches is aligned. Then the features are fed into a dynamic weight generator, which quantizes the complexity through the global variance of the features and dynamically allocates the branch weights. Then, the input is fed into a cross-branch attention layer, where channel attention and spatial attention are used to process the features to avoid feature redundancy. Finally, the input is fed into a residual fusion layer to fuse the residuals of the enhanced features and the original aligned features, thus preserving the unprocessed information. The CMAFEM fusion module includes a deformable spatial alignment layer, a channel projection and stitching layer, a cross-modal dual attention layer, and a spatial enhancement and residual fusion layer. The features input to the CMAFEM fusion module are first fed into a deformable spatial alignment layer. Within this layer, scaling, offset prediction, mask prediction, and deformable sampling operations are performed to learn the spatial offset between infrared and visible light image features, correct intermodal positional deviations, and enhance the effective region through sampling masks. Then, the features are fed into a channel projection and stitching layer. This layer performs channel projection, channel stitching, mask prediction, and deformable sampling to unify the number of bimodal feature channels and stitch them into a single tensor. Finally, the features are fed into a cross-modal dual attention layer. Within this layer, channel attention is used to highlight infrared thermal radiation and visible light textures, and then modal attention is used to dynamically allocate bimodal weights. Then, the input is the spatial augmentation and residual fusion layer. Within the spatial augmentation and residual fusion layer, the fusion features are first obtained by spatial attention weighting, and then the fusion features are added to the original bimodal features by residual fusion to retain unprocessed information. The DDU module includes a deformable sampling layer, a dynamic weight generator, a cross-stage attention layer, and a residual fusion layer. The features input to the DDU module are first fed into a deformable sampling layer, where offset prediction, mask prediction, and deformable interpolation are performed sequentially to learn the spatial offset of the target deformation and dynamically adjust the sampling position. Then, they are fed into a dynamic weight generator, where global variance is calculated before weight prediction and enhancement. The global variance quantifies the feature complexity, dynamically enhancing small targets or detailed regions. Finally, they are fed into a cross-stage attention layer, where channel attention, spatial attention, and cross-stage fusion operations are performed sequentially to enhance the interaction between the upsampled features and the output features of the dynamic weight generator, highlighting the target region. Finally, the input is fed into the residual fusion layer, which fuses the upsampled features with the original residual features through channel projection and residual connection. The MADF fusion module includes a deformable scale alignment layer, a dynamic weight generator, a multi-branch attention layer, and a residual fusion layer; The features input to the MADF fusion module are first fed into a deformable scale alignment layer. Through target scale unification, offset prediction, mask prediction, and deformable sampling, the spatial offsets of features at different scales are learned to align the positions of multi-scale targets and enhance the target region through sampling masks. Then, the features are fed into a dynamic weight generator. In the dynamic weight generator, the features are first channel-projected, then the global variance is calculated and weights are generated to determine the scale complexity based on the feature global variance and dynamically allocate scale weights. Then, the input is processed by a multi-branch attention layer. Within the multi-branch attention layer, key channels within a single scale are first filtered through channel attention, and then the target area is highlighted through spatial attention to suppress background interference. Finally, the input is fed into the residual fusion layer, where the original features are processed and residual fusion is performed to stitch together the original multi-scale features and retain the unprocessed information.

2. A non-ferrous metal target detection system for scrapped vehicles, comprising a visible light camera, an infrared thermal imager, and computer equipment; wherein, Visible light cameras are used to capture visible light images and upload them to computer devices; infrared thermal imagers are used to capture infrared images and upload them to computer devices; computer devices include memory and one or more processors; The executable code is stored in the memory; characterized in that, when the processor executes the executable code, it is used to implement the steps of the method for detecting non-ferrous metal targets in a scrapped vehicle as described in claim 1.

Images