A method and system for sorting non-magnetic metals from scrapped vehicles
Through the improved YOLOV8 network model architecture, the C2fCIB, ASPP and MDAM modules are used to solve the problem of low classification accuracy of non-magnetic metals in scrapped cars, and efficient identification and classification of non-magnetic metals is achieved, and classification accuracy and efficiency are improved.
Patent Information
- Application Number
- CN202411280630.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to effectively separate non-magnetic metals in scrapped vehicles from mixed materials, resulting in low classification accuracy and efficiency.
Using the improved YOLOV8 network model architecture, the non-magnetic metal classification model is built by introducing the C2fCIB module, the ASPP module and the MDAM module, and the non-magnetic metal classification model is used to classify scrapped automobiles with infrared cameras and computer equipment, improving feature extraction capabilities and classification accuracy.
It significantly improves the classification accuracy and detection efficiency of non-magnetic metals, can effectively capture the details and global information of scrap metals, enhance the recognition ability of different types of metals, reduce the calculation complexity, and shorten the identification time.
Smart Images

Figure CN118799657B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image processing, and in particular relates to a method and system for classifying non-magnetic metals of scrapped vehicles. Background Art
[0002] The classification of non-magnetic metals in scrapped cars generally refers to a series of activities that classify, store, disassemble and recycle non-magnetic metals in scrapped cars according to certain regulations or standards, thereby transforming them into public resources. Metal recycling from scrapped cars has always been an important part of waste management and resource reuse. In the process of dismantling scrapped cars, the classification and recycling of metal materials are key steps. The metals in scrapped cars are divided into magnetic and non-magnetic at the source, and then classified, disassembled, transported and processed according to different types of non-magnetic metals. The purpose of non-magnetic metal classification is to increase the resource value and economic value of metals, reduce metal mixing and pollution, optimize the metal recycling process, reduce processing costs, and reduce the consumption of natural resources, which has social, economic, ecological and other benefits. At present, traditional physical sorting technologies, such as magnetic separation, eddy current sorting and density separation, are difficult to effectively separate non-magnetic metals from mixed materials in scrapped cars. Summary of the invention
[0003] In view of the technical problems such as poor sorting effect of current non-metallic methods for scrapped vehicles, the present invention proposes a method for sorting non-magnetic metals of scrapped vehicles to improve the accuracy and recognition efficiency of non-metallic classification of scrapped vehicles.
[0004] In order to achieve the above object, the present invention adopts the following technical scheme:
[0005] A method for sorting non-magnetic metals of scrapped vehicles comprises the following steps:
[0006] Step 1. Obtain non-magnetic metal thermal images of scrapped cars and perform preprocessing and annotation operations to obtain a training data set;
[0007] Step 2. Build a non-magnetic metal classification model based on the improved YOLOV8 network model architecture, which includes a backbone network, a neck network, and a detection head; the backbone network includes a convolution block, a C2fCIB module, and an ASPP module;
[0008] There are five convolution blocks, which are defined as the first, second, third, fourth, and fifth convolution blocks respectively; there are four C2fCIB modules, which are defined as the first, second, third, and fourth C2fCIB modules respectively; the processing flow of the backbone network is as follows:
[0009] The feature map first passes through the first and second convolution blocks, then the first C2fCIB module, then the third convolution block, then the second C2fCIB module, then the fourth convolution block, then the third C2fCIB module, then the fifth convolution block, then the fourth C2fCIB module, and finally the ASPP module before output;
[0010] The neck network includes upsampling module, combination layer, C2fCIB module, MDAM module and convolution block;
[0011] There are two upsampling modules, which are defined as the first and second upsampling modules respectively; there are four combination layers, which are defined as the first, second, third and fourth combination layers respectively; there are four C2fCIB modules, which are defined as the fifth, sixth, seventh and eighth C2fCIB modules respectively; there are four MDAM modules, which are defined as the first, second, third and fourth MDAM modules respectively; there are two convolution blocks, which are defined as the sixth and seventh convolution blocks respectively; the processing flow of the neck network is as follows:
[0012] The output features of the ASPP module are sequentially passed through the first upsampling module and the first combination layer, and combined with the output features of the third C2fCIB module in the first combination layer;
[0013] The output features of the first combination layer are sequentially passed through the fifth C2fCIB module, the first MDAM module, the second upsampling module and the second combination layer, and combined with the output features of the second C2fCIB module in the second combination layer;
[0014] The output features of the second combination layer are sequentially passed through the sixth C2fCIB module, the second MDAM module, the sixth convolutional block and the third combination layer, and are combined with the output features of the first combination layer in the third combination layer;
[0015] The output features of the third combination layer pass through the seventh C2fCIB module, the third MDAM module, the seventh convolutional block and the fourth combination layer in sequence, and are combined with the output features of the ASPP module in the fourth combination layer;
[0016] The output features of the fourth combination layer pass through the eighth C2fCIB module and the fourth MDAM module in sequence;
[0017] There are three detection heads, and the second, third and fourth MDAM modules are respectively connected to one of the detection heads;
[0018] Step 3. Train the improved YOLOV8 network model based on the training data set in step 1 to obtain a trained non-magnetic metal classification model, and use the non-magnetic metal classification model to classify the non-magnetic metals of scrapped vehicles.
[0019] In addition, based on the above-mentioned scrapped car non-magnetic metal classification method, the present invention also proposes a scrapped car non-magnetic metal classification system, which adopts the following technical solutions:
[0020] A scrapped car non-magnetic metal classification system includes an infrared camera and a computer device, wherein the computer device includes a memory and at least one processor. The memory stores executable code.
[0021] When the processor executes the executable code, it is used to implement the method for classifying non-magnetic metals of scrapped vehicles as described above.
[0022] The present invention has the following advantages:
[0023] As described above, the present invention relates to a method and system for classifying non-magnetic metals of scrapped vehicles. Based on the original YOLOV8 network structure, the method of the present invention introduces the C2fCIB module, replaces the SPPF module with the ASPP module, and introduces the MDAM module before the detection head, and builds a non-magnetic metal classification model based on the improved YOLOV8 network model architecture, wherein the ASPP module and the MDAM module can significantly improve the model's ability to extract scrap metal features in spatial scale, frequency scale, channel dimension, spatial dimension and depth dimension, so that the model can capture the details and global information in the scrap metal image, and extract the edge, texture, shape and structure information in the scrap metal image through the filters of different frequencies corresponding to each convolution block, and has a strong recognition ability for information of different color channels and spatial structures, thereby improving the feature extraction ability of the model. The proposed C2FCIB module performs separable deep extraction of local features and global features of scrap metals, performs weighted emphasis and data screening, realizes the extraction of significant features of scrap metals, reuses weighted features, improves the model's ability to classify different types of scrap metals, and when classifying smaller scrap metals, the classification weights of other metals can be used for processing, to achieve accurate classification of smaller metals of different types, and improves the classification and detection accuracy of the model. The present invention uses the MDAM module, C2FCIB and ASPP modules to cooperate, and adds skip connections, void convolutions and significant scrap metal data weighting (i.e., in the channel attention module of the MDAM module, the weights after the previous metal weighting are used to weight subsequent recognition) strategies in the model's scrap metal data processing process. While expanding the field of view and feature extraction depth, the feature data of non-waste metal features and non-recognition channel data categories are eliminated, significantly improving the model's computational efficiency and shortening the recognition time of scrap metals in motion. The method for classifying non-magnetic metals of scrapped vehicles proposed in the present invention has a stronger feature extraction capability and realizes the fusion of global and local information, which is beneficial to improving the classification and detection accuracy of non-magnetic metals of scrapped vehicles, reducing the calculation complexity and improving the detection rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 It is a structural diagram of a non-magnetic metal classification model based on an improved YOLOV8 network model architecture in an embodiment of the present invention;
[0025] Figure 2 Flow chart of the method for classifying non-magnetic metals of scrapped vehicles in an embodiment of the present invention;
[0026] Figure 3 This is a network structure diagram of the C2fCIB module in an embodiment of the present invention;
[0027] Figure 4A network structure diagram of a CIB module in an embodiment of the present invention;
[0028] Figure 5 A network structure diagram of the MDAM module in an embodiment of the present invention;
[0029] Figure 6 A network structure diagram of a self-attention mechanism module in an embodiment of the present invention;
[0030] Figure 7 A network structure diagram of a cross-attention mechanism module in an embodiment of the present invention;
[0031] Figure 8 This is a network structure diagram of a channel attention mechanism module in an embodiment of the present invention;
[0032] Fig. 9 A network structure diagram of a detection head in an embodiment of the present invention;
[0033] Fig.10 4 is a network structure diagram of the ASPP module in an embodiment of the present invention. DETAILED DESCRIPTION
[0034] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0035] Example 1
[0036] like Figure 2 As shown, this embodiment describes a method for sorting non-magnetic metals from scrapped vehicles, comprising the following steps:
[0037] Step 1. Obtain thermal images of non-magnetic metals of scrapped vehicles, preprocess and annotate the images, and obtain a training data set for training the non-magnetic metal classification model in the following step 2.
[0038] Before using the non-magnetic metal classification model to classify the non-magnetic metal thermal images of scrapped vehicles, the non-magnetic metal thermal images of scrapped vehicles used for model training are first obtained and the images are preprocessed.
[0039] The material used in this embodiment comes from a scrapped car factory, and contains crushed copper, aluminum, stainless steel and other non-magnetic metal mixtures, which have the characteristics of complex composition, uneven particle size, and metal surface oxidation and pollution.
[0040] The training and test image data sets required in this embodiment use non-magnetic metal thermal images of scrapped cars collected by a FOTRIC 626CH online infrared camera, wherein the image preprocessing process is as follows:
[0041] Step 1.1. Image reading:
[0042] Read an image file into memory.
[0043] Step 1.2. Image scaling:
[0044] Scale the image to the 640×640 size supported by the model.
[0045] Step 1.3. Image normalization:
[0046] In this embodiment, the Sobel operator is used to normalize the pixel values of the image to the interval [0, 1].
[0047] The Sobel operator consists of two 3×3 gradient templates, one for calculating the gradient in the X direction and the other for calculating the gradient in the Y direction. In this process, the non-magnetic metal thermal imaging image of the scrapped car is transformed into a feature image that is easier to identify.
[0048] Among them, the X-direction gradient template is:
[0049] ;
[0050] The Y-direction gradient template is:
[0051] .
[0052] The specific calculation method of the Sobel operator is as follows:
[0053] Convert the image grayscale value to floating point number. Perform convolution operation on the image, use the X-direction gradient template to calculate the gradient in the X direction, and use the Y-direction gradient template to calculate the gradient in the Y direction.
[0054] For example, the edge features of scrap metal can be intuitively represented by the brightness differences in different areas of the image, and the oversaturation or dimness caused by obtaining too high or too low pixel values of scrap metal images during display can be avoided, thereby enhancing the adaptability of the model.
[0055] Sum the square of the gradient in the X direction and the gradient in the Y direction to get the gradient value of the image.
[0056] Color channel conversion converts the image from RGB format to CHW format, that is, channel-height-width format.
[0057] In the edge response map of the scrap metal image obtained by convolving the image with the Sobel operator, the response at each position represents that there is an edge similar to the Sobel operator at this position in the original image. The information is compressed. A numerical value in the response map actually represents that there is a Sobel edge pattern of corresponding intensity at this position, and the feature is extracted through convolution.
[0058] The specific steps of convolution feature extraction are as follows:
[0059] The convolution layer multiplies the image with the filter and then adds the result to the bias to extract various features in the scrap metal image, including edges, textures, shapes, and structures. The result of the convolution is a new image with the same size as the original image.
[0060] The calculation formula is: .
[0061] The model inputs a 640×640 image, which is extracted through multiple layers of special augmentation. After image comparison, the probability distribution is obtained and the image is divided into a, b, c, d, e, f, g, h, and i. Among them, a is steel, b is aluminum, c is copper, d is lead, e is magnesium alloy, f is zinc alloy, g is nickel alloy, h is titanium alloy, and i is platinum alloy. Finally, the model outputs an image including the predicted rectangular frame coordinates of the target object, the category label, and the confidence score.
[0062] Step 2. Build a non-magnetic metal classification model based on the improved YOLOV8 network model architecture. The improved YOLOV8 is improved on the structure of the original YOLOV8 network, which includes a backbone network, a neck network, and a detection head.
[0063] like Figure 1 The improved YOLOV8 network model architecture is shown. There are three specific improvements. By replacing the standard bottleneck structure (Bottleneck) in C2f with the CIB module, the entire module has a significant improvement in feature extraction. The ASPP module replaces the SPPF module and the MDAM module is used.
[0064] The C2fCIB module can extract rich scrap metal features at spatial scale, frequency scale, and depth scale through multi-branch structure and convolution operation, thereby improving the model's feature extraction capability. This module can fuse the rich scrap metal features extracted at spatial scale, frequency scale, and depth scale in the combination layer concat, capture more scrap metal frequency features, edge features, and spatial relationships, and improve detection accuracy. The C2fCIB module replaces the standard bottleneck structure Bottleneck in C2f with the CIB module, which can significantly improve the performance of the model while maintaining efficient calculation, making the model suitable for lightweight target detection applications, and can be widely used in various detection equipment with stronger universality.
[0065] The ASPP (Atrous Spatial Pyramid Pooling) module is a technology for capturing multi-scale contextual information. The ASPP module extracts spatial and depth information of scrap metal by applying atrous convolution in parallel at different dilation rates (rate=1, 3, 5) to obtain richer edge features and texture features of scrap metal. It also uses global average pooling operations to enable the model to use the larger field of view brought by atrous convolution during detection to extract global features of scrap metal targets and perform multi-scale information fusion, so that the model can effectively handle targets of different sizes.
[0066] The MDAM module combines multi-head attention, including the Self-Attention module and the Cross-Attention module, with the channel attention module to form an attention module that can capture the relationship between each element and all other elements in the input scrap metal feature map, realize information fusion between different types of scrap metals, emphasize the channels that contribute most to the task (representative scrap metal data channels), and suppress non-metallic channels or redundant channels in the feature map. The MDAM module deeply extracts the local features of scrap metals through the self-attention module to obtain representative scrap metal feature data; cross-learns multiple types of scrap metal data through the cross-attention module, and uses the better scrap metal feature weights to weight the identified scrap metals, so that the model can quickly classify multiple types of scrap metals of uneven sizes. Finally, the channel attention module uses the fused data to classify scrap metals on the color channel to achieve enhanced extraction and recognition of scrap metal colors.
[0067] The model stacks multiple convolution blocks and C2fCIB modules to enable the neural network to better extract the image features of scrap metal. After a series of convolution operations, the input scrap metal image is converted into a set of high-dimensional feature maps.
[0068] The high-dimensional feature map undergoes dilated convolution and pooling operations in the ASPP module to obtain high-dimensional feature maps of scrap metal. These feature maps contain information such as the edge, texture, and shape of the scrap metal, which is the basis for subsequent detection and classification.
[0069] In the first upsampling module Upsample and C2fCIB module, the high-dimensional feature map of scrap metal is combined and output with the feature data of the third C2fCIB module through the combination layer concat, and the data is screened and emphasized in the multi-dimensional attention module MDAM. After multiple repetitions, the data is output through the detection head Detect.
[0070] like Figure 1 As shown, the backbone network includes convolution blocks, C2fCIB modules and ASPP modules.
[0071] There are five convolution blocks, which are defined as the first, second, third, fourth, and fifth convolution blocks. The parameters of the first, second, third, fourth, and fifth convolution blocks are the same, with a convolution kernel size of k=3, a step size of s=2, and p representing the padding size.
[0072] There are four C2fCIB modules, which are respectively defined as the first, second, third and fourth C2fCIB modules.
[0073] The shortcut of the first and fourth C2fCIB modules is the quick connection. Shortcut=True means quick connection is enabled, shortcut=False means quick connection is not enabled, n means the number of repetitions, n=3d, d means the network depth or the number of convolutions. The shortcut=True of the second and third C2fCIB modules, n=6d means 6 convolutions.
[0074] act=true means enabling the activation function, and whether to enable the activation function and shortcut key connection needs to be set by yourself.
[0075] The processing flow of the backbone network is as follows:
[0076] The feature map after preprocessing in step 1 first passes through the first and second convolution blocks, then passes through the first C2fCIB module, then passes through the third convolution block, then passes through the second C2fCIB module, then passes through the fourth convolution block, then passes through the third C2fCIB module, then passes through the fifth convolution block, then passes through the fourth C2fCIB module, and then passes through the ASPP module before output.
[0077] The neck network consists of an upsampling module, a combination layer, a C2fCIB module, an MDAM module, and a convolutional block.
[0078] There are two upsampling modules, which are defined as the first and second upsampling modules respectively.
[0079] There are four combination layers concat, which are defined as the first, second, third and fourth combination layers respectively.
[0080] There are four C2fCIB modules, which are defined as the fifth, sixth, seventh and eighth C2fCIB modules respectively; the shortcut=True, n=3d of the fifth, sixth, seventh and eighth C2fCIB modules.
[0081] There are four MDAM modules, which are defined as the first, second, third and fourth MDAM modules respectively.
[0082] There are two convolution blocks, which are defined as the sixth and seventh convolution blocks respectively.
[0083] The parameters of the sixth and seventh convolution blocks are the same, with the convolution kernel size k=3, the step size s=2, and p representing the padding size.
[0084] The processing flow of the neck network is as follows:
[0085] The output features of the ASPP module are sequentially passed through the first upsampling module and the first combination layer, and combined with the output features of the third C2fCIB module in the first combination layer;
[0086] The output features of the first combination layer are sequentially passed through the fifth C2fCIB module, the first MDAM module, the second upsampling module and the second combination layer, and combined with the output features of the second C2fCIB module in the second combination layer;
[0087] The output features of the second combination layer are sequentially passed through the sixth C2fCIB module, the second MDAM module, the sixth convolutional block and the third combination layer, and are combined with the output features of the first combination layer in the third combination layer;
[0088] The output features of the third combination layer pass through the seventh C2fCIB module, the third MDAM module, the seventh convolutional block and the fourth combination layer in sequence, and are combined with the output features of the ASPP module in the fourth combination layer;
[0089] The output features of the fourth combination layer pass through the eighth C2fCIB module and the fourth MDAM module in sequence.
[0090] There are three detection heads, and the second, third and fourth MDAM modules are respectively connected to a detection head.
[0091] like Figure 3 As shown in Figure 1, the C2FCIB module is a new C2F structure. By replacing the standard bottleneck structure Bottleneck in C2f with the CIB module, the C2FCIB module is essentially the same as the C2F structure. The data splitting module Split is introduced in the C2FCIB module.
[0092] The data splitting module Split divides the metal feature map into two parts.
[0093] One part is processed by two CIB modules through depth-wise separable convolution to reduce computational complexity and parameter quantity; the other part is directly skipped and connected, and finally the two parts of the feature map are spliced together as the output feature map of the C2FCIB module.
[0094] The C2FCIB module enables the model to significantly improve its detection accuracy with a slight increase in computational complexity.
[0095] like Figure 3 As shown in the figure, the processing flow of the C2FCIB module is as follows:
[0096] First, it is processed by a convolution block, and then the feature map is split into two parts, feature maps F1 and F2, by the data splitting module.
[0097] The feature map F1 is directly skip-connected, and the feature map F2 is processed by two CIB modules in sequence. The feature map F2.1 is obtained by the first CIB module, and the feature map F2.2 is obtained by the second CIB module.
[0098] The processed feature maps F2.1 and F2.2 are concatenated with the jump-connected feature map F1 in the channel dimension to form a feature map F3; finally, the feature map F3 passes through a convolution block to obtain the final output feature map F4.
[0099] The CIB module uses depthwise convolution and point-by-point convolution to reduce computational complexity, extracting the edges of scrap metal and overall features to the greatest extent while keeping the spatial size of the feature map unchanged, but without performing cross-channel operations.
[0100] Figure 3 In the figure, c is the number of channels, n is the number of repetitions, h is the height, and w is the width.
[0101] like Figure 4 The network structure of the CIB module is shown. When the scrap metal feature map passes through the CIB module, if the shortcut connection short is enabled (i.e. short=true), it is directly output. If the shortcut connection is not enabled (i.e. short=false), it passes through the deep convolution RepVGGDW and point-by-point convolution to extract the edge and texture features of the scrap metal. After repeated multiple times, the processed scrap metal feature map is output. RepVGGDW is a depthwise separable convolutional block, which is part of the RepVGG architecture.
[0102] MDAM attention refers to a mechanism that combines multi-head attention (including cross attention and self-attention) with channel attention to capture the relationship between each element and all other elements in the input scrap metal feature map, realize information fusion between different modes or types, and emphasize the channels that contribute most to the task (scrap metal data channels with representative features), and suppress non-metallic channels or redundant channels in the feature map. It realizes the arrangement of global dependencies, feature map space, channel relationship reasoning and multi-dimensional data fusion, and shows good performance in processing non-local data information, local data information, complex structured data, multi-modal tasks and sequence-to-sequence tasks.
[0103] like Figure 5 The structure diagram of the MDAM module is shown. The processing flow of the MDAM module on the input feature map is as follows:
[0104] By performing three different linear transformations on the original input vector of the feature map, namely, respectively with the training weight matrix W Q , W K , W V Multiply them to get the Q, K, and V values (for the convenience of distinction, they can be distinguished by Qn, Kn, and Vn, where n is a variable and its value is a natural number). After passing through the fully connected layer, they enter the self-attention mechanism module and the cross-attention mechanism module respectively, and then perform weighted fusion to obtain the data f1.n. Each channel will have an f1.n value. Because there are multiple channels, there are multiple f1.n, where n is a variable. Multiple f1.n values are concatenated to obtain the weighted feature map f1, and f1 is processed by the channel attention mechanism module to obtain the weighted value f2. Finally, f1 and f2 are fully fused to obtain the output scrap metal feature map F6.
[0105] The calculation formula is as follows:
[0106] ;
[0107] ;
[0108] .
[0109] Among them, W Q , W K , W V is the trainable weight matrix and X represents the original input vector.
[0110] like Figure 6The network structure diagram of the self-attention mechanism module is shown. The self-attention mechanism is an attention mechanism that associates different positions of a single sequence to calculate the representation of the same sequence (that is, different patches in the same image pay attention to each other). It can establish global dependencies, infer the relationship between each element of the scrap metal feature map and the whole, and expand the receptive field of the image by collecting elements of the scrap metal feature map in multiple spaces.
[0111] Compared with the traditional neural convolutional network (CNN), the Self-Attention module uses Q, K, and V data for calculation, which has a larger receptive field, can obtain more contextual information, and better filter the data information of the significant features of scrap metal. For the filtered data information of the significant features of scrap metal, this mechanism can weight the input according to the contextual information of different positions, so that the model can better focus on the information related to the current position. This context sensitivity enables the Self-Attention module to capture the semantic relationship in each scrap metal feature map more quickly in the task of processing natural scrap metal images.
[0112] In addition, the Self-Attention module can simultaneously calculate all positions in the input waste metal feature map sequence and model the relationship between any two positions in the input sequence, which makes the Self-Attention module perform well in learning transfer. The waste metal feature data Q, K, and V are calculated through matrix multiplication operations (MatMul) to calculate the dot product of the query (Query) and key (Key) matrices to obtain the similarity between Q and K. Since the dot product of Q and K will produce large values, in order to prevent these values from being too large, the waste metal feature map is scaled (Scale) to avoid the problem of gradient explosion caused by excessive waste metal feature map data during the dot product process. The output waste metal feature map is then masked (Mask) to further reduce the feature map data, and a screening operation is performed at the same time. After SoftMax processing, it is fused with the vector value (Value) to obtain the waste metal feature map fs. The calculation formula is as follows:
[0113] .
[0114] where d K is the dimension of the key vector, used to scale the dot product result to ensure numerical stability.
[0115] like Figure 7The structure diagram of the Cross-Attention module is shown. The Cross-Attention module achieves information fusion between different scrap metal sequences by calculating the similarity between the Q value (such as Q3) of one scrap metal sequence and the K and V values (such as K2 and V2) of another scrap metal sequence (Q, K, and V all have weights). This allows the weight data of one scrap metal sequence to be used as the weight of another scrap metal sequence, significantly improving the model's ability to classify multiple types of scrap metals.
[0116] Waste metal feature map s The feature data K, V value of the scrap metal feature map is fused with the Q value of another scrap metal feature map in the Cross-Attention module to obtain the feature data fc. The calculation formula is as follows:
[0117] .
[0118] like Figure 8 The network structure diagram of the Channel Attention module is shown.
[0119] fs and fc are concatenated to obtain the weighted f1. The weighted f1 value is passed through a multi-layer perceptron (MLP) to obtain two sets of waste metal feature data maps. Finally, after weighted fusion and recalibration, the waste metal feature data map F5 is obtained.
[0120] like Fig. 9 As shown in the figure, the detection head Detecte adopts the decoupled-head structure Decoupled-Head, which divides the data into two major branches:
[0121] The first branch is the classification branch Cls Branch, which is responsible for predicting the category probability of each detection box. The probability distribution of each category is generated through a series of convolutional layers and fully connected layers.
[0122] Specifically, when calculating the cross entropy loss LSC Loss, the true label is smoothed so that the label is no longer a one-hot vector, but a vector with a certain degree of smoothness.
[0123] The second branch is the regression branch Box Branch, which is responsible for predicting the bounding box coordinates of each detection box.
[0124] Fig. 9 Medium cls Indicates how many types there are, c is the number of channels, and reg_max indicates the maximum number of channels in the image.
[0125] The coordinates and size of the bounding box are generated through a series of convolutional layers and fully connected layers. The location and category of the scrap metal are accurately identified through the obtained data. Bbox Loss represents the bounding box regression loss.
[0126] The formulas for LSC Loss and Bbox Loss are as follows:
[0127] ;
[0128] ;
[0129] in, Represents the true label probability after smoothing, indicating that the value is not a one-hot vector, but a vector with a certain degree of smoothness. represents the predicted probability, represents the true label probability.
[0130] like Fig.10 The network structure of ASPP (Atrous Spatial Pyramid Pooling) is shown.
[0131] ASPP (Atrous Spatial Pyramid Pooling) is a simplified SPPF module. Its working principle is mainly to expand the receptive field of the convolution kernel without increasing the amount of calculation by applying atrous convolution in parallel at different dilation rates (this model uses rate=1, 3, and 5 respectively). It can capture multi-scale contextual information and effectively process targets of different sizes. It is suitable for tasks that require high-precision feature extraction.
[0132] like Fig.10 As shown, the processing flow of the ASPP module is as follows:
[0133] The ASPP module divides the input feature map F4 (H, W, C) into five equal parts, namely feature maps fa, fb, fc, fd and fe; among them, H, W, and C represent the length, width, and number of channels of the input feature map, respectively.
[0134] Perform a 1×1 convolution on fa, and the shape of the output feature map is (H, W, C1).
[0135] Among them, C1 represents the number of channels of the output feature map obtained after convolution processing of fa.
[0136] At the same time, three parallel dilated convolutional layers are used, each with a different dilation rate, to process fb, fc, and fd respectively, with dilation rates of 1, 3, and 5 respectively. The output feature map of each dilated convolutional layer is (H, W, C 2.m ).
[0137] Among them, C 2.m It represents the number of channels of the output feature map of the corresponding dilated convolutional layer. Its size is the same as the number of channels C1. The values of m are 1, 2, and 3, corresponding to different dilated convolutional layers respectively.
[0138] A global average pooling operation is performed on fe, and the shape of the output feature map is (1, 1, C3). The number of channels is adjusted through 1×1 convolution, and it is restored to the size of the input feature map (H, W, C1) through upsampling.
[0139] Among them, C3 represents the number of channels of the output feature map obtained after the global average pooling operation.
[0140] Then all output feature maps are concatenated to form a new feature map with a shape of (H, W, 4*C1);
[0141] The ASPP module performs another 1×1 convolution to integrate the concatenated feature maps and reduce the number of channels to generate the final output feature map F5. The shape of the output feature map F5 is (H, W, C1).
[0142] Step 3. Train the improved YOLOV8 network model based on the training data set in step 1 to obtain a trained non-magnetic metal classification model, and use the non-magnetic metal classification model to classify the non-magnetic metals of scrapped vehicles.
[0143] In this embodiment, the non-magnetic metal classification model training adopts a self-distillation training strategy.
[0144] This model uses a self-collected database, the characteristics of which are as follows:
[0145] Large amount of data and diverse data: The database contains more than 6,500 thermal images, covering a large amount of scrap metals such as steel, aluminum, copper, and lead, as well as alloys of metals such as magnesium, zinc, nickel, titanium, and platinum.
[0146] Accurate data labeling: The images in the database have been manually labeled and the labeling quality is high.
[0147] Compared with commonly used image datasets including ImageNet, CIFAR-10, CIFAR-100, etc., this dataset is aimed at non-magnetic metal thermal images of scrapped cars, and the data consists of 63 models of cars from three major automobile manufacturers.
[0148] The database contains more than 6,500 thermal images, covering 13 different categories of non-magnetic metals from scrapped cars, and shows good adaptability, pertinence and applicability when training models.
[0149] When training the constructed non-magnetic metal classification model, the learning rate was set to 0.001; the model optimization algorithm adopted the RMSProp optimization algorithm; the batch size was 32, and the number of model iterations was 350.
[0150] The learning rate is an important hyperparameter in deep learning, which controls the update speed of the model during the training process. Since this model is used for image classification and recognition, after many experiments, it was found that when the learning rate is higher than 0.1, the model often has the problem of overfitting; when the learning rate is lower than 0.001, the model training takes a long time, so the learning rate is set to 0.01.
[0151] The RMSProp optimization algorithm is an adaptive learning rate optimization algorithm that dynamically adjusts the learning rate of each parameter by performing an exponentially weighted moving average on the square of the gradient to improve the efficiency and stability of the optimization process.
[0152] The formula for calculating the exponentially weighted moving average of the squared gradient is as follows:
[0153] E [ g 2 ] t = γ E [ g 2 ] t − 1 + ( 1 − γ ) g t 2 ;
[0154] E [ g 2 ] t , E [ g 2 ] t − 1 They are the exponentially weighted moving averages of the squared gradients of the current time step t and the previous time step t-1; is the decay rate, usually set to 0.9, indicating the influence of historical gradients on the current calculation; is the gradient of the current time step t; represents the gradient. When t=0, E [ g 2 ] t =0.
[0155] The parameter update calculation formula is as follows:
[0156] θ t + 1 = θ t − η E [ g 2 ] t + σ ;
[0157] is the vector of parameters for the current time step t, is the vector of parameters for the next time step t+1, is the learning rate, is a small constant used to prevent division by zero errors, usually with the value, η E [ g 2 ] t + σ is the adaptive learning rate.
[0158] The number of iterations of the model is set to 350, at which point the model performance is better and overfitting does not occur.
[0159] In model training, define a comprehensive loss function. A common approach is to combine the cross entropy (CEL) loss (for hard labels) and the Kullback-Leibler (KL) divergence (for soft labels) using a comprehensive loss function.
[0160] Defining cross entropy loss The calculation formula is as follows:
[0161] ;
[0162] in is the total number of categories, is the true label, is the probability of predicting the true label.
[0163] Defining Kullback-Leibler divergence The calculation formula is as follows:
[0164]
[0165] in, represents the true distribution, Represents the probability distribution of the model's predictions.
[0166] The comprehensive loss function is as follows:
[0167] ;
[0168] in, is the comprehensive loss function, , is the weight coefficient.
[0169] The method of the present invention sorts non-magnetic metals based on image visual recognition, and has good sorting effect. The proposed non-magnetic metal classification model based on the improved YOLOV8 network model architecture is simple to calculate and has a fast detection rate. It has a good detection effect on waste metals in real-time motion and does not require a large amount of dwell detection time. Moreover, based on the improved YOLOV8 network model, deeper details can be extracted, so it has a good processing effect on fine particles with mixed colors, shapes, textures and unclear boundaries.
[0170] After the model is trained, it is deployed and the non-magnetic metal classification model is used to classify the non-magnetic metals of scrapped cars. First, an infrared camera is used to collect thermal images of non-magnetic metals of scrapped cars in real time, and the images are uploaded to a computer device. The deployed non-magnetic metal classification model is used to classify the non-magnetic metals of scrapped cars and obtain the classification results.
[0171] The non-magnetic metal classification model based on the improved YOLOV8 network model architecture proposed in the present invention has a strong feature extraction capability, can integrate global and local information, and in the process of scrap metal recycling, through multi-module complementarity, realizes multi-dimensional and multi-scale feature extraction and detection of scrap metal, greatly improves the accuracy of scrap metal recycling, and through data weighting, emphasis and screening operations, improves the model's classification ability and classification efficiency for different types of scrap metals, so that the model meets the rigid requirements of the scrap metal classification system for rapid response and processing in the industrialized scrap metal recycling scenario with high real-time requirements. The scrap car non-magnetic metal classification method proposed in the present invention can efficiently classify and extract non-magnetic metals mixed in scrap cars, reduce the residual non-ferrous metal content in the tailings, thereby reducing dependence on manual sorting, reducing labor costs, and improving material purity and recycling efficiency. At the same time, the efficiency of the method accelerates the recycling process of scrap metals, and provides a sustainable and efficient solution to the problem of a large amount of scrap car metal tailings accumulation.
[0172] Example 2
[0173] This embodiment 2 describes a scrapped car non-magnetic metal classification system, including an infrared camera and a computer device, the computer device includes a memory and at least one processor, and the memory stores executable code.
[0174] The infrared camera is used to collect the thermal imaging image of the non-magnetic metal of the scrapped car in real time and upload the image to the computer device. When the processor executes the executable code, the non-magnetic metal classification method of the scrapped car in the above embodiment 1 is implemented.
[0175] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with the field under the guidance of this specification fall within the essential scope of this specification and should be protected by the present invention.
Claims
1. A method for sorting non-magnetic metals from scrapped vehicles, characterized in that: The steps include: Step 1. Obtain non-magnetic metal thermal images of scrapped vehicles and perform preprocessing and annotation operations to obtain a training data set; Step 2. Build a non-magnetic metal classification model based on the improved YOLOV8 network model architecture, which includes a backbone network, a neck network, and a detection head; the backbone network includes a convolution block, a C2fCIB module, and an ASPP module; There are five convolution blocks, which are defined as the first, second, third, fourth, and fifth convolution blocks respectively; there are four C2fCIB modules, which are defined as the first, second, third, and fourth C2fCIB modules respectively; the processing flow of the backbone network is as follows: The feature map first passes through the first and second convolution blocks, then the first C2fCIB module, then the third convolution block, then the second C2fCIB module, then the fourth convolution block, then the third C2fCIB module, then the fifth convolution block, then the fourth C2fCIB module, and finally the ASPP module before output; The neck network includes upsampling module, combination layer, C2fCIB module, MDAM module and convolution block; There are two upsampling modules, which are defined as the first and second upsampling modules respectively; there are four combination layers, which are defined as the first, second, third and fourth combination layers respectively; there are four C2fCIB modules, which are defined as the fifth, sixth, seventh and eighth C2fCIB modules respectively; there are four MDAM modules, which are defined as the first, second, third and fourth MDAM modules respectively; there are two convolution blocks, which are defined as the sixth and seventh convolution blocks respectively; the processing flow of the neck network is as follows: The output features of the ASPP module are sequentially passed through the first upsampling module and the first combination layer, and combined with the output features of the third C2fCIB module in the first combination layer; The output features of the first combination layer are sequentially passed through the fifth C2fCIB module, the first MDAM module, the second upsampling module and the second combination layer, and combined with the output features of the second C2fCIB module in the second combination layer; The output features of the second combination layer are sequentially passed through the sixth C2fCIB module, the second MDAM module, the sixth convolutional block and the third combination layer, and are combined with the output features of the first combination layer in the third combination layer; The output features of the third combination layer pass through the seventh C2fCIB module, the third MDAM module, the seventh convolutional block and the fourth combination layer in sequence, and are combined with the output features of the ASPP module in the fourth combination layer; The output features of the fourth combination layer pass through the eighth C2fCIB module and the fourth MDAM module in sequence; There are three detection heads, and the second, third and fourth MDAM modules are respectively connected to one of the detection heads; Step 3. Train the improved YOLOV8 network model based on the training data set in step 1 to obtain a trained non-magnetic metal classification model, and use the non-magnetic metal classification model to classify the non-magnetic metals of scrapped vehicles; The C2FCIB module is obtained by replacing the standard bottleneck structure Bottleneck in C2f with the CIB module. The C2FCIB module introduces a data splitting module, which divides the metal feature map into two parts. One part is processed by two CIB modules through depth-wise separable convolution to reduce computational complexity and parameter quantity, and the other part is directly skipped. Finally, the two parts of the feature map are spliced together as the output feature map of the C2FCIB module; The processing flow of the C2FCIB module is as follows: First, it is processed by a convolution block, and then the feature map is split into two parts, feature maps F1 and F2, by the data splitting module; The feature map F1 is directly skipped and the feature map F2 is processed by two CIB modules in sequence. The feature map F2.1 is obtained by the first CIB module, and the feature map F2.2 is obtained by the second CIB module. The processed feature maps F2.1 and F2.2 are concatenated with the jump-connected feature map F1 in the channel dimension to form a feature map F3. Finally, the feature map F3 passes through a convolution block to obtain the final output feature map F4. The CIB module uses depthwise convolution and point-by-point convolution to reduce computational complexity, extracting the edges of scrap metal and overall features to the greatest extent while keeping the spatial size of the feature map unchanged, but without performing cross-channel operations. The MDAM module uses the self-attention module to deeply extract the local features of scrap metals to obtain representative scrap metal feature data, and uses the cross-attention module to cross-learn multiple types of scrap metal data, and uses the preferred scrap metal feature weights to weight the identified scrap metals, so that the model can quickly classify multiple types of scrap metals of uneven sizes; finally, the channel attention module uses the fused data to classify the scrap metals on the color channel to achieve enhanced extraction and recognition of the scrap metal colors; the processing flow of the MDAM module is as follows: By performing three different linear transformations on the original input vector of the feature map, namely, respectively with the training weight matrix W Q , W K , W V Multiply them to get the Q, K, and V values. After passing through the fully connected layer, they enter the self-attention mechanism module and the cross-attention mechanism module respectively, and then perform weighted fusion to obtain the data f1.n; each channel will have an f1.n value. Because there are multiple channels, there are multiple f1.n, where n is a variable; multiple f1.n values are concatenated to obtain the weighted feature map f1, and f1 is processed by the channel attention mechanism module to obtain the weighted value f2. Finally, f1 and f2 are weighted fused to obtain the output scrap metal feature map; The processing flow of the ASPP module is as follows: The ASPP module divides the input feature map F4 (H, W, C) into five parts, namely feature maps fa, fb, fc, fd and fe; where H, W and C represent the length, width and number of channels of the input feature map respectively; Perform a 1×1 convolution on fa, and the shape of the output feature map is (H, W, C1); Among them, C1 represents the number of channels of the output feature map obtained after convolution processing of fa; At the same time, three parallel dilated convolutional layers are used, each with a different dilation rate, to process fb, fc, and fd respectively, with dilation rates of 1, 3, and 5 respectively. The output feature map of each dilated convolutional layer is (H, W, C 2.m ); Among them, C 2.m Represents the number of channels of the output feature map of the corresponding dilated convolutional layer. The size is the same as the number of channels C1. The value of m is 1, 2, and 3, corresponding to different dilated convolutional layers respectively. Perform a global average pooling operation on fe, the shape of the output feature map is (1,1,C3), adjust the number of channels through 1×1 convolution, and restore it to the size of the input feature map (H,W,C1) through upsampling operation; Among them, C3 represents the number of channels of the output feature map obtained after the global average pooling operation; Then all output feature maps are concatenated to form a new feature map with a shape of (H, W, 4*C1); The ASPP module performs another 1×1 convolution to generate the final output feature map F5 with a shape of (H, W, C1).
2. The method for classifying non-magnetic metals of scrapped vehicles according to claim 1, characterized in that: The detection head adopts a decoupling head structure, dividing the data into two major branches: The first branch is the classification branch, which is responsible for predicting the category probability of each detection box. The probability distribution of each category is generated through the convolutional layer and the fully connected layer. When calculating the cross entropy loss, the true label is smoothed. The second branch is the regression branch, which is responsible for predicting the bounding box coordinates of each detection box; the coordinates and size of the bounding box are generated through the convolutional layer and the fully connected layer, and the location and category of the scrap metal are accurately identified through the obtained data.
3. The method for classifying non-magnetic metals of scrapped vehicles according to claim 1, characterized in that: Before using the non-magnetic metal classification model to classify the non-magnetic metal thermal images of scrapped vehicles, first obtain the non-magnetic metal thermal images of scrapped vehicles for model training and preprocess the images. The preprocessing process is as follows: Step 1.
1. Image reading: read the image file into memory; Step 1.
2. Image scaling: Scale the image to 640×640 size supported by the model; Step 1.
3. Image normalization: Use the Sobel operator to normalize the pixel values of the image to the interval [0,1]; The Sobel operator is composed of two 3×3 gradient templates, one for calculating the gradient in the X direction and the other for calculating the gradient in the Y direction. In this process, the non-magnetic metal thermal image of the scrapped car is transformed into a feature image that is easier to identify; The specific calculation method of the Sobel operator is as follows: Convert the image grayscale value to floating point number; perform convolution operation on the image, use the X-direction gradient template to calculate the gradient in the X direction, and use the Y-direction gradient template to calculate the gradient in the Y direction; Sum the square of the gradient in the X direction and the gradient in the Y direction to get the gradient value of the image; Color channel conversion converts the image from RGB format to CHW format, that is, channel-height-width format.
4. The method for sorting non-magnetic metals from scrapped vehicles according to claim 1, characterized in that: In step 3, when training the constructed non-magnetic metal classification model, the learning rate is set to 0.001; the model optimization algorithm adopts the RMSProp optimization algorithm; the batch size is 32, and the number of model iterations is 350; Define the cross entropy loss L CE The calculation formula is as follows: Where N is the total number of categories, y i is the true label, is the probability of predicting the true label; Define Kullback-Leibler divergence D KL The calculation formula of (P||Q) is as follows: Among them, P(i) represents the true distribution, and Q(i) represents the probability distribution predicted by the model; The comprehensive loss function is as follows: L=αL CE +βD KL (P||Q); Among them, L is the comprehensive loss function, and α and β are weight coefficients.
5. The method for sorting non-magnetic metals of scrapped vehicles according to claim 4, characterized in that: The RMSProp optimization algorithm is an adaptive learning rate optimization algorithm that dynamically adjusts the learning rate of each parameter by performing an exponentially weighted moving average on the square of the gradient to improve the efficiency and stability of the optimization process. The formula for calculating the exponentially weighted moving average of the squared gradient is as follows: E[g 2 ] t , E[g 2 ] t-1 are the exponentially weighted moving averages of the squared gradients of the current time step t and the previous time step t-1, γ is the decay rate, and g t is the gradient of the current time step t, g represents the gradient; when t = 0, E[g 2 ] t =0; The parameter update calculation formula is as follows: θ t is the vector of parameters for the current time step t, θ t+1 is the vector of parameters for the next time step t+1, η is the learning rate, σ is a constant used to prevent division by zero errors, is the adaptive learning rate.
6. A scrapped car non-magnetic metal classification system, comprising an infrared camera and a computer device, the computer device comprising a memory and at least one processor, the memory storing executable code, characterized in that: When the processor executes the executable code, the method for classifying non-magnetic metals of scrapped vehicles as claimed in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Part target detection method and system based on infrared thermal imaging data
CN117422977A
Target detection algorithm based on improved YOLOv8s
CN117456167A
Remote sensing image landslide mass detection method based on LS-YOLO network
CN117611877A
Remote sensing image land utilization classification method based on multiple scales
CN117953299A
