Confusable non-ferrous metal crushing material identification and classification method based on image processing algorithm

Abstract

The present application relates to a kind of based on image processing algorithm's easily confused non-ferrous metal broken material identification classification method, comprising the following steps: using improved 5 × 5 median filter algorithm to the small amount of image that industrial camera is previously collected to weaken or reject processing of random noise interference;Then the mask image production and fine granularity division are carried out to the image after filtering processing to obtain a large number of fine granularity pixel units;According to the mask image, fine granularity pixel unit is automatically labeled to obtain sufficient sample data, and sample data is balanced and enhanced to form overall data set;ResNet-RS improved residual neural network is built and using the data set prepared in advance is trained to obtain image recognition classification model;The result of recognition classification is optimized after using the model to the image collected in real time and recognizing classification neighbor augmentation. The present application method can accurately identify the type of non-ferrous metal broken material, so as to greatly improve the sorting efficiency and accuracy of non-ferrous metal broken material.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and industrial detection, and more particularly to an image processing algorithm-based identification and classification method for easily confused nonferrous metal fragments. Background Art

[0002] As we all know, non-ferrous metal scrap is a green energy-carrying resource with high recycling value, making it the best alternative to metal smelting ore. The rational use of scrap metal scrap for smelting can maximize resource recycling and sustainable development, reducing waste and environmental pollution.

[0003] In particular, non-ferrous metal scraps have the characteristics of irregular shapes, large size differences, complex surface conditions, large differences between the same type, and easy confusion between different types. Existing non-ferrous metal scrap visual recognition methods are mostly based on traditional image classification algorithms, including color feature matching, shape feature matching, and texture feature matching methods, but the recognition and classification accuracy is low. In addition, most methods only focus on the recognition and classification of single scraps, and do not consider the simultaneous recognition of multiple scraps, which cannot meet the requirements of actual industrial inspection. On the other hand, the recognition objects are all based on public image datasets, and the recognition scenes and objects are mostly everyday scenes and objects. Their image features are structured, and there is little confusion between different categories. There is little research on the recognition objects such as scrap metal scraps, which have complex surfaces, irregular shapes, and high confusion between different categories. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for identifying and classifying easily confused non-ferrous metal scraps based on an image processing algorithm, which can greatly improve the sorting accuracy and efficiency of non-ferrous metal scraps and reduce the waste of metal resources and environmental pollution.

[0005] The technical solution adopted by the present invention to solve the technical problem is to construct a method for identifying and classifying easily confused non-ferrous metal scraps based on an image processing algorithm, comprising the following steps:

[0006] S1. The image pre-collected by the industrial camera is processed using an improved 5×5 median filter algorithm, and then a mask image is produced and the SEEDS algorithm is used to divide the image into fine-grained pixel units;

[0007] S2. Automatically label fine-grained pixel units according to the mask image to obtain sufficient sample data, and perform data balancing and enhancement on the sample data to form an overall data set;

[0008] S3. Build the improved ResNet-RS residual neural network and use the pre-prepared data set to train it to obtain an image recognition classification model;

[0009] S4. Use the model obtained by offline training to identify and classify the fine-grained pixel units collected in real time, processed by the improved 5×5 median filtering algorithm and fine-grained divided, and perform neighborhood augmentation optimization on the results of identification and classification to improve the accuracy of identification and classification of non-ferrous metal crushed materials.

[0010] According to the above scheme, the improved 5×5 median filter algorithm processing method in steps S1 and S3 is: adaptively adjust the median filter window size for edge pixels, remove the range of the median filter window where the edge pixels of the image are located that exceeds the 5×5 pixel size, and only calculate the median of all pixel values ​​in the window to replace the pixel value of the center pixel of the current window, thereby improving the effect of removing random noise at the edge of the image.

[0011] According to the above scheme, in step S1, the content in the image is segmented manually, and image processing software is used to produce a high-precision real mask image; the SEEDS algorithm is used to divide the collected non-ferrous metal crushed material image into blocks.

[0012] According to the above scheme, in step S2, the method for automatically labeling fine-grained pixel units is: using the mask image as a reference, all fine-grained pixel areas generated by the SEEDS algorithm are automatically marked into three categories: copper, aluminum and background, and fine-grained division is performed according to the above three categories to obtain a large number of fine-grained pixel units.

[0013] According to the above scheme, in step S2, the method for balancing and enhancing the sample data to form an overall data set is: using a downsampling strategy to introduce a Gaussian mixture model to perform unsupervised clustering on a larger proportion of background data, randomly selecting equal amounts of clustering results and adding them to the overall data set to achieve the purpose of sample balancing, and then performing various image transformations on the sample data to achieve data enhancement to improve the generalization ability of the subsequent model, and finally randomly selecting 80% of the overall sample data as a training set, and the remaining 20% ​​as a validation set.

[0014] According to the above scheme, in step S3, the ResNet-RS residual neural network is used for training and identification of non-ferrous metal scrap data, the above-mentioned fine-grained division method is used to obtain data samples as a data set, and the stochastic gradient descent method is used to train the model.

[0015] According to the above scheme, in step S4, the method for identifying and classifying the image collected in real time is: the real-time input image is processed by an improved 5×5 median filtering algorithm and then fine-grained division is preprocessed into fine-grained pixel units of the size specified by the model, and then the image recognition and classification model obtained by offline training is used to identify and classify it, and finally the classified fine-grained pixel units are fused and restored to the original complete image.

[0016] According to the above scheme, in step S4, the method of performing neighborhood augmentation optimization on the recognition and classification results is: summing up each type of fine-grained pixel within the coarse-grained pixel and multiplying it by the prediction accuracy of the corresponding category of the model, and calculating the weight values ​​of the three categories in the coarse-grained pixel. The calculation method is as follows:

[0017]

[0018] Where Pixeli (i = Al, Cu, Bk) represents the number of fine-grained pixels of category i in the coarse-grained pixel area, and Pi (i = Al, Cu, Bk) represents the recognition accuracy of category i;

[0019] For each larger coarse-grained pixel area, if Bk is greater than the sum of Al and Cu, the entire coarse-grained pixel area is classified as background, otherwise it is classified as aluminum or copper. For non-background areas, if the amount of Al exceeds Cu, the entire area will be classified as aluminum, otherwise it will be classified as copper. In addition, areas with an area of ​​less than 1000 pixels are removed from the result image.

[0020] The implementation of the method for identifying and classifying easily confused non-ferrous metal scraps based on an image processing algorithm of the present invention has the following beneficial effects:

[0021] (1) The improved 5×5 median filtering algorithm of the present invention can improve the effect of removing random noise at the edge of the image, weaken or eliminate the random noise interference in the image data of non-ferrous metal crushing materials, thereby improving the training and recognition effects of the image model in subsequent steps.

[0022] (2) The present invention uses the SEEDS algorithm to perform fine-grained segmentation on the filtered nonferrous metal crushed material image to obtain a large number of fine-grained pixel units. The fine-grained pixel units are automatically labeled to obtain sufficient data samples, which are then subjected to sample balancing and data enhancement to form a large data set, providing sufficient data as a guarantee for the subsequent training of the ResNet-RS residual network model.

[0023] (3) The present invention builds a ResNet-RS residual neural network for the training and recognition of non-ferrous metal scrap data. After data balancing and data enhancement, the data samples obtained by the above-mentioned fine-grained partitioning algorithm are divided into 80% as the training set and 20% as the validation set for model training. After training, the final model has the best overall accuracy of 93.92%, which effectively improves the recognition accuracy of non-ferrous metal scrap and has better recognition and classification results than traditional recognition and classification algorithms.

[0024] (4) The present invention optimizes the above-mentioned coarse recognition results by performing neighborhood augmentation. By coarse-grained division of the coarse recognition result image and increasing the fine-grained pixel unit to expand the neighborhood range, more neighborhood information is comprehensively counted and decided, thereby optimizing the coarse recognition results. The final average recognition accuracy after optimization is 98%, which is a significant improvement in recognition accuracy compared with traditional methods.

[0025] In summary, the method for identifying and classifying easily confused non-ferrous metal scraps implemented by the present invention using the improved median filtering algorithm, SEEDS algorithm, ResNet-RS residual neural network algorithm model, and neighborhood augmentation optimization strategy can improve the existing metal scrap sorting process, improve the recognition and classification accuracy of metal scraps, and is of great significance for subsequent actuators to improve the recovery purity and recovery value of waste metals. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0027] Figure 1 This is a general flow chart of a method for identifying and classifying easily confused non-ferrous metal scraps based on an image processing algorithm according to an embodiment of the present invention;

[0028] Figure 2 A flowchart of a data set for a method for identifying and classifying easily confused nonferrous metal scraps based on an image processing algorithm according to an embodiment of the present invention;

[0029] Figure 3 The present invention provides a flowchart of online image recognition and classification of a method for identifying and classifying easily confused non-ferrous metal scraps based on an image processing algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0031] like Figure 1 As shown in FIG, a method for identifying and classifying easily confused nonferrous metal scraps based on an image processing algorithm includes the following steps:

[0032] S1: An industrial camera pre-collects a small amount of non-ferrous metal scrap image data and processes it using an improved 5×5 median filter algorithm. The improvement involves adaptively adjusting the median filter window size for edge pixels. This removes any edge pixels that fall outside the 5×5 pixel median filter window. The median of all pixels within the window is calculated to replace the pixel value at the center of the window, effectively removing random noise from the image edges. This improved 5×5 median filter algorithm can reduce or eliminate random noise interference in non-ferrous metal scrap image data, improving image model training and recognition in subsequent steps.

[0033] S2: After the images are denoised using an improved median filter, the dataset is generated, which consists of four steps: mask image creation, fine-grained image segmentation, automatic fine-grained pixel data annotation, and sample data equalization and enhancement. These four steps yield a large dataset of non-ferrous metal scrap, providing ample data for subsequent model training.

[0034] S3: The non-ferrous metal scrap data set obtained in S2 is used to train an improved residual neural network vertical classification model suitable for the identification and classification of non-ferrous metal scrap. The improved residual neural network ResNet-RS is used as the image recognition and classification network. The use of regularization and scaling strategies can not only make the model converge faster but also achieve higher accuracy and better performance. In addition, by postponing the downsampling of the 1x1 convolution kernel to the 3x3 convolution kernel in the residual block, the large amount of information loss caused by the small convolution kernel can be effectively reduced. The short-circuit characteristic of the residual network can effectively solve the problem of performance degradation caused by the increase in the number of network layers. When the number of network layers is deep, the network only copies the previously learned features to achieve identity mapping, but inserting the residual layer allows the deep network to learn new features, thereby improving the overall performance of the network. Therefore, each network block of the hidden layer in this model contains two residual layers to establish an improved residual neural network with better performance. The definition of the residual layer is as follows:

[0035]

[0036] where x l and x l+1 are the input and output vectors of the middle network layer respectively. A residual unit generally contains a multi-layer structure. The function F represents the residual mapping that the network needs to learn. h(x l )=x l Represents an identity map.

[0037] A ResNet-RS network was built for image recognition and classification of non-ferrous metal scrap. A fixed input fine-grained pixel unit size of 32×32×3 accelerated classification. The entire network was divided into four blocks, each containing four layers and two residual loops, resulting in 18 layers. Each time the image size was halved, the number of convolution kernels doubled. Finally, the fully connected layer had three output channels, outputting scores for the copper, aluminum, and background categories. The SGD algorithm was used for backpropagation, using the Adam optimizer with an initial learning rate of 0.01. After approximately 150 rounds of training, the network accuracy and loss reached a stable state. The model finally converged to a best overall accuracy of 93.92%, which was saved for subsequent online recognition and classification.

[0038] S4: The image collected in real time by the industrial camera is processed by improved median filtering and then divided into fine-grained pixel units of the size specified by the model. The image recognition and classification model obtained in S3 is then used to identify and classify it. The classified fine-grained pixel units are then fused and restored to the original complete image.

[0039] S5: Since the complete material in the result of fine-grained pixel area fusion may contain three different pixels of red, blue and black (i.e. three categories of metal crushed materials), there is a certain error. Therefore, the fused complete material image is coarse-grained to obtain a larger fine-grained pixel unit. The coarse-grained unit contains multiple fine-grained pixel units, which can obtain more neighborhood information, thereby further optimizing the coarse recognition results. The specific optimization process is to sum up each type of fine-grained pixel unit in the coarse-grained area and multiply it by the prediction accuracy of the corresponding category of the model to calculate the weight values ​​of the three categories in the coarse-grained unit. The calculation method is as follows:

[0040]

[0041] Among them, Pixeli (i = Al, Cu, Bk) represents the number of fine-grained pixel units of type i in the coarse-grained area, and Pi (i = Al, Cu, Bk) represents the recognition accuracy of type i.

[0042] For each coarse-grained region, if Bk is greater than the sum of Al and Cu, the entire coarse-grained region is classified as background; otherwise, it is classified as aluminum or copper. For non-background regions, if the amount of Al exceeds that of Cu, the entire region is classified as aluminum; otherwise, it is classified as copper. Furthermore, regions with an area less than 1000 are removed from the resulting image. The optimized results show a significant improvement over the coarse recognition results in step S403, with classification accuracy increased to 98%, very close to the true results, demonstrating the effectiveness of the neighborhood augmentation optimization algorithm.

[0043] Further, if Figure 2 As shown, the S2 specifically includes the following steps:

[0044] S201: First, a small amount of non-ferrous metal crushed material image data that has been pre-collected and processed with an improved median filter and contains special conditions such as uneven lighting, crushed material coating, and background pollution is manually extracted to obtain different types of crushed material areas and background areas. The original image is divided into three categories: copper, aluminum, and background, and manually annotated with red, blue, and black colors to produce a mask image.

[0045] S202: Use the SEEDS method, which works well, to perform fine-grained division on the original image to obtain a large number of fine-grained pixel units. This is specifically divided into the following three steps:

[0046] (1) Initialize the fine-grained pixel unit to St and set the fine-grained pixel unit area at the same interval. This step completes the division of the entire image into rectangles of the same size.

[0047] (2) Update St at the region level or pixel level to obtain a new S. If E(S)>E(St), then St=S; where E(S) is the energy equation, which is expressed as:

[0048] E(s)=H(s)+γG(s)

[0049] Among them, H(s) is the color feature in the fine-grained pixel unit area, and G(s) is the boundary feature.

[0050] a) Color features: Regarding the color feature value, a color histogram is performed for each fine-grained pixel unit. The horizontal axis is the color interval and the vertical axis is cAk(j) (normalized number of pixels). The calculation method is:

[0051]

[0052] Where j is the j bin of the histogram; Ak is the fine-grained pixel unit k; I(i) is the color value of pixel i; Hj is the color interval; Z is the normalization parameter; δ is the indicator function, which is 1 if the condition is met and 0 otherwise. The color features of the fine-grained pixel unit can be calculated as follows:

[0053]

[0054] The calculation method of Ψ(cAk) is as follows:

[0055]

[0056] According to the above formula, the more uniform the color of each fine-grained pixel unit area is, the larger the H(s) value will be. Therefore, the entire iterative process will make the color of each fine-grained pixel unit tend to be consistent.

[0057] b) Boundary eigenvalue: The calculation method of boundary eigenvalue is similar to that of color eigenvalue. An N×N window (N is generally 3) is constructed with the pixel on the boundary as the center, and the pixel in the window is counted to which fine-grained pixel unit area it belongs. The calculation method is as follows:

[0058]

[0059] Then accumulate the statistical results of all windows:

[0060]

[0061] This step tries to make the pixels in a window belong to the same fine-grained pixel unit area (this is impossible to achieve due to the existence of boundary areas). Therefore, the larger G(s) is, the fewer boundary pixels there are and the smoother the boundary is.

[0062] (3) Continue to iterate step 2 until the upper limit of times is reached, and the obtained St is the final partitioning result.

[0063] S203: Automatically classify and label the fine-grained pixel data obtained in S202 with the help of the corresponding mask image obtained in S201. The red, blue and black parts in the mask image represent copper, aluminum and background areas, respectively. After locating a fine-grained pixel area from the original image, find and extract the corresponding area with reference to the mask image, and automatically classify the corresponding area in the original image by counting the pixel information in the area of ​​the mask image. As long as there are metal pixels in the fine-grained pixel unit area, the fine-grained pixel unit area is classified as material, while ensuring that all background samples are pure and without any foreground metal pixels, thereby improving recognition accuracy.

[0064] S204: Since background data accounts for a large proportion of the sample data, data balancing is required to ensure accurate and effective subsequent model training. A Gaussian mixture model is introduced using a downsampling strategy to perform unsupervised clustering on the larger background data. Equal clustering results are randomly selected and added to the overall dataset to achieve sample balance. Various image transformations are then performed on the sample data to achieve data enhancement, improving the generalization ability of the subsequent model. Finally, 80% of the overall sample data is randomly selected as the training set, and the remaining 20% ​​is used as the validation set.

[0065] Further, if Figure 3 As shown, the S4 specifically includes the following steps:

[0066] S401: The image captured in real time by the industrial camera and processed with the improved median filter is divided into fine-grained pixel units using the SEEDS algorithm mentioned in step S2. The pixel units are then resized to the input size required by the model. This pixel unit is then fed into the ResNet-RS residual neural network model pre-trained in step S2.

[0067] S402: Use the pre-trained ResNet-RS residual neural network model to perform coarse recognition on the fine-grained pixel unit data input from step S401, fill the corresponding fine-grained pixel unit area in the image according to the category recognized by the ResNet-RS residual neural network model, and fill the classification results of copper, aluminum and background with red, blue, and black respectively.

[0068] S403: The fine-grained pixel units after coarse recognition in step S402 are fused and reset to obtain a complete coarse recognition image result. Due to the large number of fine-grained pixel unit divisions, the ResNet-RS residual neural network model cannot ensure that all fine-grained pixel unit classification results are correct. Therefore, it is necessary to perform neighborhood augmentation optimization on the results of fine-grained pixel area fusion.

[0069] In summary, this method uses an improved 5×5 median filtering algorithm to weaken or even eliminate the random noise of the image. The fine-grained partitioning algorithm provides a large data set required for model training and preprocesses the filtered image during the online recognition process. An improved residual neural network model is constructed based on ResNet-RS and combined with the neighborhood augmentation optimization algorithm, which is innovatively applied to the recognition and classification of non-ferrous metal crushed materials, and has practical application value in the field of industrial detection.

[0070] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A method for identifying and classifying easily confused nonferrous metal scraps based on an image processing algorithm, characterized in that: The following steps are involved: S1. The image pre-collected by the industrial camera is processed using an improved 5×5 median filter algorithm, and then a mask image is created and the SEEDS algorithm is used to divide the image into fine-grained pixel units; S2. Automatically label fine-grained pixel units according to the mask image to obtain sufficient sample data, and perform data balancing and enhancement on the sample data to form an overall data set; S3. Build the improved ResNet-RS residual neural network and use the pre-prepared data set to train it to obtain an image recognition classification model; Each network block of the hidden layer in the image recognition classification model contains two residual layers, which are defined as follows: in and They are the input and output vectors of the middle network layer respectively. A residual unit generally contains a multi-layer structure. Function represents the residual mapping that the network is to learn, represents the identity mapping; Build a ResNet-RS network for non-ferrous metal scrap image recognition and classification. The entire network is divided into four blocks. Each network block contains a four-layer network and two residual loops. The final fully connected layer has three output channels to output scores for copper, aluminum, and background categories. S4. Use the model obtained by offline training to identify and classify the fine-grained pixel units collected in real time, processed by the improved 5×5 median filtering algorithm and fine-grained divided, and perform neighborhood augmentation optimization on the results of identification and classification to improve the accuracy of identification and classification of non-ferrous metal crushed materials.

2. The method for identifying and classifying easily confused nonferrous metal scraps based on image processing algorithm according to claim 1 is characterized in that: The improved 5×5 median filter algorithm processing method in steps S1 and S3 is as follows: adaptively adjust the median filter window size for edge pixels, remove the range of the median filter window where the edge pixels of the image are located that exceeds the 5×5 pixel size, and only calculate the median of all pixel values ​​within the window to replace the pixel value of the center pixel of the current window, thereby improving the removal effect of random noise on the edge of the image.

3. The method for identifying and classifying easily confused nonferrous metal scraps based on image processing algorithm according to claim 1, characterized in that: In step S1, the content in the image is segmented manually, and image processing software is used to produce a high-precision true mask image; the SEEDS algorithm is used to divide the collected non-ferrous metal crushed material image into blocks.

4. The method for identifying and classifying easily confused nonferrous metal scraps based on an image processing algorithm according to claim 1, characterized in that: In step S2, the method for automatically labeling fine-grained pixel units is: using the mask image as a reference, all fine-grained pixel areas generated by the SEEDS algorithm are automatically marked into three categories: copper, aluminum and background, and fine-grained division is performed according to the above three categories to obtain a large number of fine-grained pixel units.

5. The method for identifying and classifying easily confused nonferrous metal scraps based on image processing algorithm according to claim 1, characterized in that: In step S2, the method for performing data balancing and enhancement on the sample data to form an overall data set is: using a downsampling strategy to introduce a Gaussian mixture model to perform unsupervised clustering on a larger proportion of background data, randomly selecting equal amounts of clustering results and adding them to the overall data set to achieve sample balancing, then performing various image transformations on the sample data to achieve data enhancement to improve the generalization ability of the subsequent model, and finally randomly selecting 80% from the overall sample data as a training set, and the remaining 20% ​​as a validation set.

6. The method for identifying and classifying easily confused nonferrous metal scraps based on image processing algorithm according to claim 1, characterized in that: In step S3, the ResNet-RS residual neural network is used for training and identification of non-ferrous metal scrap data. The above-mentioned fine-grained partitioning method is used to obtain data samples as a data set, and the stochastic gradient descent method is used to train the model.

7. The method for identifying and classifying easily confused nonferrous metal scraps based on image processing algorithm according to claim 1, characterized in that: In step S4, the method for identifying and classifying the image collected in real time is as follows: the real-time input image is processed by an improved 5×5 median filtering algorithm and then fine-grained division is preprocessed into fine-grained pixel units of the size specified by the model, and then the image recognition and classification model obtained by offline training is used to identify and classify it, and finally the classified fine-grained pixel units are fused and restored to the original complete image.

8. The method for identifying and classifying easily confused nonferrous metal scraps based on an image processing algorithm according to claim 1, characterized in that: In step S4, the method for performing neighborhood augmentation optimization on the recognition and classification results is: summing up each type of fine-grained pixel within the coarse-grained pixel and multiplying it by the prediction accuracy of the corresponding category of the model, and calculating the weight values ​​of the three categories in the coarse-grained pixel. The calculation method is as follows: in, Represents the type of coarse-grained pixel area The number of fine-grained pixels, Representative types Recognition accuracy; For each larger coarse-grained pixel area, if Bk is greater than the sum of Al and Cu, the entire coarse-grained pixel area is classified as background, otherwise it is classified as aluminum or copper. For non-background areas, if the amount of Al exceeds Cu, the entire area will be classified as aluminum, otherwise it will be classified as copper. In addition, areas with an area of ​​less than 1000 pixels are removed from the result image.

Images