An image recognition-based waste metal classification and recognition method and system

Abstract

The application relates to the technical field of industrial visual detection, and particularly discloses a waste metal classification and recognition method and system based on image recognition, which obtains metal surface visual information through an image acquisition system, extracts multi-level depth features after preprocessing, and generates a preliminary classification result and a confidence evaluation; when the confidence is insufficient, a multi-modal verification mechanism is started, laser-induced breakdown spectroscopy technology is used to obtain element composition data, and a structured light three-dimensional scanning technology is used to obtain surface topological features; the element data is matched with a composition database to generate a composition verification result, and the topological features are compared with a topological feature library to generate a topological feature verification result; finally, a weighted fusion algorithm is used to integrate the three types of results to generate a final classification decision, and a sorting mechanism is controlled to complete accurate sorting.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial visual inspection, in particular to a waste metal classification and identification method and system based on image recognition. BACKGROUND

[0002] With the rapid development of circular economy, the waste metal recycling industry has higher requirements for the automation and intelligence level of sorting technology. Traditional waste metal sorting mainly relies on manual work, which is low in efficiency, high in labor intensity, and the sorting accuracy is significantly affected by the experience and fatigue of workers, which is difficult to meet the needs of large-scale industrial processing.

[0003] The prior art has the following disadvantages:

[0004] In recent years, automatic sorting technology based on machine vision has been widely used. This technology collects metal surface images, extracts color, texture and other features using image processing algorithms, and combines classifiers to realize material identification. However, in actual industrial environments, waste metal surfaces often have varying degrees of oxidation, oil stains, coatings or wear, which can significantly change their visual appearance features, leading to misjudgment of classification systems based on single visual information. For example, severely oxidized copper materials may be misjudged as rusty steel, and aluminum alloys with oil stains may be confused with stainless steel. SUMMARY

[0005] The purpose of the present application is to provide a waste metal classification and identification method and system based on image recognition to solve the problems in the background.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] A waste metal classification and identification method based on image recognition, comprising the following steps:

[0008] S1: Obtain the surface visual information of the waste metal to be sorted, and extract its multi-level depth features after preprocessing the surface visual information, including: surface color and texture features, middle layer contour shape features and deep layer semantic features;

[0009] S2: Use a deep neural network to identify the multi-level depth features to generate a preliminary classification result of the waste metal and a corresponding confidence evaluation value; when the confidence evaluation value is lower than a preset threshold, collect the element spectrum information and three-dimensional structure information of the corresponding waste metal;

[0010] S3: Perform spectral analysis on the element spectrum information to obtain element composition data, and match the element composition data with a standard composition database to generate a composition verification result;

[0011] The point cloud processing is performed on the three-dimensional structure information to obtain surface topological features, the surface topological features are compared with a reference topography library, and a topography verification result is generated.

[0012] S4: comprehensively judging the preliminary classification result, the composition verification result and the topography verification result, generating a final classification decision, and controlling a sorting execution mechanism to complete accurate sorting of the waste metal according to the final classification decision.

[0013] As a further scheme of the present application, the multi-level deep feature extraction process comprises:

[0014] The multi-scale convolution kernel group is used to perform parallel convolution operation on the preprocessed surface visual information, color distribution features and texture detail features at different scales are extracted, and the multi-scale features are fused to generate the surface color texture features.

[0015] Based on the surface color texture features, the spatial pyramid pooling layer is used to capture the macro contour structure and local geometric shape of the waste metal, and the edge enhancement algorithm is used to strengthen the contour boundary information, thereby generating the middle layer contour shape features.

[0016] The middle layer contour shape features are input into the deep residual network, the deep semantic correlation between the features is mined through cascaded nonlinear transformation, and the deep semantic features with high discriminability are output.

[0017] As a further scheme of the present application, the specific implementation process of the preliminary classification result comprises:

[0018] The surface color texture features, the middle layer contour shape features and the deep semantic features are cross-level feature fused to form a comprehensive feature representation; the comprehensive feature representation is input into the multi-layer perceptron of the deep neural network for nonlinear transformation; the probability distribution of the waste metal belonging to each preset category is obtained through forward propagation calculation; and the category corresponding to the maximum value in the probability distribution is selected as the preliminary classification result.

[0019] As a further scheme of the present application, the confidence evaluation value acquisition process comprises:

[0020] The information entropy of the probability distribution is calculated, the information entropy is used to represent the uncertainty degree of the classification decision; the clustering tightness of the feature vector corresponding to the preliminary classification result in the feature space is calculated; the uncertainty degree and the clustering tightness are weighted and summed, and the summed result is normalized, and the normalized value is taken as the confidence evaluation value.

[0021] As a further scheme of the present application, the element spectrum information is spectrally analyzed to obtain element composition data, which specifically comprises:

[0022] The collected element spectrum information is preprocessed to eliminate environmental noise and instrument background interference; the preprocessed spectrum signal is subjected to multi-resolution analysis by using continuous wavelet transform to extract spectrum feature peaks at different scales; the chemical element types corresponding to each feature peak are identified by a feature peak matching algorithm; the relative content of each element is calculated based on the integral of the feature peak intensity to finally generate element composition data containing element types and content.

[0023] As a further scheme of the application, the element composition data is matched with a standard composition database to generate a composition verification result, specifically comprising:

[0024] The cosine similarity of the element composition data and the element content vectors of each standard grade in the standard composition database is calculated; higher weights are given to main alloy elements according to element importance; the standard grade with the highest weighted similarity is selected as the candidate matching result; when the highest weighted similarity is greater than or equal to a preset confidence threshold, the standard grade is directly output as the composition verification result; when the highest weighted similarity is less than the preset confidence threshold, a matching failure flag is output and an artificial review process is started.

[0025] As a further scheme of the application, the three-dimensional structure information is subjected to point cloud processing to obtain surface topological features, specifically comprising:

[0026] The collected three-dimensional point cloud data is subjected to denoising and smoothing processing to eliminate measurement errors and abnormal points; a voxel grid downsampling method is used to reduce the point cloud density while maintaining shape features; the normal vector and curvature features of the local surface of the point cloud are calculated by principal component analysis; the macro relief features and micro roughness features of the point cloud data are extracted to finally generate multi-scale fused surface topological features.

[0027] As a further scheme of the application, the surface topological features are compared with a reference morphology library to generate a morphology verification result, specifically comprising:

[0028] The Hausdorff distance of the surface topological features and the standard sample morphology feature vectors in the reference morphology library is calculated; a hierarchical matching strategy is adopted to first perform coarse matching to screen out a candidate sample set and then perform fine matching to determine the optimal matching sample; the morphology verification result is generated based on the distance similarity score and matching consistency index in combination with a preset confidence threshold; when the matching score is greater than or equal to the preset threshold, the corresponding material category is output, otherwise an artificial intervention process is triggered.

[0029] As a further scheme of the application, the preliminary classification result, the composition verification result and the morphology verification result are comprehensively evaluated to generate a final classification decision, specifically comprising:

[0030] Based on the confidence index corresponding to each result, different confidence weights are assigned to the preliminary classification result, component verification result, and morphology verification result. A weighted fusion method is used to comprehensively evaluate the three results, with the component verification result having the highest weight, the morphology verification result having the second highest weight, and the preliminary classification result having the lowest weight. When the weighted comprehensive score is greater than or equal to a preset threshold, the category with the highest score is output as the final classification decision. When the comprehensive score is less than the preset threshold, manual review is initiated.

[0031] A scrap metal sorting and identification system based on image recognition, comprising:

[0032] A multi-level visual feature extraction module is used to obtain the surface visual information of the scrap metal to be sorted. After preprocessing the surface visual information, its multi-level depth features are extracted, including: surface color and texture features, middle contour and shape features and deep semantic features.

[0033] The visual preliminary classification and confidence assessment module uses a deep neural network to identify multi-level deep features to generate preliminary classification results of scrap metal and corresponding confidence assessment values; when the confidence assessment value is lower than a preset threshold, the elemental spectral information and three-dimensional structural information of the corresponding scrap metal are collected.

[0034] A multimodal data acquisition and processing module is used to perform spectral analysis on elemental spectral information to obtain elemental composition data, match the elemental composition data with a standard component database, and generate component verification results.

[0035] The three-dimensional structural information is processed into point cloud to obtain surface topological features. The surface topological features are then compared with the benchmark topological library to generate topological verification results.

[0036] The multi-source verification and decision fusion module comprehensively evaluates the preliminary classification results, composition verification results, and morphology verification results to generate a final classification decision. Based on the final classification decision, the module controls the sorting execution mechanism to complete the precise sorting of waste metals.

[0037] The beneficial effects of this invention are:

[0038] (1) By employing parallel convolution operations with multi-scale convolution kernel groups to extract surface color and texture features, combining spatial pyramid pooling with edge enhancement algorithms to obtain mid-level contour and shape features, and utilizing deep residual networks to mine deep semantic features, this invention can comprehensively capture various detailed information about the surface of scrap metal. This multi-level feature extraction method not only improves the expressive power of individual feature types but also enhances the complementarity between features at different levels through cross-level feature fusion. These features are input into a deep neural network for nonlinear transformation, making the preliminary classification results of scrap metal more accurate and reliable. This provides a solid foundation for subsequent component verification and morphology verification, thereby ensuring the efficiency and accuracy of the entire sorting process.

[0039] (2) After generating the preliminary classification results, this invention introduces composition verification and morphology verification steps based on elemental spectral information and three-dimensional structural information. By performing weighted similarity matching between the elemental composition data and a standard composition database, and by denoising, downsampling, and extracting features from the point cloud data before comparing it with a benchmark morphology database, composition verification results and morphology verification results are obtained, respectively. These two verification methods provide in-depth analysis of waste metals from both chemical composition and physical morphology perspectives, improving the accuracy and reliability of classification decisions. Weighted fusion decisions are made based on the confidence indices corresponding to each verification result, further ensuring the stability and reliability of the final classification results. Even in complex or uncertain situations, the correctness of the classification decision can be ensured by initiating a manual review process. Attached Figure Description

[0040] The invention will now be further described with reference to the accompanying drawings.

[0041] Figure 1 This is a flowchart of the method of the present invention;

[0042] Figure 2 This is a flowchart of the system in this invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Please see Figure 1 As shown, this invention is a method for classifying and identifying scrap metal based on image recognition, comprising the following steps:

[0045] S1: Obtain the surface visual information of the scrap metal to be sorted. After preprocessing the surface visual information, extract its multi-level depth features, including: surface color and texture features, middle contour and shape features, and deep semantic features.

[0046] S2: Use deep neural networks to identify multi-level deep features to generate preliminary classification results of scrap metal and corresponding confidence assessment values; when the confidence assessment value is lower than the preset threshold, collect the elemental spectral information and three-dimensional structural information of the corresponding scrap metal.

[0047] S3: Perform spectral analysis on the elemental spectral information to obtain elemental composition data, match the elemental composition data with the standard component database, and generate component verification results;

[0048] The three-dimensional structural information is processed into point cloud to obtain surface topological features. The surface topological features are then compared with the benchmark topological library to generate topological verification results.

[0049] S4: Comprehensively evaluate the preliminary classification results, component verification results, and morphology verification results to generate the final classification decision. Based on the final classification decision, control the sorting execution mechanism to complete the precise sorting of waste metal.

[0050] In S1, surface visual information of the scrap metal to be sorted is acquired. After preprocessing the surface visual information, its multi-level depth features are extracted, including: surface color and texture features, mid-level contour and shape features, and deep semantic features, specifically including:

[0051] The system acquires surface visual information of scrap metal to be sorted, including a high-resolution industrial camera, a uniform illumination unit, and a mechanical conveyor. The specific acquisition process is as follows: the scrap metal is transported to the image acquisition station via the mechanical conveyor, triggering a photoelectric sensor to initiate the image acquisition process; the uniform illumination unit uses a ring-shaped LED white light source, tilted at a 45-degree angle to illuminate the metal surface to eliminate specular reflections and highlight surface texture details; the high-resolution industrial camera is vertically positioned over the metal surface, using a global shutter mode to acquire digital images. The image resolution is set to 2448×2048 pixels, the color depth to 24-bit true color, and the frame rate is matched to the conveyor belt speed at 30 frames per second.

[0052] The preprocessing of surface visual information includes the following steps: First, image denoising is performed using a nonlocal mean denoising algorithm with a search window size of 21×21 pixels, a similarity window size of 7×7 pixels, and a Gaussian function standard deviation of 1.2. Next, color correction is performed by establishing a color correction matrix using a standard color chart and converting the image color space to the standard sRGB color space using a multinomial regression algorithm. Then, background segmentation is performed using an improved GrabCut segmentation algorithm, based on a Gaussian mixture model of the foreground and background, and accurately separating the metal target from the conveyor belt background through an iterative energy minimization process. Finally, image enhancement is performed using a contrast-limited adaptive histogram equalization method with a tile grid size of 8×8 and a contrast limit coefficient of 2.0 to enhance the visibility of surface details.

[0053] The extraction of multi-level deep features is accomplished through three sequential stages, each extracting and transforming feature information at different levels. First, the preprocessed surface visual information undergoes parallel convolution operations using a multi-scale convolution kernel group. This operation uses convolution kernels with different receptive fields to process the input image simultaneously. Smaller convolution kernels focus on capturing fine-grained texture details and local color variations in the image, with a stride of 1 pixel and padding set to "same" to ensure the spatial size of the output feature map remains unchanged. Larger convolution kernels are used to extract color distribution features and macroscopic texture patterns over a wider range, with a stride of 2 or 3 pixels depending on the needs. The output of each convolution kernel is processed by a non-linear activation function, using ReLU to introduce non-linear transformation capabilities. The feature maps obtained from convolution operations at different scales are integrated through a feature fusion layer. This fusion layer uses channel concatenation to combine multi-scale features into a unified multi-channel feature representation, ultimately generating comprehensive surface color and texture features.

[0054] Based on the obtained surface color and texture features, a spatial pyramid pooling layer is further used to capture the macroscopic contour structure and local geometry of scrap metal. The spatial pyramid pooling layer employs three different scales of pooling windows: 4x4, 8x8, and 16x16 grid divisions. Max pooling is performed within each grid cell to extract salient features within that region. The pooling outputs at different scales are flattened into one-dimensional feature vectors and then concatenated to form a multi-resolution feature representation. During this process, an edge enhancement algorithm is used to post-process the feature map, employing the Sobel operator to calculate the gradient magnitude and direction of the feature map, thus strengthening the contour boundary information. Gradient calculations are performed in both horizontal and vertical directions, obtaining the horizontal and vertical gradient components through convolution operations. The square root of the sum of the squares of the two components yields the final gradient magnitude. The enhanced boundary information is then weighted and fused with the spatial pyramid pooling features, with the weight coefficients adaptively adjusted according to feature importance, ultimately generating a mid-level contour shape feature containing rich geometric information.

[0055] The mid-level contour shape features are input into a deep residual network for further feature transformation and semantic information mining. The deep residual network consists of multiple stacked residual blocks, each containing two convolutional layers and a shortcut connection. The first convolutional layer uses a 1x1 kernel for dimensionality reduction, reducing the number of input feature channels to one-quarter of the original. The second convolutional layer uses a 3x3 kernel for feature transformation, extracting spatial feature information. The third convolutional layer uses a 1x1 kernel for dimensionality increase, restoring the number of feature channels. Batch normalization is performed after each convolutional layer to standardize the output features, making their mean 0 and variance 1. The shortcut connection directly adds the input and output of the residual block, avoiding the vanishing gradient problem. Through the cascading processing of multiple residual blocks, nonlinear transformations are layered, gradually mining deep semantic relationships between features. The output features of the final layer undergo global average pooling, compressing the spatial dimension to 1x1 while preserving channel dimension information, outputting highly discriminative deep semantic features. The entire network is trained end-to-end, and the network parameters are optimized through the backpropagation algorithm, which enables the deep semantic features to have better class discrimination ability.

[0056] In S2, the process of using a deep neural network to identify multi-level deep features to generate preliminary classification results for scrap metal includes the following steps. First, surface color and texture features, mid-level contour and shape features, and deep semantic features are fused across levels using a weighted feature concatenation method. Each level of features is normalized to have the same numerical range, and then different weight coefficients are assigned based on the importance of the features. The weight coefficient for surface color and texture features is set to 0.3, the weight coefficient for mid-level contour and shape features is set to 0.4, and the weight coefficient for deep semantic features is set to 0.3. The weighted feature vectors are then concatenated along the feature dimensions to form a comprehensive feature representation vector. The dimension of this vector is the sum of the dimensions of each input feature, preserving all the information from the original features at each level.

[0057] The resulting comprehensive feature representation is input into a multilayer perceptron (MLP) of a deep neural network for nonlinear transformation. This MLP consists of three fully connected layers with 512, 256, and 128 neurons respectively. The first fully connected layer receives the comprehensive feature representation as input, performs a linear transformation using matrix multiplication and bias addition, and then introduces nonlinearity using the ReLU activation function. The second fully connected layer further transforms the output of the first layer, also using the ReLU activation function. The third fully connected layer outputs a feature vector with the same dimension as the preset number of classes. A Dropout layer is added after each fully connected layer with a dropout rate of 0.2 to prevent overfitting. The weight parameters of each layer are initialized using the Xavier method, and the bias parameters are initialized to 0.

[0058] The probability distribution of scrap metal belonging to each preset category is obtained through forward propagation. The output of the third fully connected layer is input into the softmax function, which converts the output value into a probability distribution. The softmax function performs an exponential operation on the score of each category and then divides it by the sum of the exponents of all category scores, ensuring that the sum of the probabilities of all categories is 1. During the calculation, the exponential operation result is numerically stabilized by subtracting the maximum value in the score vector to avoid numerical overflow. In the final probability distribution vector, each element corresponds to the probability value of a preset category, and all probability values ​​are between 0 and 1, with a sum of 1.

[0059] The category corresponding to the maximum value in the probability distribution is selected as the preliminary classification result. All elements in the probability distribution vector are traversed to find the index of the element with the maximum probability value. The correspondence between this index and the preset category label is determined through a pre-established mapping table. The maximum probability value and its corresponding category label are recorded as the preliminary classification result. Simultaneously, all probability values ​​in the probability distribution vector are recorded for subsequent confidence level calculations. The entire recognition process is executed in a GPU-accelerated environment, using a batch processing method, processing 16 samples at a time to improve computational efficiency.

[0060] The process of obtaining the confidence assessment value specifically includes the following calculation steps. First, the information entropy of the probability distribution is calculated to characterize the degree of uncertainty in the classification decision. The formula for calculating the information entropy is to multiply the probability value of each category in the probability distribution vector by the base-2 logarithm of that probability value, then take the negative sign and sum them. In specific calculations, for each probability value in the probability distribution vector, it is first determined whether it is greater than 0. If the probability value is 0, then the contribution of that item is 0; if the probability value is greater than 0, the logarithm of that probability value is calculated, multiplied by the probability value, and then negative. The calculation results for all categories are summed to obtain the final information entropy value. The value of the information entropy ranges from 0 to log2C, where C is the number of categories. The larger the information entropy value, the higher the uncertainty of the classification decision.

[0061] Simultaneously, the cluster density of the feature vectors corresponding to the preliminary classification results in the feature space is calculated. Cluster density is characterized by calculating the distance between the feature vector and the center of the feature vectors of the same category. First, the coordinates of the center points of the feature vectors of each category saved during the training phase are obtained. These center points are obtained by calculating the average value of the feature vectors of all similar samples in the training set. Then, the Euclidean distance between the current sample feature vector and the center point of its category is calculated. The distance is calculated using the square root method after the sum of squares. First, the sum of squares of the differences in each dimension of the two vectors is calculated, and then the square root of the sum of squares is taken. To facilitate subsequent processing, the distance value is normalized, mapping it to the range of 0 to 1. The normalization process uses the maximum distance value obtained statistically during the training phase for scaling.

[0062] The uncertainty level and cluster density are weighted and summed. Uncertainty is represented by information entropy, and cluster density by normalized distance. Since a higher information entropy indicates higher uncertainty, while a higher distance indicates lower cluster density, consistency between these two metrics is necessary. The information entropy is divided by the maximum possible entropy log₂C, normalizing it to between 0 and 1. The normalized information entropy and distance are then weighted and summed using weighted coefficients determined based on experimental results; the weighting coefficient for information entropy is set to 0.6, and for distance, it is set to 0.4. The weighted sum is a value between 0 and 1, with a higher value indicating lower confidence.

[0063] The weighted summation result is normalized to map the values ​​to a standard range of 0 to 1. The normalization process uses a linear transformation method: subtracting the minimum value (0) from the weighted summation result and dividing by the maximum value (1), while maintaining the numerical range. Then, the normalized value is inverted (subtracted from 1), resulting in a higher confidence level. The final value is output as the confidence assessment value. When the confidence assessment value is lower than a preset threshold (0.7), the system determines the preliminary classification result is unreliable and requires initiating a multimodal data acquisition process to obtain elemental spectral information and three-dimensional structural information for further verification.

[0064] Elemental spectral information is acquired through laser-induced breakdown spectroscopy (LAS). This process is performed by a LAS spectrometer, whose core components include a pulsed laser, a spectrometer, a focusing optical system, and a signal acquisition unit. Specifically, when the scrap metal to be tested arrives at the testing station, the pulsed laser is triggered and emits a high-energy, nanosecond-width laser pulse. This pulse is focused by a focusing lens, forming a spot with extremely high power density on the metal surface, instantly vaporizing trace amounts of material and exciting the generation of high-temperature plasma. During the cooling process, the excited atoms and ions within this plasma emit characteristic spectra with specific wavelengths. This spectral signal is captured by the collecting optical system and coupled to an optical fiber, transmitted to the spectrometer for dispersion, and finally received by a charge-coupled device (CCD) detector and converted into a digital spectral signal containing the intensity-wavelength relationship—the required elemental spectral information.

[0065] The acquisition of 3D structural information is achieved through structured light 3D scanning technology. This process is completed collaboratively by a structured light projection module and an image acquisition module. The projection module projects a series of coded grating stripe patterns (such as sine stripes and Gray code) onto the surface of the scrap metal. Due to the surface undulations, these stripes undergo geometric distortion. A high-resolution industrial camera located at another viewpoint simultaneously acquires a series of distorted stripe images. Based on the principle of optical triangulation, the absolute phase distribution map of the object's surface is reconstructed from the distorted stripes using phase calculation algorithms (such as phase shift and multi-frequency heterodyne methods). Combined with the geometric parameters between the camera and the projector obtained through high-precision calibration in advance, the phase values ​​are converted into 3D spatial coordinates of each point on the object's surface, ultimately generating dense 3D point cloud data, which is the required 3D structural information.

[0066] In S3, spectral analysis is performed on the elemental spectral information to obtain elemental composition data. This data is then matched with a standard composition database to generate composition verification results. Point cloud processing is performed on the 3D structural information to obtain surface topological features. These features are then compared with a benchmark morphology library to generate morphology verification results. Specifically, this includes:

[0067] The process of obtaining elemental composition data through spectral analysis of elemental spectral information includes the following steps. First, the acquired elemental spectral information is preprocessed to eliminate environmental noise and instrument background interference. The preprocessing includes dark current correction, wavelength calibration, and intensity normalization. Dark current correction involves subtracting the background signal from the original spectral signal when no light source is present. Wavelength calibration uses the characteristic spectral lines of a standard mercury-argon lamp as a reference to precisely calibrate the wavelength axis of the spectrometer. Intensity normalization converts the spectral intensity values ​​to a standard range of 0 to 1, eliminating the influence of laser energy fluctuations. The preprocessed spectral signal is smoothed using a Savitzky-Golay filter with a window size of 11 points and a polynomial order of 3, suppressing random noise while preserving spectral characteristics.

[0068] Continuous wavelet transform was employed to perform multi-resolution analysis on the preprocessed spectral signal, extracting spectral characteristic peaks at different scales. Morlet wavelets were selected as the basis function for the continuous wavelet transform due to their good time-frequency localization properties. The scale parameter varied from 10 to 200 in step size 2, corresponding to different frequency components. Modulus maxima detection was performed on the transform coefficients at each scale to identify local extrema as candidate characteristic peak locations. By connecting correlated extrema at different scales, a multi-scale description of the characteristic peaks was formed. For each characteristic peak, its center wavelength position, peak height, full width at half maximum (FWHM), and peak area were recorded. Finally, all significant spectral characteristic peaks were extracted, forming a characteristic peak list.

[0069] The chemical element type corresponding to each characteristic peak is identified using a characteristic peak matching algorithm. An elemental standard spectral database is established, containing the standard wavelength positions and relative intensities of each element's characteristic peaks. A spectral line matching algorithm is used to compare the detected characteristic peaks with the standard spectral lines in the database. A certain wavelength drift is allowed during the matching process, with a drift range of ±0.1 nm. For each detected characteristic peak, its matching score with each standard spectral line is calculated, taking into account wavelength deviation, intensity consistency, and peak shape similarity. The standard spectral line with the highest matching score is selected as the element type corresponding to that characteristic peak. When multiple characteristic peaks point to the same element, the relative intensities of these characteristic peaks must be consistent with the relative intensities in the standard spectral line database.

[0070] The relative abundance of each element is calculated based on the integral of characteristic peak intensities, ultimately generating elemental composition data containing element types and abundances. For each identified element, its most representative characteristic peak is selected for intensity integration. The integration interval is within ±0.5 nm of the characteristic peak's center wavelength, and the integration area is calculated using the trapezoidal rule. Based on a pre-established calibration curve, the characteristic peak area is converted into elemental abundance. The calibration curve is established by measuring standard samples and includes the correspondence between characteristic peak area and abundance at different abundance levels. For cases with multiple elements coexisting, a matrix effect correction algorithm is used to consider the mutual influence between elements. The final elemental composition data includes the type, abundance, and measurement uncertainty of each element, in a structured data table format.

[0071] The process of matching elemental composition data with a standard component database to generate component verification results specifically includes the following steps: Calculating the cosine similarity between the elemental composition data and the element content vectors of each standard brand in the standard component database. First, the elemental composition data is converted into feature vectors, with the vector dimensions consistent with the standard component database, each dimension corresponding to the content value of one element. For elements not detected in the elemental composition data, their content value is set to 0. Each brand in the standard component database is also represented as a vector of the same dimension. The cosine similarity is calculated using the vector dot product formula, multiplying the dot product of two vectors by their respective magnitudes. The calculation result ranges from 0 to 1, with a larger value indicating a higher similarity.

[0072] Major alloying elements are assigned higher weights based on their importance. The weighting is determined by the element's crucial role in the alloy: a weight coefficient of 1.2 for major alloying elements, 1.0 for minor elements, and 0.8 for trace elements. Weighted similarity is calculated during the cosine similarity calculation, multiplying each dimension by its corresponding weight coefficient. These weight coefficients are determined through expert experience and experimental verification to ensure that important elements have a greater influence on the matching results. The weighted similarity calculation better distinguishes between different grades with similar compositions, improving matching accuracy.

[0073] The standard brand with the highest weighted similarity is selected as the candidate matching result. All brands in the standard component database are traversed, and the weighted similarity between each brand and the sample to be tested is calculated. All similarity values ​​and their corresponding brand information are recorded. The brand with the highest similarity value is found and selected as the best matching candidate. The second highest similarity value and its corresponding brand are also recorded for subsequent reliability evaluation. When multiple brands have the same highest similarity, the brand with the higher priority in the brand standard is selected as the candidate matching result.

[0074] When the highest weighted similarity is greater than or equal to the preset confidence threshold, the standard brand name is directly output as the component verification result. The confidence threshold is set to 0.85, a value obtained through statistical analysis of a large amount of experimental data. The output results include the matched brand name, similarity value, and matching confidence level. When the highest weighted similarity is less than the preset confidence threshold, a matching failure flag is output, and a manual review process is initiated. The matching failure flag contains detailed matching information, including the best-matching brand name, similarity value, and second-best-matching brand name. The manual review process involves professional technicians confirming the matching results, and other analytical methods are used for verification when necessary.

[0075] The process of processing 3D structural information into point clouds to obtain surface topological features specifically includes the following steps: The acquired 3D point cloud data is denoised and smoothed to eliminate measurement errors and outliers. A statistical outlier removal algorithm is used to calculate the average distance between each point and its k neighboring points, where k is set to 50. A distance threshold of 1.5 times the average distance is set, and points exceeding this threshold are marked as noise points and removed. Smoothing is performed using the moving least squares method to locally fit the point cloud surface, with the fitting polynomial having an order of 2. During smoothing, the sharpness of feature edges is maintained to avoid over-smoothing that could lead to loss of detail. The processed point cloud data exhibits better uniformity and consistency.

[0076] A voxel grid downsampling method is used to reduce point cloud density while preserving shape features. The voxel grid size is determined based on the point cloud density and feature size, typically set to twice the average point spacing. Within each voxel grid, the point closest to the voxel center is retained as a representative point, and other points are removed. The downsampled point cloud is reduced to one-quarter of its original size, while retaining the main shape features. For regions with high curvature, an adaptive downsampling strategy is employed, appropriately reducing the voxel grid size to retain more detail. The downsampling process is iterative until the point cloud density reaches the preset requirement.

[0077] Principal component analysis (PCA) is used to calculate the normal vector and curvature features of the local surface of the point cloud. For each point in the point cloud, all points within a 2 mm radius are selected as the local neighborhood. The covariance matrix of the neighborhood points is calculated, and eigenvalue decomposition is performed on the covariance matrix. The eigenvector corresponding to the smallest eigenvalue is the normal vector direction of that point. The curvature feature is obtained through eigenvalue calculation, and the ratio of the smallest eigenvalue to the sum of the eigenvalues ​​is used as the curvature estimate. Weighted least squares is used in the calculation of normal vector and curvature, with points closer to the center point having a higher weight. The calculated normal vector and curvature information are used for subsequent feature extraction.

[0078] Macroscopic undulation features and microscopic roughness features are extracted from point cloud data to generate multi-scale fused surface topological features. Macroscopic undulation features are obtained by calculating the height distribution parameters of the point cloud surface, including the surface arithmetic mean deviation, root mean square deviation, and surface skewness. Microscopic roughness features are obtained by wavelet decomposition of the point cloud surface, extracting wavelet coefficients at different scales as feature descriptions. Multi-scale feature fusion employs a feature concatenation method, combining macroscopic and microscopic features into a unified feature vector. The dimension of the feature vector is determined according to actual needs, typically containing 30 to 50 feature parameters. The final generated surface topological features can comprehensively describe the morphological characteristics of the material surface.

[0079] The process of comparing surface topological features with a benchmark morphology library to generate morphology verification results specifically includes the following steps: Calculating the Hausdorff distance between the surface topological features and the morphology feature vectors of each standard sample in the benchmark morphology library. The Hausdorff distance is calculated using the bidirectional maximum-minimum distance method. First, the distance from the feature of the test sample to the nearest point in each set of standard sample features is calculated, and the maximum value among these distances is taken as the one-way Hausdorff distance. Then, the distance from the standard sample feature to the nearest point in the set of feature of the test sample is calculated, and the maximum value is also taken. Finally, the Hausdorff distance is the larger of the two one-way distances. Euclidean distance is used as the distance metric between points during the distance calculation process.

[0080] A hierarchical matching strategy is adopted. First, coarse matching is performed to filter out the candidate sample set, and then fine matching is performed to determine the optimal matching sample. In the coarse matching stage, the dimensionality-reduced feature vectors are used, and principal component analysis is employed to reduce the feature dimension to 10. The coarse distance between the test sample and all standard samples is calculated, and the 20 samples with the smallest distances are selected as the candidate sample set. In the fine matching stage, the complete feature vectors are used to calculate the precise Hausdorff distance between the test sample and each sample in the candidate sample set. The weights of each feature dimension are considered during the fine matching process, with higher weight coefficients set for important features.

[0081] Based on the distance similarity score and matching consistency index, combined with a preset confidence threshold, a morphology verification result is generated. The distance similarity score is obtained by mapping Hausdorff distance to a range of 0 to 1 using an exponential decay function. The matching consistency index is calculated by comparing the degree of difference between the test sample and the standard sample across various feature dimensions. The confidence threshold is set to 0.8, a value determined through optimization using extensive experimental data. When the matching score is greater than or equal to the preset threshold, the corresponding material category is output as the morphology verification result. The result includes information such as the matching score, matching level, and reliability index.

[0082] When the matching score falls below a preset threshold, a manual intervention process is triggered. This intervention involves a professional operator confirming the matching results, using visualization tools to view the 3D display of the point cloud data, and comparing the morphological features of the test sample with those of the standard sample. The operator can adjust the matching parameters or select a different matching algorithm to re-match. The manually confirmed results are recorded in the system for subsequent model optimization and database updates. The entire process ensures the accuracy and reliability of the morphological verification results.

[0083] In S4, the preliminary classification results, composition verification results, and morphology verification results are comprehensively evaluated to generate a final classification decision. Based on the final classification decision, the sorting execution mechanism is controlled to complete the precise sorting of scrap metal, specifically including:

[0084] The process of comprehensively evaluating the preliminary classification results, composition verification results, and morphology verification results to generate the final classification decision includes the following steps. First, based on the confidence index corresponding to each result, different confidence weights are assigned to the preliminary classification results, composition verification results, and morphology verification results. The confidence weight for the preliminary classification results is set to 0.2, the confidence weight for the morphology verification results is set to 0.3, and the confidence weight for the composition verification results is set to 0.5. The weight allocation is based on the reliability of each detection method. Composition verification results directly reflect the elemental composition of the material and have the highest reliability; morphology verification results reflect the surface characteristics of the material and are less affected by environmental factors; the preliminary classification results are based on image recognition and are easily affected by surface conditions, therefore they are assigned the lowest weight. The confidence index for each result comes from the calculated values ​​in its respective processing, including quantitative indicators such as the information entropy of the probability distribution and the similarity score of feature matching.

[0085] A weighted fusion algorithm is used to comprehensively evaluate the three results. The weighted fusion process first normalizes the confidence indices of each result, placing them within the range of 0 to 1. The normalization method uses min-max scaling to map the original confidence values ​​to a standard interval. Then, the normalized confidence value of each result is multiplied by its corresponding weight coefficient to obtain a weighted confidence score. The three weighted confidence scores are added together to obtain the final comprehensive score. The comprehensive score ranges from 0 to 1, with higher values ​​indicating higher confidence in the classification result. In the entire weighted fusion process, the component verification result has the highest weight and the greatest impact on the final decision; the morphology verification result has the second highest weight; and the preliminary classification result has the lowest weight, mainly serving as an auxiliary verification.

[0086] When the weighted overall score is greater than or equal to a preset threshold, the category with the highest score is output as the final classification decision. The preset threshold is set to 0.75, a value obtained through statistical analysis and optimization of a large amount of experimental data. The score calculation is based on the category probability distribution corresponding to each result. The category probability of each result is multiplied by its weighted confidence score, and then the weighted probability values ​​of the same category are summed to obtain the final score for each category. The category with the highest final score is selected as the output result. The overall score of the decision and the scores of each category are recorded simultaneously for subsequent quality traceability and process analysis. The output results use a standardized data format and include key information such as material category, decision score, and timestamp.

[0087] When the overall score falls below a preset threshold, a manual review process is initiated. This review is performed by professional technicians who can view detailed information about each test result, including raw data, processing procedures, intermediate results, and confidence levels. The system provides a visual interface displaying multimodal information such as image recognition results, elemental spectra, and 3D point cloud data. Technicians make a comprehensive judgment based on their professional knowledge and experience, manually selecting or inputting the final classification result. The results of the manual review are recorded in the system and simultaneously fed back to the machine learning model for subsequent model optimization and parameter adjustment. The entire process ensures the accuracy and reliability of the final classification decision, guaranteeing the precise sorting of scrap metal.

[0088] Please see Figure 2 As shown, a scrap metal sorting and identification system based on image recognition includes:

[0089] A multi-level visual feature extraction module is used to obtain the surface visual information of the scrap metal to be sorted. After preprocessing the surface visual information, its multi-level depth features are extracted, including: surface color and texture features, middle contour and shape features and deep semantic features.

[0090] The visual preliminary classification and confidence assessment module uses a deep neural network to identify multi-level deep features to generate preliminary classification results of scrap metal and corresponding confidence assessment values; when the confidence assessment value is lower than a preset threshold, the elemental spectral information and three-dimensional structural information of the corresponding scrap metal are collected.

[0091] A multimodal data acquisition and processing module is used to perform spectral analysis on elemental spectral information to obtain elemental composition data, match the elemental composition data with a standard component database, and generate component verification results.

[0092] The three-dimensional structural information is processed into point cloud to obtain surface topological features. The surface topological features are then compared with the benchmark topological library to generate topological verification results.

[0093] The multi-source verification and decision fusion module comprehensively evaluates the preliminary classification results, composition verification results, and morphology verification results to generate a final classification decision. Based on the final classification decision, the module controls the sorting execution mechanism to complete the precise sorting of waste metals.

[0094] The working principle of this invention is as follows: A high-resolution industrial camera acquires surface visual information of scrap metal to be sorted under standardized lighting conditions. After preprocessing including denoising, color correction, background segmentation, and image enhancement, multi-scale convolutional kernels are used in parallel convolution operations to extract surface color and texture features. Spatial pyramid pooling combined with an edge enhancement algorithm is used to obtain mid-level contour shape features, and a deep residual network is used to mine deep semantic features. Subsequently, the multi-level deep features are fused and input into a deep neural network to generate preliminary classification results and confidence assessment values. When the confidence level is below a threshold, laser-induced breakdown spectroscopy and structured light 3D scanning technologies are triggered to collect elemental spectral information and 3D structural information, respectively. Elemental composition data is obtained by performing wavelet transform and feature peak analysis on the spectral information, and weighted similarity matching is performed with a standard component database to generate component verification results. Simultaneously, point cloud data is denoised, downsampled, and feature extracted to obtain surface topological features, and Hausdorff distance calculation is used to compare with a benchmark morphology database to generate morphology verification results. Finally, a weighted fusion decision is made based on the confidence index of each result. When the comprehensive score reaches the threshold, the final classification result is output to control the sorting execution mechanism to complete accurate sorting; otherwise, the manual review process is initiated.

[0095] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. An image recognition-based scrap metal classification and identification method, characterized in that, The method comprises the following steps: S1: obtaining surface visual information of the waste metal to be sorted, pre-processing the surface visual information, and extracting multi-level depth features of the surface visual information, including surface color texture features, middle layer contour shape features and deep layer semantic features; The extraction process of the multi-level depth features is as follows: parallel convolution operation is performed on the pre-processed surface visual information by using a multi-scale convolution kernel group to extract color distribution features and texture detail features at different scales, and the multi-scale features are fused to generate the surface color texture features; based on the surface color texture features, the macroscopic contour structure and local geometric shape of the waste metal are captured through a spatial pyramid pooling layer, and the contour boundary information is strengthened by combining an edge enhancement algorithm to generate the middle layer contour shape features; the middle layer contour shape features are input into a deep residual network to mine deep semantic correlations between the features through cascaded nonlinear transformations, and deep semantic features are output; S2: using a deep neural network to recognize the multi-level depth features to generate a preliminary classification result of the waste metal and a corresponding confidence evaluation value; when the confidence evaluation value is lower than a preset threshold, collecting element spectrum information and three-dimensional structure information of the corresponding waste metal; the confidence evaluation value is obtained by: calculating the information entropy of the probability distribution, which is used to represent the uncertainty degree of the classification decision; meanwhile, the clustering tightness of the feature vector corresponding to the preliminary classification result in the feature space is calculated; the uncertainty degree and the clustering tightness are weighted and summed, and the summed result is normalized to obtain the confidence evaluation value; S3: performing spectrum analysis on the element spectrum information to obtain element composition data, matching the element composition data with a standard component database to generate a component verification result, specifically including: calculating the cosine similarity of the element composition data and each standard grade element content vector in the standard component database; giving higher weights to main alloy elements according to the importance of elements; selecting the standard grade with the highest weighted similarity as the candidate matching result; when the highest weighted similarity is greater than or equal to a preset confidence threshold, directly outputting the standard grade as the component verification result; when the highest weighted similarity is less than the preset confidence threshold, outputting a matching failure flag and starting an artificial review process; performing point cloud processing on the three-dimensional structure information to obtain surface topological features, and comparing the surface topological features with a reference morphology library to generate a morphology verification result; S4: comprehensively judging the preliminary classification result, the component verification result and the morphology verification result to generate a final classification decision, and controlling a sorting execution mechanism to complete accurate sorting of the waste metal according to the final classification decision.

2. The image recognition-based scrap metal classification and recognition method according to claim 1, characterized in that, The specific implementation process of the preliminary classification result includes: cross-level feature fusion is performed on the surface color texture features, the middle layer contour shape features and the deep layer semantic features to form a comprehensive feature representation; the comprehensive feature representation is input into a multilayer perceptron of the deep neural network for nonlinear transformation; the probability distribution of the waste metal belonging to each preset category is obtained through forward propagation calculation; and the category corresponding to the maximum value in the probability distribution is selected as the preliminary classification result.

3. The image recognition-based scrap metal classification and recognition method according to claim 1, characterized in that, The spectral analysis on the element spectrum information is performed to obtain element composition data, and specifically includes: The collected element spectrum information is preprocessed to eliminate environmental noise and instrument background interference; the preprocessed spectrum signal is analyzed by using continuous wavelet transform to extract spectrum feature peaks at different scales; the chemical element types corresponding to each feature peak are identified by a feature peak matching algorithm; the relative content of each element is calculated based on the integral of the feature peak intensity, and finally the element composition data containing the element types and contents is generated.

4. The image recognition-based scrap metal classification and recognition method according to claim 1, characterized in that, The point cloud processing on the three-dimensional structure information is performed to obtain surface topological features, and specifically includes: The collected three-dimensional point cloud data is denoised and smoothed to eliminate measurement errors and abnormal points; the point cloud density is reduced under the premise of maintaining shape features by using a voxel grid downsampling method; the normal vector and curvature features of the local surface of the point cloud are calculated by principal component analysis; the macro relief features and micro roughness features of the point cloud data are extracted, and finally the multi-scale fused surface topological features are generated.

5. The image recognition-based scrap metal classification and recognition method according to claim 1, characterized in that, The surface topological features are compared with the reference morphology library to generate a morphology verification result, and specifically includes: The Hausdorff distance between the surface topological features and the standard sample morphology feature vectors in the reference morphology library is calculated; a hierarchical matching strategy is adopted to first perform coarse matching to screen out a candidate sample set, and then perform fine matching to determine the optimal matching sample; the morphology verification result is generated according to the distance similarity score and the matching consistency index, combined with a pre-set confidence threshold; when the matching score is greater than or equal to the pre-set threshold, the corresponding material category is output, otherwise the manual intervention process is triggered.

6. The image recognition-based scrap metal classification and recognition method according to claim 1, characterized in that, The preliminary classification result, the composition verification result and the morphology verification result are comprehensively evaluated to generate a final classification decision, and specifically includes: According to the confidence indexes corresponding to each result, different credibility weights are given to the preliminary classification result, the composition verification result and the morphology verification result; the three results are comprehensively evaluated by using weighted fusion, wherein the weight of the composition verification result is the highest, the weight of the morphology verification result is the second, and the weight of the preliminary classification result is the lowest; when the weighted comprehensive score is greater than or equal to the pre-set threshold, the category with the highest score is output as the final classification decision; when the comprehensive score is less than the pre-set threshold, manual review is started.

7. An image recognition-based scrap metal classification and recognition system, characterized by, A waste metal classification and identification method based on image recognition is used to perform the method of any one of claims 1-6, comprising: A multi-level visual feature extraction module is used to obtain the surface visual information of the waste metal to be sorted, and after preprocessing the surface visual information, multi-level deep features are extracted, including surface color texture features, middle layer contour shape features and deep layer semantic features; A visual preliminary classification and confidence evaluation module is used to identify the multi-level deep features by using a deep neural network to generate a preliminary classification result of the waste metal and a corresponding confidence evaluation value; when the confidence evaluation value is lower than a pre-set threshold, element spectrum information and three-dimensional structure information of the corresponding waste metal are collected; A multi-modal data acquisition and processing module is configured to perform spectral analysis on element spectral information to obtain element composition data, match the element composition data with a standard component database, and generate a component verification result; A point cloud processing is performed on the three-dimensional structure information to obtain surface topological features, which are compared with a reference topography library to generate a topography verification result; A multi-source verification and decision fusion module is configured to comprehensively judge the preliminary classification result, the component verification result, and the topography verification result to generate a final classification decision, and control a sorting execution mechanism to complete precise sorting of the waste metal according to the final classification decision.

Images