Intelligent identification and sorting method of secondary metals based on LIBS spectrum and multi-modal sensing

Abstract

This invention discloses an intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing, relating to the fields of visual inspection and recycled metal sorting technology. It generates multimodal registration data by acquiring 3D point clouds and RGB images of materials on a conveyor belt, searches for extreme values ​​of surface flatness and color variance, and obtains the optimal excitation target point. A dual-pulse laser is controlled for cleaning and excitation to obtain a pure spectrum. Chemical characteristic spectral lines and physical morphological features are extracted and input into a deep learning model containing a cross-modal attention mechanism, dynamically assigning feature weights and outputting the alloy grade. Based on the grade matching standard density, the real-time mass of the material is calculated using 3D volume, and the dynamic aerodynamic impulse is calculated according to the momentum theorem to control the air valve array for precise blowing. This method solves the problems of ineffective spectra, easy misjudgment of single components, and missorting under fixed air pressure caused by traditional LIBS blind testing, achieving high-speed and high-purity recycling of waste metals.

Description

Technical Field

[0001] This invention belongs to the field of visual inspection and recycled metal sorting technology, and specifically relates to a method for intelligent identification and sorting of recycled metals based on LIBS spectroscopy and multimodal sensing. Background Technology

[0002] With the rapid iteration of global manufacturing and the in-depth implementation of the "dual carbon" strategy, the resource recycling and utilization of solid waste, especially scrap metal, is of great strategic significance for reducing energy consumption in primary mineral mining and reducing greenhouse gas emissions. In the recycled metal industry chain, high-speed, high-purity, and refined sorting of piles of mixed scrap metal (such as different grades of aluminum alloys, copper alloys, and stainless steel) is a core link in improving the quality and economic added value of recycled metal smelting.

[0003] Current industrial sorting applications are mostly in high-speed conveyor belt environments, and existing technologies mainly rely on manual sorting, X-ray transmission / fluorescence (XRT / XRF) sorting, and traditional laser-induced breakdown spectroscopy (LIBS) technology. Manual sorting is extremely inefficient and cannot distinguish alloys with similar compositions with the naked eye; XRF technology has a long detection cycle and extremely poor sensitivity to light elements (such as magnesium, aluminum, and silicon), making it unsuitable for high-speed production lines. Traditional LIBS technology, introduced in recent years, uses high-energy lasers to break down metal surfaces to generate plasma, analyzing characteristic spectra to identify chemical composition; it is fast and can detect light elements. However, traditional LIBS uses a fixed center point "blind" approach in practical applications and relies solely on spectral information for material determination; at the sorting end, a high-pressure valve array with a fixed pressure is typically used for purging and separation.

[0004] This reveals the following problems with existing technologies: 1. Scrap metal surfaces are often heavily corroded, oily, or coated, making it easy for traditional LIBS's "blind" approach to hit non-metallic areas, resulting in a poor spectral signal-to-noise ratio; 2. Single spectral information cannot capture the physical characteristics of materials, and misjudgment of alloy grades is likely when the spectrum is interfered with by impurities; 3. The fixed-pressure sorting logic does not consider the differences in volume and mass of different metal blocks, causing large, heavy materials to be unable to be blown away and miss sorting, while small, light materials are blown away and randomly enter the impurity bin, thus limiting the sorting purity. Summary of the Invention

[0005] (a) Technical problems to be solved To address the problems in related technologies, this invention provides a method for intelligent identification and sorting of recycled metals based on LIBS spectroscopy and multimodal sensing, thereby overcoming the aforementioned technical problems in existing related technologies.

[0006] (II) Technical Solution To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: Firstly, a method for intelligent identification and sorting of recycled metals based on LIBS spectroscopy and multimodal sensing is provided, specifically as follows: S1. Extract the real-time three-dimensional point cloud signal and two-dimensional color image signal of the waste metal material moving on the conveyor belt, and generate a multimodal registration data matrix through spatial coordinate mapping. S2. Using the surface smoothness features and color variance features of the multimodal registration data matrix as optimization parameters of the objective function, the optimal three-dimensional coordinate matrix of the excitation target point for laser-induced breakdown spectrum is obtained by using the local window sliding search algorithm. S3. Based on the three-dimensional coordinate matrix of the optimal excitation target point, control the dual-pulse laser to clean and excite the waste metal material, collect the plasma emission spectrum, and convert it into a discrete spectral sequence signal; S4. Baseline calibration is performed on the discrete spectral sequence signal using the asymmetric least squares method to extract the chemical feature spectral matrix; the physical morphological feature matrix of the multimodal registration data matrix is ​​extracted using a convolutional neural network. S5. Input the chemical feature spectral line matrix and physical morphology feature matrix into the deep learning recognition model, and use the inner product scaling function of the cross-modal attention mechanism to calculate the dynamic weight allocation matrix of each modal feature; output the probability distribution sequence based on the dynamic weight allocation matrix to determine the alloy grade label; S6. Calculate the global volume features and spatial contour reference features of the multimodal registration data matrix; calculate the real-time mass features based on the alloy grade label matching standard density and the global volume features; calculate the dynamic aerodynamic impulse matrix based on the real-time mass features and spatial contour reference features, and control the valve array to perform sorting. Preferably, step S1 includes the following steps: S11. Establish a data acquisition synchronization time window model for multimodal sensors; when a photoelectric switch trigger signal is detected, start the 3D laser profilometer and high frame rate RGB camera; extract historical background point clouds and scrap metal material point clouds, use a pass-through filter to remove the conveyor belt background, and stitch them together to generate the original three-dimensional point cloud sequence. S12. Extract the spatial bounding box features of the original three-dimensional point cloud sequence; using the pinhole camera projection equation based on the camera intrinsic parameter matrix and extrinsic parameter rotation and translation matrix, map the RGB pixels of the two-dimensional color image signal to the three-dimensional spatial coordinate system of the original three-dimensional point cloud sequence to obtain a multimodal registration data matrix containing six-dimensional attributes (X,Y,Z,R,G,B). Preferably, step S2 includes the following steps: S21. Define the search space of the local window sliding search algorithm; define the side length of the three-dimensional optimization window as 3 times the diameter feature of the laser spot; slide the three-dimensional optimization window in the top projection area of ​​the multimodal registration data matrix at a set step size; S22. For any point cloud set within a sliding window, use principal component analysis to fit a local micro-tangent plane, calculate the angle between the normal vector of the local micro-tangent plane and the absolute Z-axis, and generate surface smoothness features; extract the RGB pixel values ​​of all points within the sliding window, calculate the pixel variance, and generate color variance features. S23. Minimize the target penalty value as the global optimization objective. The target penalty value is equal to the surface smoothness feature multiplied by the first fixed weight coefficient plus the color variance feature multiplied by the second fixed weight coefficient. Traverse all sliding windows and output the three-dimensional coordinates of the center point of the window with the minimum target penalty value as the optimal excitation target point three-dimensional coordinate matrix. Preferably, step S3 includes the following steps: S31. Based on the three-dimensional coordinate matrix of the optimal excitation target point, combined with the real-time movement speed characteristics of the conveyor belt and the system processing delay time characteristics, the dynamic prediction coordinate matrix of the laser firing moment is calculated using the kinematic compensation equation. S32. Send the dynamic prediction coordinate matrix to the two-dimensional high-speed scanning galvanometer controller; trigger the dual-pulse laser to emit the first nanosecond-level cleaning laser pulse, and use plasma shock waves to peel off the surface oxide layer and paint coating at the target coordinates. S33. After a preset microsecond time interval, the dual-pulse laser is triggered to emit a second nanosecond-level probe laser pulse, which breaks through the exposed metal substrate to generate plasma; the plasma emission spectrum is collected by an echelle grating spectrometer and an enhanced charge-coupled device, and a discrete spectral sequence signal containing wavelength and absolute light intensity is output. Preferably, step S4 includes the following steps: S41. Extract the discrete spectral sequence signal, calculate the continuous baseline background noise features using an asymmetric least squares smoothing algorithm; subtract the baseline background noise features from the discrete spectral sequence signal to obtain a clean spectral sequence. S42. Using the Lorentz line fitting function, perform peak search on the pure spectral sequence, extract the local maximum peak features and full width at half maximum features at the wavelengths of preset feature elements, normalize the peak intensities of different elements, and construct a chemical feature spectral matrix. S43. Extract the multimodal registration data matrix from S12 and convert it into a four-channel RGB-D image containing depth information; input the four-channel RGB-D image into a pre-trained two-dimensional residual convolutional neural network, and after multi-layer convolution and pooling dimensionality reduction, extract global texture and morphological feature vectors to generate a physical morphological feature matrix. S44. After fully connecting the chemical feature spectral matrix of S42 and the physical morphology feature matrix of S43 to reduce the dimension to the same dimension, perform orthogonal concatenation of row vectors to generate a cross-modal joint feature matrix. Preferably, step S41 includes the following steps: S411. Set the smoothing parameters and initial asymmetric penalty weights for the asymmetric least squares smoothing algorithm. S412. Construct a penalized least squares objective function based on a discrete difference matrix; set the original spectral sequence and the fitting baseline; during the iteration process, when the current fitting baseline feature is greater than the original light intensity feature of the discrete spectral sequence signal, assign the real-time asymmetric penalty weight feature as 1 minus the initial asymmetric penalty weight feature; when the current fitting baseline feature is less than the original light intensity feature, the real-time asymmetric penalty weight feature is the initial asymmetric penalty weight feature. S413. When the relative rate of change of the baseline feature vectors in two adjacent iterations is less than the preset convergence tolerance, stop the iteration and output the final continuous baseline background noise features. Preferably, step S5 includes the following steps: S51. Input the cross-modal joint feature matrix into the deep learning recognition model, and perform linear mapping on the chemical feature spectral matrix and the physical morphology feature matrix respectively to generate a query matrix, a bond matrix and a value matrix. S52. Calculate the original association scores of each modality feature using the inner product scaling function of the cross-modal attention mechanism, and transform the original association scores into a dynamic weight allocation matrix using the normalized exponential function. S53. Multiply the weight coefficients in the dynamic weight allocation matrix element by element with the corresponding physical value matrix and sum them to output the weighted context feature vector. S54. Input the weighted context feature vector into the fully connected classification layer of the deep learning recognition model, and use the weight matrix and bias of the fully connected classification layer to perform a linear mapping to a logical output tensor with a dimension equal to the total number of alloy categories. S55. Substitute the logic output tensor into the Softmax function to calculate the conditional probability distribution sequence of the material belonging to each label category; Preferably, step S52 includes the following steps: S521. Extract the chemical query matrix and physical bond matrix generated in S51, perform a dot product operation on the transpose of the chemical query matrix and the physical bond matrix, and calculate the cross-covariance feature matrix between the chemical signal and the physical visual signal. S522. Divide the cross-covariance feature matrix by the square root feature of the key matrix dimension and perform numerical scaling to obtain the original association score matrix; S523. Substitute the original correlation score matrix into the Softmax normalization equation; calculate the ratio of the natural index value of the current feature dimension score to the sum of the natural index values ​​of all feature dimensions, and output a dynamic weight allocation matrix with a sum of 1. Preferably, step S6 includes the following steps: S61. Perform voxelization downsampling on the multimodal registration data matrix and set the side length reference feature of a single voxel; count the total number of effective voxels containing the material point cloud, multiply the total number of effective voxels by the volume of a single voxel, and calculate the global volume feature; extract the geometric center coordinates of the spatial bounding box feature to obtain the spatial contour reference feature. S62. Establish an alloy standard density database; based on the alloy grade label output in S55, retrieve the corresponding standard density feature from the database; multiply the standard density feature by the global volume feature in S61 to calculate the real-time mass feature of the material. S63. Based on Newton's second law and the momentum theorem of aerodynamics, calculate the lateral target velocity characteristic required to blow the material with the real-time mass characteristic away from the parabolic trajectory of the conveyor belt; multiply the real-time mass characteristic by the lateral target velocity characteristic to calculate the dynamic aerodynamic impulse matrix; S64. Based on the dynamic aerodynamic impulse matrix, combined with the nozzle cross-sectional area and rated flow velocity of the high-pressure valve, the target opening pressure threshold and target opening duration characteristics of the high-pressure valve are calculated by back-calculating using the Bernoulli equation of fluid mechanics. S65. Based on the spatial contour reference features extracted in S61 and the system conveyor belt speed, calculate the time delay of the material arriving in front of the air valve array; when the time delay arrives, send the target opening air pressure threshold and target opening duration features to the corresponding electromagnetic air valve to accurately blow the target material into the corresponding collection bin. Secondly, a smart identification and sorting system for recycled metals based on LIBS spectroscopy and multimodal sensing is also provided to implement the aforementioned smart identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing. The system includes a multimodal feature sensing module, a laser dynamic targeting and dual-pulse excitation module, a cross-modal feature decoupling and splicing fusion module, a deep attention classification module, and a dynamic impulse sorting module; wherein: The aforementioned multimodal feature perception module is used to eliminate background point clouds and image distortion, and generate a multimodal registration data matrix; The aforementioned laser dynamic targeting and dual-pulse excitation module is used to obtain the optimal excitation target by using surface flatness and color variance as optimization targets; and to perform cleaning and excitation based on motion compensation control of the dual-pulse laser to obtain a one-dimensional discrete spectral sequence signal. The aforementioned cross-modal feature decoupling and splicing fusion module is used to extract the chemical feature spectral line matrix of the spectrum and to extract the physical morphological feature matrix of the vision using a convolutional network; after the two are fully connected and dimensionality reduced, they are orthogonally spliced ​​to generate a cross-modal joint feature matrix. The aforementioned deep attention classification module is used to input the cross-modal joint feature matrix into the deep model, calculate the dynamic weight allocation matrix using the inner product scaling function, and determine the alloy grade based on the weighted features. The aforementioned dynamic impulse sorting module is used to calculate the global volume characteristics and spatial contour reference characteristics of the material; based on the alloy grade label matching standard density, it calculates the real-time mass in combination with the global volume, and outputs a dynamic aerodynamic impulse matrix based on the momentum theorem to perform precise air valve sorting.

[0007] (III) Beneficial Effects The present invention has the following beneficial effects: This invention integrates 3D point cloud and RGB vision technology, and establishes an objective extreme value optimization model for the angle between normal vectors and color variance. This enables the laser to adaptively avoid rust and coatings, and accurately lock onto the flattest and purest golden target point on the material surface. Combined with the dual-pulse laser's shock wave cleaning followed by breakdown detection mechanism, impurities are eliminated from the physical source, resulting in a qualitative leap in the effective acquisition rate of the spectrum of complex scrap steel.

[0008] This invention constructs a high-dimensional orthogonal feature space that combines intrinsic chemical composition with extrinsic physical morphology. It utilizes convolutional neural networks to extract prior information on the physical geometry and texture of materials and introduces a cross-modal attention mechanism. Through an inner product scaling mathematical model, the system can dynamically allocate decision weights for physical and chemical features based on fluctuations in the spectral signal-to-noise ratio. Even when the spectral signal is weak, cross-validation can be performed through physical morphology, thereby improving the accuracy of alloy grade identification.

[0009] This invention abandons the crude logic of traditional blind blowing with timed and pressured air valves, and constructs a mathematical execution chain that includes density matching identification, three-dimensional volume extraction, real-time mass calculation, and aerodynamic impulse derivation. Based on the momentum theorem, the system tailors the optimal high-pressure air valve opening time and air pressure threshold for each piece of metal on the conveyor belt, solving the problem of missorting caused by mixed materials of different sizes, such as materials that cannot be blown or blown away, and ensuring the high purity of the final recycled product.

[0010] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to the present invention. Figure 2 This is a schematic diagram of the modules of the intelligent identification and sorting system for recycled metals based on LIBS spectroscopy and multimodal sensing according to the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the invention, and not all embodiments. Based on the embodiments of the invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the invention.

[0014] To address the technical problems raised in the background section, please refer to [link / reference]. Figure 1 This invention provides a method for intelligent identification and sorting of recycled metals based on LIBS spectroscopy and multimodal sensing, including: S1. Extract the real-time three-dimensional point cloud signal and two-dimensional color image signal of the waste metal material moving on the conveyor belt, and generate a multimodal registration data matrix through spatial coordinate mapping. S2. Using the surface smoothness features and color variance features of the multimodal registration data matrix as optimization parameters of the objective function, the optimal three-dimensional coordinate matrix of the excitation target point for laser-induced breakdown spectrum is obtained by using the local window sliding search algorithm. S3. Based on the three-dimensional coordinate matrix of the optimal excitation target point, control the dual-pulse laser to clean and excite the waste metal material, collect the plasma emission spectrum, and convert it into a discrete spectral sequence signal; S4. Baseline calibration is performed on the discrete spectral sequence signal using the asymmetric least squares method to extract the chemical feature spectral matrix; the physical morphological feature matrix of the multimodal registration data matrix is ​​extracted using a convolutional neural network. S5. Input the chemical feature spectral line matrix and physical morphology feature matrix into the deep learning recognition model, and use the inner product scaling function of the cross-modal attention mechanism to calculate the dynamic weight allocation matrix of each modal feature; output the probability distribution sequence based on the dynamic weight allocation matrix to determine the alloy grade label; S6. Calculate the global volume features and spatial contour reference features of the multimodal registration data matrix; calculate the real-time mass features based on the alloy grade label matching standard density and the global volume features; calculate the dynamic aerodynamic impulse matrix based on the real-time mass features and spatial contour reference features, and control the valve array to perform sorting. The above embodiments, by pre-calculating the three-dimensional surface flatness and color variance of the material, break through the technical bottleneck of traditional LIBS technology which uses a fixed center point for blind drilling. The laser excitation point is adaptively locked to the flattest and rust-free exposed area of ​​the material, eliminating the interference of impurities on the spectral signal-to-noise ratio from the physical execution source. Through a dual-pulse laser's pre-cleaning and post-detection mechanism, micron-level oxide films are further peeled off. The mathematical calculation rules for cross-modal attention weights are clearly defined. Through inner product scaling and Softmax normalization equations, the model can dynamically adjust the decision weights of chemical composition and physical appearance according to the degree of spectral fluctuations, solving the problem of single-modal misjudgment. By calculating real-time mass and customizing aerodynamic impulse for each piece of metal according to the momentum theorem, high-precision sorting of complex-shaped recycled metals is achieved. The above embodiment S1 includes the following steps: S11. Establish a data acquisition synchronization time window model for multimodal sensors; when a photoelectric switch trigger signal is detected, start the 3D laser profilometer and high frame rate RGB camera; extract historical background point clouds and scrap metal material point clouds, use a pass-through filter to remove the conveyor belt background, and stitch them together to generate the original three-dimensional point cloud sequence. In specific implementation, the above embodiment S11 specifically involves: configuring a high-frequency laser triangulation profiler and a global shutter RGB area array camera at the starting end of the sorting production line; and setting the rated operating speed characteristics of the conveyor belt. V c When the photoelectric switch detects that the material front has cut off the light beam, the microcontroller triggers a hardware interrupt, marking the current moment as the trigger origin. t 0; The system extracts the threshold value in the Z-axis (height) direction. Z belt Using a direct-pass filtering algorithm to remove all Z< Z belt +1mm background point, retaining only the three-dimensional coordinate set of the material itself. P raw ={ p 1, p 2,..., pi ,..., p N},in, p i =( x i , y i , z i ) indicates the first i The three-dimensional spatial coordinates of each discrete scanning point, where N represents the total number of valid discrete three-dimensional coordinate points belonging to the material body that are retained after the pass-through filtering process. Furthermore, to eliminate multimodal data registration errors caused by high-speed motion, the system uses a field-programmable gate array (FPGA) as a unified hardware trigger source. When the photoelectric switch is triggered, the FPGA outputs a nanosecond-level synchronization pulse signal (Sync-Pulse), which synchronously triggers the line scan cycle of the 3D laser profilometer and the global shutter exposure of the RGB camera, ensuring the synchronous acquisition of three-dimensional spatial coordinates and two-dimensional color pixels in the time dimension. S12. Extract the spatial bounding box features of the original three-dimensional point cloud sequence; using the pinhole camera projection equation based on the camera intrinsic parameter matrix and extrinsic parameter rotation and translation matrix, map the RGB pixels of the two-dimensional color image signal to the three-dimensional spatial coordinate system of the original three-dimensional point cloud sequence to obtain a multimodal registration data matrix containing six-dimensional attributes (X,Y,Z,R,G,B). In specific implementation, the above embodiment S12 specifically involves: extracting the optical intrinsic parameter matrix obtained by the RGB camera during the offline calibration stage. K cam The extrinsic rotation matrix from the camera coordinate system to the 3D profilometer coordinate system R ext Translation vector T ext For any point in the point cloud set p i =( x i , y i , z i Using the homogeneous coordinate projection equation: ,in, s For depth scaling; [ u i , v i ,1] T Represents a homogeneous column vector of coordinates in the pixel coordinate system; K cam It is a 3×3 optical intrinsic parameter matrix; Rext |T ext [ is a 3×4 extrinsic augmentation matrix, consisting of a 3×3 rotation matrix.] R ext Translation vector of 3×1 T ext It is pieced together; x i , x i , y i ,1] T Given a homogeneous column vector of coordinates of a point in three-dimensional space; calculate the corresponding pixel coordinates of that point on the two-dimensional image. u i , v i Extract the color pixel value at that pixel coordinate. r i , g i , b i This data is then concatenated with three-dimensional coordinates to generate a multimodal registration data matrix. M fused ={( x i , y i , z i , r i , g i , b i This step mathematically aligns the physical space with the optical image. The above embodiment S1 achieves high-dimensional registration of heterogeneous sensor data by establishing a strict internal and external parameter projection matrix, providing an accurate physical benchmark for subsequent density-based mass calculations. The above embodiment S2 includes the following steps: S21. Define the search space of the local window sliding search algorithm; define the side length of the three-dimensional optimization window as 3 times the diameter feature of the laser spot; slide the three-dimensional optimization window in the top projection area of ​​the multimodal registration data matrix at a set step size; In specific implementation, the above embodiment S21 specifically involves: the known characteristics of the spot diameter after LIBS laser focusing are as follows: D laser (e.g., 0.5mm), set the side length of the sliding window. W s =3× D laserOn the two-dimensional projection plane (X−Y plane) of the material, with step size S step = D laser Generate a sliding mesh sequence; Extract the three-dimensional coordinates of all points in the multimodal registration data matrix, set its Z-axis coordinate to zero, and orthogonally project it onto the XY absolute horizontal plane to obtain the two-dimensional top surface projection contour region of the material; within the two-dimensional top surface projection contour region, slide the three-dimensional optimization window according to a set step size; S22. For any point cloud set within a sliding window, use principal component analysis to fit a local micro-tangent plane, calculate the angle between the normal vector of the local micro-tangent plane and the absolute Z-axis, and generate surface smoothness features; extract the RGB pixel values ​​of all points within the sliding window, calculate the pixel variance, and generate color variance features. In specific implementation, the above embodiment S22 specifically refers to: for the first k For a subset of point clouds within a sliding window, calculate the covariance matrix of its 3D coordinates, and extract the eigenvector corresponding to the smallest eigenvalue as the normal vector of the local micro-tangent plane. n k 1 =( n x , n y , n z ); Calculate the Z-axis of this normal vector relative to the system's absolute perpendicular axis (laser incident direction). 1 The cosine of the angle between (0,0,1) Value; defines surface smoothness characteristics F fl ( k )=1−cos( i k The closer this value is to 0, the more level the surface is, and the less likely laser defocusing will occur. Simultaneously, extract the grayscale value sequence of all points within the window. I j =0.299 r j +0.587 g j +0.114 b j Calculate local color variance features A larger variance indicates the presence of paint edges or severe rust spots on the surface; among which, j This represents the traversal index of the valid pixels contained within the current sliding window, ∈[1,m]. m This represents the total number of pixels within the window. rj , g j , b j They represent the first j The absolute intensity values ​​of the red, green, and blue channels of each pixel; 0.299, 0.587, and 0.114 are objective weighting coefficients for grayscale conversion of standard NTSC / PAL video formats, derived from the physiological perception characteristics of human vision, and are used to losslessly reduce the dimensionality of the color RGB signal to a one-dimensional grayscale signal that reflects the degree of dirt on the physical surface. S23. Minimize the target penalty value as the global optimization objective. The target penalty value is equal to the surface smoothness feature multiplied by the first fixed weight coefficient plus the color variance feature multiplied by the second fixed weight coefficient. Traverse all sliding windows and output the three-dimensional coordinates of the center point of the window with the minimum target penalty value as the optimal excitation target point three-dimensional coordinate matrix. In specific implementation, the above embodiment S23 specifically involves: constructing the target penalty function: L ta ( k )= α * F fl ( k )+ β * F cv (k); where α and β For the pre-defined normalized weighting coefficients, the rules are set as follows: the target penalty function aims to find the region with the minimum penalty value; when laser focus depth tolerance takes priority, the following settings are made: α > β At this point, the optimizer imposes a greater penalty on surface unevenness, forcing the system to search for the flattest area; when prioritizing avoiding paint coatings, the settings are adjusted accordingly. β > α The system is forced to search for areas with the smallest color variance that are unpainted and rust-free; all candidate windows on the material surface are traversed to find the area that minimizes color variance. L ta ( k Window index that reaches the global minimum k opt Extract the geometric center coordinates of the window and output the optimal three-dimensional coordinate matrix of the excitation target. P opt =[ X opt , Y opt , Z opt ]; Furthermore, the aforementioned α , βThe surface smoothness feature set of all sliding windows of the current material is obtained by extracting it and calculating its global normalized variance. Var fl Extract the color variance feature set and calculate its global normalized variance. Var cv Based on the objective weighting method of information entropy, let , This calculation method avoids subjective manual assignment. When the material is extremely smooth overall but the surface is severely rusted, the color variance exhibits drastic fluctuations. Var cv (Extremely large), the system automatically assigns β The extremely high weighting allows for the imposition of a very high penalty value on the rusted area during optimization, ensuring that the laser target strictly avoids the rusted area. The above embodiment S2 transforms the abstract problem of finding a flat and clean surface into a mathematical problem of solving the extreme value of the angle between the normal vectors and the pixel variance, eliminating the blindness of manually set rules and ensuring that the laser pulse always hits the optimal gold point with the highest metal reflectivity and the thinnest oxide layer. The above embodiment S3 includes the following steps: S31. Based on the three-dimensional coordinate matrix of the optimal excitation target point, combined with the real-time movement speed characteristics of the conveyor belt and the system processing delay time characteristics, the dynamic prediction coordinate matrix of the laser firing moment is calculated using the kinematic compensation equation. In specific implementation, the above embodiment S31 specifically involves: extracting the real-time speed feedback from the high-speed encoder of the conveyor belt. v c Record the system computation time Δ from the end of data acquisition in S1 to this point. t calc The displacement Δ of the material on the Y-axis (direction of conveyor belt movement) is calculated using the kinematic compensation equation. Y = v c ×Δ t calc Update the target coordinates to obtain the dynamic prediction coordinate matrix. P pr =[ X opt , Y opt +Δ Y , Z opt ]; S32. Send the dynamic prediction coordinate matrix to the two-dimensional high-speed scanning galvanometer controller; trigger the dual-pulse laser to emit the first nanosecond-level cleaning laser pulse, and use plasma shock waves to peel off the surface oxide layer and paint coating at the target coordinates. In specific implementation, the above embodiment S32 specifically involves: the galvanometer controller based on... P pr Adjust the X-axis and Y-axis deflection voltages to control the Nd:YAG laser to emit a first cleaning laser pulse with a wavelength of 1064nm, a pulse width of 8ns, and an energy of E1; this low-energy pulse is not intended to generate a strong spectrum, but rather to vaporize the micron-scale impurity layer on the surface using the thermal ablation effect. S33. After a preset microsecond time interval, the dual-pulse laser is triggered to emit a second nanosecond-level probe laser pulse, which breaks through the exposed metal substrate to generate plasma; the plasma emission spectrum in the 300nm to 800nm ​​band is collected by an echelle grating spectrometer and an enhanced charge-coupled device, and a discrete spectral sequence signal containing wavelength and absolute light intensity is output. In specific implementation, the above embodiment S33 specifically involves: setting a precise pulse interval characteristic Δ in the system hardware timer. t pulse =2.5μs; after the delay, the laser emits a second probe laser pulse with energy E2. At this time, the laser directly interacts with the high-purity metal substrate, generating high-temperature plasma; the ICCD starts shutter integration after a delay of 1.0μs, acquiring a one-dimensional discrete spectral sequence signal. S raw ( l )=[ I ( l 1), I ( l 2),..., I ( l N )],in l I is the wavelength, and I is the light intensity; Furthermore, the pulse interval characteristic Δ t pulse =0.5+(G / 50), with a value range of 0.5μs-5.0μs; based on the average gray value of the material extracted from the RGB image, when the gray value is extremely low (dark surface color, high absorbance), shorten Δ t pulse The pulse duration is extended to 0.5 μs to prevent premature plasma quenching; when the gray value is high (high reflectivity), it is extended to 5.0 μs; the energy of the second probe laser pulse is E2=50+(2G / 5, with a value range of 50mJ-150mJ. The above steps call the ablation threshold of the preliminary material classification in the database, and set E2 to 1.5 to 2.0 times the plasma breakdown threshold of its matrix material to ensure sufficient excitation; The above embodiment S3 compensates for the physical displacement of the material during the operation through strict kinematic compensation, ensuring that the laser hits accurately; through the dual-pulse timing control mechanism, it uses physical shock waves to complete the online polishing at the micro level, solving the fatal defect that dirt on the surface of scrap metal causes the spectrum to be invalid. The above embodiment S4 includes the following steps: S41. Extract the discrete spectral sequence signal, calculate the continuous baseline background noise features using an asymmetric least squares smoothing algorithm; subtract the baseline background noise features from the discrete spectral sequence signal to obtain a clean spectral sequence. The above embodiment S41 includes the following steps: S411. Set the smoothing parameters for the asymmetric least squares smoothing algorithm. l smooth Compared with the initial asymmetric penalty weight sigma p asym ; In specific implementation, the above embodiment S411 specifically involves: setting the smoothing parameters of the asymmetric least squares smoothing algorithm. l smooth The optimization interval is

[10] 3 10 6 Initial asymmetric penalty weights p asym The optimization interval is [0.001, 0.05]. Furthermore, the system introduces a generalized cross-validation (GCV) criterion, performing a grid search on the pure background noise spectrum without laser excitation, and selecting the parameter combination that minimizes the GCV validation error as a fixed parameter. l smooth and p asym ; S412. Construct a penalized least squares objective function based on the discrete difference matrix; let the original spectral sequence be... y The fitted baseline is z During the iteration process, the weight matrix is ​​dynamically updated using asymmetric rules. W When the current fitted baseline feature is greater than the original light intensity feature of the discrete spectral sequence signal (i.e., the baseline is above the signal), a real-time weight is assigned. w i = p asym (Minimum weights allow the baseline to traverse noise); assign real-time weights when the current fitted baseline feature is less than or equal to the original light intensity feature (i.e., the baseline is below the signal). w i =1− p asym (Extreme weighting, forcing the baseline to fit the bottom of the signal); In specific implementation, the above embodiment S412 is as follows: the least squares objective function is: ; H ( z () represents the value of the objective function to be minimized; y i The original spectral sequence was obtained from the CCD of the spectrometer. i The actual light intensity value of each channel; z i The baseline features to be fitted are on the CCD of the spectrometer. i The light intensity value of each channel; w i The diagonal weight matrix is ​​dynamically updated based on asymmetric rules. W The diagonal elements in; S413. When the relative rate of change of the baseline feature vectors of two adjacent iterations is less than the preset convergence tolerance, stop the iteration and output the final continuous baseline background noise feature. This step removes the continuous background light interference caused by laser bremsstrahlung and amplifies the discrete line spectral features of the feature elements. In specific implementation, the above embodiment S413 specifically involves: using the matrix differentiation rule to set the gradient of the objective function to 0, and solving the linear equation system. Where D is a second-order difference matrix; where, W It is a diagonal matrix composed of dynamic weights; T This indicates the transpose; by algebraically solving this system of equations, a smooth and background-fitting baseline vector can be directly obtained. z The iteration stops when the relative rate of change of the L2 norm of the baseline feature vectors in two consecutive iterations reaches a certain threshold, and the final continuous baseline background noise features are output. z base ; Calculate the pure spectral sequence S pure = y - z base ; S42. Using the Lorentz line fitting function, perform peak search on the pure spectral sequence, extract the local maximum peak features and full width at half maximum features at the wavelengths of preset feature elements, normalize the peak intensities of different elements, and construct a chemical feature spectral matrix. In specific implementation, the above embodiment S42 specifically involves: targeting the preset characteristic element wavelengths in the database (such as aluminum Al 396.15nm, magnesium Mg 285.21nm, copper Cu 324.75nm, etc.); at each center wavelength... l Within a local window of 0, using the Lorentz equation Perform nonlinear least squares fitting; L ( l() represents the fitted continuous light intensity function; l Wavelength is the independent variable; l 0 represents the center wavelength constant determined by the physical properties of atomic transitions; I max Γ represents the absolute peak intensity of the characteristic peak; Γ is the full width at half maximum (FWHM) characteristic, reflecting the Stark broadening effect caused by plasma electron density; Extracting the peak intensity features obtained from the fitting I max With the half-height and full-width feature Γ; all target elements I max Normalized by dividing by the reference peak intensity of the matrix element (such as iron or aluminum), a chemical characteristic spectral matrix with dimension 1×C is generated. F chem ; Furthermore, to address situations where extreme pollution leads to extremely low spectral signal-to-noise ratios, the coefficient of determination R is calculated after Lorentz fitting. 2 If the target element's R 2 If the value is less than 0.8, the characteristic peak extraction of this element is deemed unsuccessful, and it is placed in the chemical characteristic spectral matrix. F chem The corresponding value in the matrix is ​​forcibly set to zero; at this point, the system will rely entirely on the physical morphological feature matrix in the subsequent cross-modal attention mechanism for a baseline downgraded recognition. S43. Extract the multimodal registration data matrix from S12 and convert it into a four-channel RGB-D image containing depth information; input the four-channel RGB-D image into a pre-trained two-dimensional residual convolutional neural network, and after multi-layer convolution and pooling dimensionality reduction, extract global texture and morphological feature vectors to generate a physical morphological feature matrix. In specific implementation, the above embodiment S43 specifically involves: ... M fused Projected onto a two-dimensional plane, missing pixels are filled using bilinear interpolation, specifically by generating a fixed-size [224×224×4] RGB-D tensor; input into a pre-trained ResNet-50 backbone network; after convolution operations of 4 residual blocks and a global average pooling layer, the high-dimensional image is compressed into a physical morphological feature matrix of dimension 1×2048. F phys The feature matrix contains the physical geometry information of the material; based on the physical geometry information of the material, it can be determined whether the material is an extruded profile, a die casting, or a machined waste wire; the physical geometry information includes aspect ratio, surface curvature variance, profile moment characteristics, and voxel filling rate; extruded profiles are regular long strips, die castings have complex surface curvature, and machined waste wires have a very large aspect ratio and loose profile; Furthermore, the pre-training process of the pre-trained ResNet-50 backbone network is as follows: A training set of approximately 50,000 four-channel RGB-D images covering various types of scrap steel, scrap aluminum, and scrap copper under different lighting and rusting conditions is constructed; hard labels reflecting physical morphology are assigned to the images using manual labeling (e.g., 0-straight extruded parts, 1-complex curved die-cast parts, 2-curved turned wire, 3-fragmented stamping scrap); the model adopts the ResNet-50 basic architecture, modifying the input layer to 4 channels to be compatible with depth maps; transfer learning is performed using ImageNet weights, freezing the parameters of the first two residual blocks, and focusing on fine-tuning the last two residual blocks; the Adam optimizer is used, with an initial learning rate set to 0.0001; the cross-entropy loss function is used; training stops when the accuracy on the validation set does not increase for 5 consecutive epochs; after training, the fully connected classification layer is stripped, and the global average pooling layer is retained as the output, with an output feature vector of dimension 1×2048, which is the physical morphology feature matrix; S44. After fully connecting the chemical feature spectral matrix of S42 and the physical morphology feature matrix of S43 to reduce the dimension to the same dimension, perform orthogonal concatenation of row vectors to generate a cross-modal joint feature matrix. In specific implementation, the above embodiment S44 specifically involves: utilizing the weight matrix of the fully connected layers of the pre-trained ResNet-50 backbone network. W c1 and W p1 ,Will F chem and F phys Linearly mapped to a unified hidden layer dimension respectively. d mode Perform a matrix concatenation operation to generate a cross-modal joint feature matrix with a dimension of 1×256. F joint =[ F chem1 ⊕ F phys1 ]; ⊕ represents the concatenation operation of tensors along the feature dimension; F chem1 These are the eigenvectors of the chemical characteristic spectral line matrix after linear dimensionality reduction. F phys1 The eigenvectors are the physical morphological feature matrices after linear dimensionality reduction; orthogonal concatenation ensures that physical and chemical features have equal initial mathematical dimensions in the subsequent attention mechanism; The above embodiment S4 constructs a high-dimensional orthogonal feature space that combines internal chemical composition with external physical morphology. Traditional LIBS is prone to misjudgment when it encounters severe surface contamination that leads to a decrease in spectral signal-to-noise ratio. However, this invention introduces physical appearance features extracted by convolutional networks, providing a multi-dimensional cross-validation data source for subsequent attention fusion. The above embodiment S5 includes the following steps: S51. Input the cross-modal joint feature matrix into the deep learning recognition model, and perform linear mapping on the chemical feature spectral matrix and the physical morphology feature matrix respectively to generate a query matrix, a bond matrix and a value matrix. In specific implementation, the above embodiment S51 specifically involves: during the online inference stage, extracting the pre-trained deep learning recognition model; the model includes a one-layer cross-modal attention module (Cross-Attention, containing four attention heads, each with a dimension of 32), and an MLP classification head composed of two fully connected layers (with 128 and 64 neurons respectively, using the GELU activation function); using the linear mapping matrix of the model input layer, mapping the 1×128 dimension chemical feature spectral line matrix into a query matrix. Q chem Mapping a 1×128 dimension physical morphological feature matrix to a bond matrix K phys AND-value matrix V phys ; Furthermore, the offline pre-training process of the deep learning recognition model is as follows: constructing a multimodal training set containing 100,000 paired data points; using Focal Loss as the loss function (setting the focusing parameter γ=2.0, and the balancing parameter...). α =0.25 to address the imbalance problem of rare grades of scrap metal samples); AdamW optimizer is used for backpropagation; batch size is set to 128, and training epochs are set to 200 (this number of epochs is objectively set based on the convergence characteristic that the validation set loss curve tends to flatten out around 180 epochs); after training, the weight matrix is ​​fixed for the above inference stage; S52. Calculate the original association scores of each modality feature using the inner product scaling function of the cross-modal attention mechanism, and transform the original association scores into a dynamic weight allocation matrix using the normalized exponential function. The above embodiment S52 includes the following steps: S521. Extract the chemical query matrix generated in S51. Q chem = F chem1 * W Q With physical bond matrix Kphys = F phys1 * W K The cross-covariance feature matrix between the chemical query matrix and the transpose of the physical bond matrix is ​​calculated by performing a dot product operation. E = Q chem *( K phys ) T , where T represents transpose; In specific implementation, the above embodiment S521 is as follows: W Q and W K These are the learnable parameter weight matrices used to generate the query matrix and key matrix in the attention mechanism, and their values ​​are fixed by the gradient descent algorithm during the offline pre-training stage. S522, Divide the cross-covariance feature matrix by the square root feature of the key matrix dimension. Numerical scaling is performed to prevent the inner product from becoming too large and causing gradient vanishing, resulting in the original correlation score matrix. ; S523. Substitute the original correlation scoring matrix into the Softmax normalization equation; calculate the ratio of the natural index value of the current feature dimension score to the sum of the natural index values ​​of all feature dimensions, and output a dynamic weight allocation matrix with a sum of 1. A weight =Softmax( S score This matrix quantifies the model's dependence on visual physical morphological features when the signal-to-noise ratio of the spectral signal fluctuates. If a certain alloy element peak in the current LIBS spectrum is extremely weak (low confidence in the chemical feature), the attention mechanism will use dot product operations to discover that it is highly correlated with the die-cast texture in the visual features, thereby automatically amplifying the physical value matrix. V phys Weighting coefficients; The above mechanism theoretically belongs to the cross-modal attention architecture, and its core logic lies in: utilizing the chemical query matrix Q chem To retrieve the physical bond matrix K phys The calculated dynamic weight allocation matrix A weight Directly affects the physical value matrix V physThat is, the confidence level of chemical spectral features determines how much supplementary information the system needs to extract from visual physical form, ultimately forming a dynamic feature fusion with chemical composition as the main component and physical form as the auxiliary component. S53, Connect the weight coefficients in the dynamic weight allocation matrix with the corresponding physical value matrix. V phys Element-wise multiplication and summation are performed to output a weighted context feature vector that integrates chemical and physical properties. C fused = A weight * V phys + F chem1 ; S54. Input the weighted context feature vector into the fully connected classification layer of the deep learning recognition model, and use the weight matrix of the fully connected classification layer. W out With bias b out Perform a linear mapping, mapping to a dimension equal to the total number of alloy categories. N class Logical output tensor Logits ; S55. Substitute the logic output tensor into the Softmax function to calculate the conditional probability distribution sequence of the material belonging to each label category. Extract the alloy grade label corresponding to the feature with the highest probability value; where, P i This indicates that the material belongs to the first... i The conditional probability value of a type of alloy; exp represents an exponential function with the natural constant e as the base; Logits j This represents the corresponding tensor in the output tensor of the fully connected layer. j The original logical scores for each category; the denominator is all N class The sum of the index scores of each alloy category ensures that the sum of the probabilities of the output sequence is always 1. The above embodiment S6 includes the following steps: S61. Perform voxelization downsampling on the multimodal registration data matrix and set the side length reference feature of a single voxel; count the total number of effective voxels containing the material point cloud, multiply the total number of effective voxels by the volume of a single voxel, and calculate the global volume feature; extract the geometric center coordinates of the spatial bounding box feature to obtain the spatial contour reference feature. In specific implementation, the above embodiment S61 specifically involves: setting the voxel grid side length reference feature. L v The volume characteristics of a single voxel Vvoxel = L v 3 Divide the three-dimensional space into a three-dimensional grid, and count the data points containing at least one data point. x i , y i , z i Total number of effective voxel grids N valid ; Calculate global volume features V total = N valid × V voxel ; Furthermore, to eliminate volume estimation errors caused by irregular scrap metal (such as internal cavities in curled scrap wire), the system calls a preset morphology compensation coefficient based on the physical morphological characteristics output by S5. K v (The morphological compensation coefficient) K v The volume of solid extruded parts was obtained through offline calibration experiments. Typical waste material samples were selected, and the actual volume was measured using the displacement method. The ratio of the measured volume to the voxel-estimated volume was determined after multiple sets of regression fitting. K v =0.95, curled waste yarn K v =0.60), calculate the corrected global volume feature. V 1 total = V total × K v ; Simultaneously, the extreme values ​​of the point cloud in the X, Y, and Z dimensions are extracted to construct a spatial bounding box, and the geometric center coordinates of the spatial bounding box are calculated. X center , Y center , Z center As a spatial profile reference feature, the center coordinates will serve as the physical point of application for calculating the valve purging torque in the S5 stage; S62. Establish an alloy standard density database; based on the alloy grade label output in S55, retrieve the corresponding standard density characteristics from the database. r std (For example, the density of 6063 aluminum is 2.7 g / cm³) 3 The standard density feature is multiplied by the global volume feature in S61 to calculate the real-time mass feature of the material.M real = r std × V 1 total ; S63. Based on Newton's second law and the momentum theorem of aerodynamics, calculate the lateral target velocity characteristic required to blow the material with the real-time mass characteristic away from the parabolic trajectory of the conveyor belt. v y Multiply the real-time mass characteristic by the lateral target velocity characteristic to calculate the required dynamic aerodynamic impulse matrix. I req = M real × v y ; S64. Based on the dynamic aerodynamic impulse matrix, combined with the nozzle cross-sectional area and rated flow velocity of the high-pressure valve, the target opening pressure threshold and target opening duration characteristics of the high-pressure valve are calculated by back-calculating using the Bernoulli equation of fluid mechanics. In specific implementation, the above embodiment S64 specifically involves: based on the dynamic aerodynamic impulse matrix, combined with the nozzle cross-sectional area of ​​the high-pressure air valve. A nozzle With gas density r air The target opening pressure threshold of the high-pressure valve can be calculated by back-calculating using fluid dynamics equations. P valve Target activation duration feature Δ t open The calculation formula is: impulse I req = F thrust ×Δ t open According to the fluid momentum theorem, the formula for the dynamic thrust generated by high-speed airflow jet is: F thrust =2 C d * A nozzle * P valve ;in, C d This is the dimensionless flow coefficient of the air valve nozzle (obtained through fluid simulation or hardware calibration, typically taken as 0.6-0.8). A nozzle The absolute physical cross-sectional area constant inside the nozzle; the system prioritizes fixing the working air pressure threshold based on the mechanical response limit of the air valve solenoid. P valve(e.g., 0.6 MPa), calculate the required on-time Δ t open =( I req ) / (2 C d * A nozzle * P valve If the time exceeds the physical time window for the material to pass through the air valve, the air pressure threshold will be automatically increased proportionally. P valve ; S65. Based on the spatial contour reference features extracted in S61 and the system conveyor belt speed, calculate the time delay of the material arriving in front of the air valve array; when the time delay arrives, send the target opening air pressure threshold and target opening duration features to the corresponding electromagnetic air valve to accurately blow the target material into the corresponding collection bin. In specific implementation, the above embodiment S65 specifically refers to: the geometric center of the spatial contour reference feature extracted in S13. Y center With system conveyor belt speed v c Calculate the time stamp of the delay when the material arrives directly in front of the valve array. T delay = D stv / ( v c - Y center );in, D stv It represents the absolute physical distance constant between the starting point of the scanning baseline of the 3D laser profilometer, along the direction of the conveyor belt movement, and the center line of the nozzle of the high-pressure air valve array at the end, in the system hardware layout. When the delayed timestamp arrives, the target opening pressure threshold and opening duration are sent to the corresponding solenoid valve to accurately blow the target material into the corresponding collection hopper. The above embodiment S5 constructs an end-to-end intelligent recognition and physical execution closed loop; by introducing a cross-modal attention mechanism, the pain point of decreased recognition rate caused by sensor signal fluctuations is solved at the mathematical level; and at the sorting execution end, the traditional timed and pressured blind blowing mode is abandoned. By recognizing the grade, matching the density, combining 3D volume, calculating the mass and deriving the momentum fluid dynamics and kinematics calculation, the air valve impulse is customized for each piece of scrap metal, regardless of size, solving the missorting problem of large heavy materials not being blown and small light materials being blown away and bounced around.

[0015] Please see Figure 2A smart identification and sorting system for recycled metals based on LIBS spectroscopy and multimodal sensing is provided, realizing the aforementioned smart identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing. The system includes a multimodal feature sensing module, a laser dynamic targeting and dual-pulse excitation module, a cross-modal feature decoupling and splicing fusion module, a deep attention classification module, and a dynamic impulse sorting module. Wherein: The aforementioned multimodal feature perception module is used to eliminate background point clouds and image distortion, and generate a multimodal registration data matrix; The aforementioned laser dynamic targeting and dual-pulse excitation module is used to obtain the optimal excitation target by using surface flatness and color variance as optimization targets; and to perform cleaning and excitation based on motion compensation control of the dual-pulse laser to obtain a one-dimensional discrete spectral sequence signal. The aforementioned cross-modal feature decoupling and splicing fusion module is used to extract the chemical feature spectral line matrix of the spectrum and to extract the physical morphological feature matrix of the vision using a convolutional network; after the two are fully connected and dimensionality reduced, they are orthogonally spliced ​​to generate a cross-modal joint feature matrix. The aforementioned deep attention classification module is used to input the cross-modal joint feature matrix into the deep model, calculate the dynamic weight allocation matrix using the inner product scaling function, and determine the alloy grade based on the weighted features. The aforementioned dynamic impulse sorting module is used to calculate the global volume characteristics and spatial contour reference characteristics of the material; based on the alloy grade label matching standard density, combined with the global volume to calculate the real-time mass, and based on the momentum theorem to output a dynamic aerodynamic impulse matrix to perform precise valve sorting; This invention represents a leap from the traditional single-point blind testing and blind constant-pressure blowing of sorting equipment to intelligent dynamic targeting, multi-dimensional cross-modal fusion, and customized blowing. The complexity of scrap metal lies not only in the similarity of its internal chemical composition but also in the uncontrollability of its external physical form and surface contamination. Therefore, this invention deeply couples physical form recognition (3D vision) with chemical composition recognition (LIBS spectroscopy), eliminating sensor blind spots through mathematical logic. Simultaneously, at the execution end, the recognition results are converted into physical density, and combined with volume reconstruction, a closed loop for mass calculation is established. Aerodynamic impulse replaces empirical air pressure control. This constructs a closed-loop link combining precise environmental perception, multi-dimensional cognitive decision-making, and flexible physical execution. Even on harsh industrial production lines with strong interference and numerous impurities, it maintains extremely high alloy grade recognition and sorting accuracy, establishing a solid technical foundation for high-value-added recycling of recycled metals.

[0016] The following detailed explanation is provided with reference to specific embodiments: Example

[0017] This embodiment uses a production line at a recycling plant that separates 6063 wrought aluminum alloy (used in building doors and windows, high value) from ADC12 die-cast aluminum alloy (used in automobile engine housings, contains many impurities, low value) as an example to illustrate the method of the present invention in detail: At a width of 1.2 meters and a running speed v c On a high-speed conveyor belt with a speed of 3.0 m / s, a piece of scrap aluminum alloy profile with some white fluorocarbon paint on its surface passes through the inspection area. Triggered by a photoelectric switch, the 3D laser profilometer and RGB camera simultaneously acquire data; after removing the conveyor belt background, the system uses intrinsic and extrinsic parameter matrix projection to generate a multimodal registration data matrix containing 3500 points. M fused Set voxel side length L v =2mm, a total of 1875 effective voxels containing point clouds were counted, and the global volumetric feature of the material was calculated. V 1 total =1875×8=15000mm 3 =15cm 3 Extract the geometric center coordinates of its bounding box. Y center =125.4mm; The system initiates a local window sliding search algorithm, with the window side length set to 1.5mm; in the area with white paint, the local color variance features are calculated. F cv Up to 185.6; in the inclined area of ​​the profile edge, the surface flatness characteristics are calculated by fitting the angle between the normal vectors of the micro-cut planes. F fl =0.82; Substitute into the penalty function L ta ( k ) = 0.4 × F fl +0.6× F cv These areas had extremely high penalty values ​​and were eliminated; the system finally found the globally optimal window with the minimum penalty value at the unpainted and flat cross-section of the profile, and output the optimal three-dimensional coordinate matrix of the excitation target point. P opt =[X=452.1,Y=132.5,Z=25.4]; System calculation time Δ t calc=12ms, compensate for conveyor belt displacement ΔY=3.0m / s×12ms=36mm; the galvanometer quickly deflects to the dynamic prediction coordinates [452.1,168.5,25.4]; the laser emits the first cleaning pulse with an energy of 30mJ, ablates the extremely thin natural oxide film on the target surface; after a hardware timer delay of 2.5μs, the second detection pulse with an energy of 80mJ is emitted, which breaks down the pure aluminum substrate to generate plasma; the spectrometer acquires the original spectral sequence; After removing the continuous bremsstrahlung baseline using the asymmetric least squares (AsLS) method, the pure spectral peaks were extracted using Lorentz fitting. It was found that Al had an extremely strong peak at 396.15 nm, Mg had a distinct characteristic peak at 285.21 nm, while Si had an extremely weak peak at 288.15 nm (consistent with the chemical characteristics of low silicon content in 6-series wrought aluminum), generating a chemical characteristic spectral matrix. F chem ; Simultaneously, the ResNet-50 network extracts RGB-D image features, identifies its physical geometric features with elongated, straight edges (complex curved surface shapes not found in die-cast parts), and generates a physical morphology feature matrix. F phys ; Chemical characteristic spectral line matrix F chem With physical morphological feature matrix F phys The input is concatenated into a deep learning recognition model; the dot product of the chemical query matrix and the physical bond matrix is ​​calculated through a cross-modal attention mechanism. Due to the high degree of consistency between the two features, the dynamic weight allocation matrix output by the Softmax function is given highly concentrated weights (e.g., 0.85) in the corresponding dimension; the fully connected layer outputs the final conditional probability distribution sequence: P=[0.01(ADC12),0.98(6063),0.01(pure copper),...]; the system determines that the material is 6063 wrought aluminum alloy; Access the database to match the standard density characteristics of 6063 aluminum alloy. r std =2.7g / cm 3 The calculated volume is 15cm². 3 Calculate the real-time quality characteristics of the material. M real =2.7 × 15 = 40.5 g = 0.0405 kg; The required lateral velocity to blow it into the recovery bins 0.5 meters apart is set to 2.7 × 15 = 40.5 g = 0.0405 kg; v y =4.0m / s, calculate the required dynamic aerodynamic impulse. I req =0.0405×4.0=0.162N*s; The system reverse-engineers the fluid dynamics equations and sets the working air pressure of the air valve. P valve =0.6MPa, calculate the time Δ required for a single air valve to open to achieve this impulse. t open =12ms; based on the physical distance from the sensor to the valve array D =2.0m and belt speed 3.0m / s, calculate the delay timestamp; when the material reaches the front of the array, the controller precisely triggers the corresponding position of air valve No. 42, which opens for 12ms with an air pressure of 0.6MPa; the 0.162N*s impulse generated by the airflow overcomes the inertia brought by the 40.5g mass of the material, and accurately blows it into the 6063 deformed aluminum special material bin, avoiding the phenomenon of material flying away due to excessive impulse or falling into the miscellaneous material bin due to insufficient impulse, thus realizing the high-value closed-loop recycling of resources.

[0018] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0019] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A method for intelligent identification and sorting of recycled metals based on LIBS spectroscopy and multimodal sensing, characterized in that, Includes the following steps: S1. Extract the real-time three-dimensional point cloud signal and two-dimensional color image signal of the waste metal material moving on the conveyor belt, and generate a multimodal registration data matrix through spatial coordinate mapping. S2. Using the surface smoothness features and color variance features of the multimodal registration data matrix as optimization parameters of the objective function, the optimal three-dimensional coordinate matrix of the excitation target point for laser-induced breakdown spectrum is obtained by using the local window sliding search algorithm. S3. Based on the three-dimensional coordinate matrix of the optimal excitation target point, control the dual-pulse laser to clean and excite the waste metal material, collect the plasma emission spectrum, and convert it into a discrete spectral sequence signal; S4. Baseline calibration is performed on the discrete spectral sequence signal using the asymmetric least squares method to extract the chemical feature spectral matrix; The physical morphological feature matrix of the multimodal registration data matrix is ​​extracted using a convolutional neural network; S5. Input the chemical feature spectral line matrix and physical morphology feature matrix into the deep learning recognition model, and use the inner product scaling function of the cross-modal attention mechanism to calculate the dynamic weight allocation matrix of each modal feature; output the probability distribution sequence based on the dynamic weight allocation matrix to determine the alloy grade label; S6. Calculate the global volume features and spatial contour reference features of the multimodal registration data matrix; Based on the alloy grade label matching standard density, and combined with the global volume characteristics, the real-time mass characteristics are calculated. Based on the real-time quality characteristics and spatial contour reference characteristics, the dynamic aerodynamic impulse matrix is ​​calculated, and the valve array is controlled to perform sorting.

2. The intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to claim 1, characterized in that, S1 includes the following steps: S11. Establish a data acquisition synchronization time window model for multimodal sensors; when a photoelectric switch trigger signal is detected, start the 3D laser profilometer and high frame rate RGB camera; extract historical background point clouds and scrap metal material point clouds, use a pass-through filter to remove the conveyor belt background, and stitch them together to generate the original three-dimensional point cloud sequence. S12. Extract the spatial bounding box features of the original 3D point cloud sequence; using the pinhole camera projection equation based on the camera intrinsic parameter matrix and extrinsic parameter rotation and translation matrix, map the RGB pixels of the 2D color image signal to the 3D spatial coordinate system of the original 3D point cloud sequence to obtain the multimodal registration data matrix.

3. The intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to claim 2, characterized in that, S2 includes the following steps: S21. Define the search space of the local window sliding search algorithm; define the side length of the three-dimensional optimization window as 3 times the diameter feature of the laser spot; slide the three-dimensional optimization window in the top projection area of ​​the multimodal registration data matrix at a set step size; S22. For any point cloud set within a sliding window, use principal component analysis to fit a local micro-tangent plane, calculate the angle between the normal vector of the local micro-tangent plane and the absolute Z-axis, and generate surface smoothness features; extract the RGB pixel values ​​of all points within the sliding window, calculate the pixel variance, and generate color variance features. S23. Minimize the target penalty value as the global optimization objective. The target penalty value is equal to the surface smoothness feature multiplied by the first fixed weight coefficient plus the color variance feature multiplied by the second fixed weight coefficient. Traverse all sliding windows and output the three-dimensional coordinates of the center point of the window with the minimum target penalty value as the optimal excitation target point three-dimensional coordinate matrix.

4. The intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to claim 3, characterized in that, S3 includes the following steps: S31. Based on the three-dimensional coordinate matrix of the optimal excitation target point, combined with the real-time movement speed characteristics of the conveyor belt and the system processing delay time characteristics, the dynamic prediction coordinate matrix of the laser firing moment is calculated using the kinematic compensation equation. S32. Send the dynamic prediction coordinate matrix to the two-dimensional high-speed scanning galvanometer controller; trigger the dual-pulse laser to emit the first nanosecond-level cleaning laser pulse, and use plasma shock waves to peel off the surface oxide layer and paint coating at the target coordinates. S33. After a preset microsecond time interval, the dual-pulse laser is triggered to emit a second nanosecond-level probe laser pulse, which breaks through the exposed metal substrate to generate plasma. The plasma emission spectrum is collected by an echelle grating spectrometer and an enhanced charge-coupled device, and a discrete spectral sequence signal containing wavelength and absolute light intensity is output.

5. The intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to claim 4, characterized in that, S4 includes the following steps: S41. Extract the discrete spectral sequence signal, calculate the continuous baseline background noise features using an asymmetric least squares smoothing algorithm; subtract the baseline background noise features from the discrete spectral sequence signal to obtain a clean spectral sequence. S42. Using the Lorentz line fitting function, perform peak search on the pure spectral sequence, extract the local maximum peak features and full width at half maximum features at the wavelengths of preset feature elements, normalize the peak intensities of different elements, and construct a chemical feature spectral matrix. S43. Extract the multimodal registration data matrix from S12 and convert it into a four-channel RGB-D image containing depth information; input the four-channel RGB-D image into a pre-trained two-dimensional residual convolutional neural network, and after multi-layer convolution and pooling dimensionality reduction, extract global texture and morphological feature vectors to generate a physical morphological feature matrix. S44. After fully connecting the chemical feature spectral matrix of S42 and the physical morphological feature matrix of S43 to reduce the dimension to the same dimension, perform orthogonal concatenation of row vectors to generate a cross-modal joint feature matrix.

6. The intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to claim 5, characterized in that, S41 includes the following steps: S411. Set the smoothing parameters and initial asymmetric penalty weights for the asymmetric least squares smoothing algorithm. S412. Construct a penalized least squares objective function based on a discrete difference matrix; set the original spectral sequence and the fitting baseline; during the iteration process, when the current fitting baseline feature is greater than the original light intensity feature of the discrete spectral sequence signal, assign the real-time asymmetric penalty weight feature as 1 minus the initial asymmetric penalty weight feature; when the current fitting baseline feature is less than the original light intensity feature, the real-time asymmetric penalty weight feature is the initial asymmetric penalty weight feature. S413. When the relative rate of change of the baseline feature vectors in two adjacent iterations is less than the preset convergence tolerance, stop the iteration and output the continuous baseline background noise features.

7. The intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to claim 5, characterized in that, S5 includes the following steps: S51. Input the cross-modal joint feature matrix into the deep learning recognition model, and perform linear mapping on the chemical feature spectral matrix and the physical morphology feature matrix respectively to generate a query matrix, a bond matrix and a value matrix. S52. Calculate the original association scores of each modality feature using the inner product scaling function of the cross-modal attention mechanism, and transform the original association scores into a dynamic weight allocation matrix using the normalized exponential function. S53. Multiply the weight coefficients in the dynamic weight allocation matrix element by element with the corresponding physical value matrix and sum them to output the weighted context feature vector. S54. Input the weighted context feature vector into the fully connected classification layer of the deep learning recognition model, and use the weight matrix and bias of the fully connected classification layer to perform a linear mapping to a logical output tensor with a dimension equal to the total number of alloy categories. S55. Substitute the logic output tensor into the Softmax function to calculate the conditional probability distribution sequence of the material belonging to each label category.

8. The intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to claim 7, characterized in that, S52 includes the following steps: S521. Extract the chemical query matrix and physical bond matrix generated in S51, perform a dot product operation on the transpose of the chemical query matrix and the physical bond matrix, and calculate the cross-covariance feature matrix between the chemical signal and the physical visual signal. S522. Divide the cross-covariance feature matrix by the square root feature of the key matrix dimension and perform numerical scaling to obtain the original association score matrix; S523. Substitute the original correlation score matrix into the Softmax normalization equation; calculate the ratio of the natural index value of the current feature dimension score to the sum of the natural index values ​​of all feature dimensions, and output a dynamic weight allocation matrix with a sum of 1.

9. The intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing according to claim 7, characterized in that, S6 includes the following steps: S61. Perform voxelization downsampling on the multimodal registration data matrix and set the side length reference feature of a single voxel; count the total number of effective voxels containing the material point cloud, multiply the total number of effective voxels by the volume of a single voxel, and calculate the global volume feature; extract the geometric center coordinates of the spatial bounding box feature to obtain the spatial contour reference feature. S62. Establish an alloy standard density database; based on the alloy grade label output in S55, retrieve the corresponding standard density feature from the alloy standard density database; multiply the standard density feature by the global volume feature in S61 to calculate the real-time mass feature of the material. S63. Based on Newton's second law and the momentum theorem of aerodynamics, calculate the lateral target velocity characteristic of the material blown off the conveyor belt by the parabolic trajectory of the real-time mass characteristic; multiply the real-time mass characteristic by the lateral target velocity characteristic to calculate the dynamic aerodynamic impulse matrix; S64. Based on the dynamic aerodynamic impulse matrix, combined with the nozzle cross-sectional area and rated flow velocity of the high-pressure valve, the target opening pressure threshold and target opening duration characteristics of the high-pressure valve are calculated by back-calculating using the Bernoulli equation of fluid mechanics. S65. Based on the spatial contour reference features extracted in S61 and the system conveyor belt speed, calculate the delay timestamp of the material arriving in front of the air valve array; when the delay timestamp arrives, send the target opening air pressure threshold and target opening duration features to the corresponding electromagnetic air valve to accurately blow the target material into the corresponding collection bin.

10. A smart identification and sorting system for recycled metals based on LIBS spectroscopy and multimodal sensing, characterized in that, The system implementing the intelligent identification and sorting method for recycled metals based on LIBS spectroscopy and multimodal sensing as described in any one of claims 1-9, comprises: The multimodal feature perception module is used to eliminate background point clouds and image distortion, and generate a multimodal registration data matrix; The laser dynamic targeting and dual-pulse excitation module is used to obtain the optimal excitation target by using surface flatness and color variance as optimization targets; the dual-pulse laser is controlled by motion compensation to perform cleaning and excitation to obtain a one-dimensional discrete spectral sequence signal. The cross-modal feature decoupling and splicing fusion module is used to extract the chemical feature spectral line matrix of the spectrum and extract the physical morphological feature matrix of the vision using a convolutional network; the two are fully connected and dimensionality reduced, and then orthogonally spliced ​​to generate a cross-modal joint feature matrix. The deep attention classification module is used to input the cross-modal joint feature matrix into the deep model, calculate the dynamic weight allocation matrix using the inner product scaling function, and determine the alloy grade based on the weighted features. The dynamic impulse sorting module is used to calculate the global volume characteristics and spatial contour reference characteristics of the material; based on the alloy grade label, it matches the standard density and calculates the real-time mass in combination with the global volume; and based on the momentum theorem, it outputs a dynamic aerodynamic impulse matrix to perform precise air valve sorting.

Images