Method, device, system and storage medium for multi-modal data analysis of scrap recycling

Abstract

The present application relates to a multi-modal data analysis method, system, device and storage medium for scrap steel recycling, the method comprising: collecting multi-modal data of scrap steel to be recycled to obtain visual data, three-dimensional geometric data and impact acoustic data; extracting scrap steel features from the visual data, the three-dimensional geometric data and the impact acoustic data to obtain scrap steel feature parameters; performing hierarchical classification and attribute analysis on the scrap steel to be recycled based on the scrap steel feature parameters to obtain classification results and attribute prediction results; and performing recycling analysis and construction on the scrap steel to be recycled according to the classification results and the attribute prediction results to obtain a scrap steel recycling report. The present application constructs a comprehensive digital characterization system for scrap steel by simultaneously collecting visual, three-dimensional geometric and impact acoustic data, overcoming the defect of insufficient single sensing information.

Description

Technical Field

[0001] This invention relates to the technical field of scrap steel recycling, and in particular to a multimodal data analysis method, apparatus, system, and storage medium for scrap steel recycling. Background Technology

[0002] As a crucial link in the green and low-carbon development of the steel industry, the efficient and accurate sorting and value assessment technologies for scrap steel have become a core requirement for the industry's transformation and upgrading. Traditional scrap steel recycling processes rely primarily on simple identification using single sensors, making it difficult to quickly and accurately classify and quantitatively analyze the internal composition of scrap steel materials from complex sources and in diverse forms. This reliance on simple technology is not only inefficient and costly in sorting, but also suffers from strong subjectivity and large errors in judging the value attributes of scrap steel, such as material type and characteristics, leading to unstable recycling quality and making it difficult to guarantee the accuracy and economic efficiency of subsequent smelting and batching. Existing methods typically lack the ability to collaboratively analyze the multi-dimensional attributes of scrap steel, failing to generate integrated and standardized recycling decision reports, thus hindering the in-depth development of the scrap steel recycling industry towards intelligentization. Summary of the Invention

[0003] The main objective of this invention is to provide a multimodal data analysis method, apparatus, system, and storage medium for scrap steel recycling. By simultaneously acquiring visual, three-dimensional geometric, and impact acoustic data, a comprehensive digital characterization system for scrap steel is constructed, overcoming the limitations of insufficient information from a single sensor. Through automated feature extraction and parametric integration, multi-source heterogeneous data is transformed into standard feature parameters, achieving an objective quantitative description of the surface morphology, internal structure, and material properties of scrap steel, significantly improving the consistency and repeatability of identification.

[0004] To achieve the above objectives, the present invention provides a multimodal data analysis method for scrap steel recycling, comprising: Multimodal data acquisition was performed on the scrap steel to be recycled to obtain visual data, three-dimensional geometric data and impact acoustic data; The visual data, the three-dimensional geometric data, and the impact acoustic data are used to extract scrap steel features to obtain scrap steel feature parameters; Based on the characteristic parameters of the scrap steel, the scrap steel to be recycled is classified hierarchically and analyzed for attributes to obtain classification results and attribute prediction results. Based on the classification results and the attribute prediction results, a recycling analysis is performed on the scrap steel to be recycled to obtain a scrap steel recycling report.

[0005] Furthermore, the process of acquiring multimodal data from the scrap steel to be recycled, obtaining visual data, three-dimensional geometric data, and impact acoustic data, includes: The surface image sequence of the scrap steel to be recycled is acquired by a visual acquisition device, and the surface image sequence is combined into the visual data. The surface point cloud coordinates of the scrap steel to be recycled are obtained by a 3D scanning device, and the 3D geometric data is constructed based on the surface point cloud coordinates. Acoustic sensors are used to acquire the acoustic signal sequence generated when a standard impact force is applied to the scrap steel to be recycled, and the acoustic signal sequence is recorded as the impact acoustic data. When an occluded area is detected in the surface image sequence, the three-dimensional scanning device is controlled to perform supplementary scanning of the occluded area, and the supplementary point cloud coordinates obtained from the supplementary scanning are merged into the surface point cloud coordinate sequence, and the three-dimensional geometric data is updated.

[0006] Further, the step of extracting scrap steel features from the visual data, the three-dimensional geometric data, and the impact acoustic data to obtain scrap steel feature parameters includes: The color distribution histogram and surface texture gradient sequence are extracted and integrated from the visual data to obtain the first feature set; Extract the point cloud coordinates from the three-dimensional geometric data, calculate the contour morphology parameters of the scrap steel to be recycled based on the point cloud coordinates, and integrate all the contour geometric parameters to obtain the second feature set; The time-domain energy envelope and frequency-domain peak distribution are extracted and integrated from the impact acoustic data to obtain a third feature set; The first feature set, the second feature set, and the third feature set are merged to obtain the scrap steel feature parameters.

[0007] Further, the point cloud coordinate set is extracted from the three-dimensional geometric data, and the contour morphology parameters of the scrap steel to be recycled are calculated based on the point cloud coordinate set. All the contour geometric parameters are then integrated to obtain a second feature set, including: Traverse the point cloud coordinates in the point cloud coordinate set and calculate the spatial distance between each point cloud coordinate and all its adjacent coordinates. Each spatial distance value is compared with a preset distance threshold. When a spatial distance value is greater than the preset distance threshold, both point cloud coordinate points corresponding to the spatial distance value are marked as contour boundary points. Connect all the boundary points of the contour to obtain a two-dimensional contour polygon; The ratio between the area and perimeter of the two-dimensional contour polygon is calculated to obtain the contour compactness parameter. The contour compactness parameter is then integrated with the contour boundary points to obtain the contour geometric parameters. Integrate all the aforementioned contour morphology parameters to obtain the second feature set.

[0008] Furthermore, the hierarchical classification and attribute analysis of the scrap steel to be recycled based on the scrap steel characteristic parameters, to obtain classification results and attribute prediction results, includes: The scrap steel feature parameters are identified according to the first-level classification rules of the preset identification rule set to obtain the primary material category. Based on the primary material category, the corresponding secondary classification rule is selected from the preset identification rule set. The secondary material identification is performed on the scrap steel feature parameters and the primary material category according to the secondary classification rule to obtain the classification result. The scrap steel feature parameters are labeled based on a pre-set attribute mapping table to obtain attribute prediction labels. The attribute prediction label is integrated with the attribute prediction label to obtain the attribute prediction result.

[0009] Furthermore, based on a pre-set attribute mapping table, the characteristic parameters of the scrap steel are labeled to obtain attribute prediction labels, including: Traverse each mapping rule in the attribute mapping table, wherein the mapping rule includes rule feature conditions and target attribute labels; The scrap steel feature parameters are matched sequentially with each of the mapping rules. When it is detected that the scrap steel feature parameters satisfy the rule feature conditions of the current mapping rule, the corresponding target attribute label is marked as a candidate label. When there are multiple candidate labels, all candidate labels are sorted by priority according to the attribute mapping table, and the candidate label with the highest priority is selected as the attribute prediction label. If no candidate labels are generated after the traversal is completed, the predefined default attribute label will be output as the attribute prediction label.

[0010] Further, the step of constructing a scrap steel recycling report by performing recycling analysis on the scrap steel to be recycled based on the classification results and the attribute prediction results includes: The classification results are matched with a preset recycling strategy library to obtain a basic recycling processing path and a unit weight benchmark value. The attribute prediction results are evaluated based on the recycling evaluation criteria in the recycling strategy library to obtain a quality evaluation coefficient. The recycling value is calculated based on the unit weight benchmark value and the quality evaluation coefficient to obtain a recycling value score; Based on the basic recycling processing path and the recycling value score, the corresponding report template is extracted from the recycling strategy library. The classification results, attribute prediction results, recycling value score, and basic recycling processing path are filled into the report template to obtain the scrap steel recycling report.

[0011] The present invention also provides a multimodal data analysis device for scrap steel recycling, applied to the multimodal data analysis method for scrap steel recycling described in any one of the above-mentioned methods, comprising: The acquisition module is used to acquire multimodal data of the scrap steel to be recycled, and obtain visual data, three-dimensional geometric data and impact acoustic data; The analysis module is used to extract scrap steel features from the visual data, the three-dimensional geometric data and the impact acoustic data to obtain scrap steel feature parameters; The association module is used to perform hierarchical classification and attribute analysis on the scrap steel to be recycled based on the scrap steel characteristic parameters, and to obtain classification results and attribute prediction results. The processing module is used to perform recycling analysis on the scrap steel to be recycled based on the classification results and the attribute prediction results, and to obtain a scrap steel recycling report.

[0012] This invention also provides a multimodal data analysis system for scrap steel recycling, comprising: Memory, used to store programs; A processor is used to execute the program to implement the various steps of the multimodal data analysis method for scrap steel recycling described in any of the above-mentioned embodiments.

[0013] The present invention also provides a storage medium storing computer instructions for causing a computer to perform any of the methods described above.

[0014] The present invention provides a multimodal data analysis method, apparatus, system, and storage medium for scrap steel recycling, which has the following beneficial effects: By simultaneously acquiring visual, 3D geometric, and impact acoustic data, a comprehensive digital characterization system for scrap steel was constructed, overcoming the limitations of insufficient information from a single sensor. Through automated feature extraction and parametric integration, multi-source heterogeneous data was transformed into standard feature parameters, enabling an objective quantitative description of the surface morphology, internal structure, and material properties of scrap steel, significantly improving the consistency and repeatability of identification. Based on these feature parameters, hierarchical material classification and simultaneous prediction of compositional properties were performed, achieving an automated process from coarse sorting to fine identification, greatly improving processing efficiency. By automatically linking the analysis results with recycling strategies and value standards, a structured recycling decision report was generated, realizing a closed loop from identification and assessment to decision-making. Attached Figure Description

[0015] Figure 1This is a flowchart of a multimodal data analysis method for scrap steel recycling provided by the present invention; Figure 2 This is a structural diagram of a multimodal data analysis device for scrap steel recycling provided by the present invention; Figure 3 This is a structural diagram of a multimodal data analysis system for scrap steel recycling provided by the present invention.

[0016] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0019] Reference Figure 1 As shown, this invention provides a multimodal data analysis method for scrap steel recycling, comprising: Step S1: Perform multimodal data acquisition on the scrap steel to be recycled to obtain visual data, three-dimensional geometric data and impact acoustic data; Specifically, visual data acquisition is achieved using a hyperspectral imager or a high-resolution industrial camera. Under specific lighting conditions, this device captures continuous image frames of the scrap steel surface, forming visual data containing spatial information and rich spectral reflectance information. Three-dimensional geometric data acquisition relies on a structured light 3D scanner. This device projects an coded grating pattern onto the scrap steel surface, and a matching camera captures the deformed pattern. Triangulation is used to calculate and generate three-dimensional geometric data centered on a point cloud coordinate set. This dataset accurately characterizes the physical dimensions, volume, and surface morphology of the scrap steel. Impact acoustic data acquisition is accomplished using a high-sensitivity acoustic sensor array. When the scrap steel collides naturally or under guided conditions with a trigger baffle of known material and structure during transport, the sensor array records the transient vibration sound wave signal generated by the collision, forming impact acoustic data. To ensure the spatiotemporal consistency of the multi-source data, all sensors are synchronously activated by a unified hardware trigger signal, and each frame of data is timestamped identically. During implementation, if the visual data detects areas with severe glare or shadow occlusion, it will trigger the 3D scanner to perform a supplementary scan of that area to obtain supplementary point clouds to improve the 3D model.

[0020] Step S2: Extract scrap steel features from the visual data, the three-dimensional geometric data, and the impact acoustic data to obtain scrap steel feature parameters; Specifically, for visual data, a color distribution histogram is calculated to quantify the distribution of dominant hues and colors such as rust and coatings on the surface of scrap steel. Gradient operators are used to calculate the texture direction and intensity of each pixel in the image, generating a surface texture gradient sequence to describe microscopic morphologies such as surface roughness, scratches, or rolling textures. For three-dimensional geometric data, the spatial relationship between each point in the point cloud and its neighbors is traversed to calculate statistical distribution features such as local curvature and normal vectors. Boundary points where spatial distances abruptly change in the point cloud are identified, and these points are connected to form a two-dimensional contour polygon. The contour morphology parameters, such as the area-to-perimeter ratio of the polygon, are then calculated. These parameters collectively constitute a second set of features describing the overall shape and structural complexity of the scrap steel. For impact acoustic data, the processing mainly involves time-frequency analysis of the waveform signal, extracting its time-domain energy envelope to characterize the rate of impact energy decay, and obtaining its frequency domain peak distribution through a fast Fourier transform to identify the dominant frequency characteristics of the material's stimulated response. These acoustic features are closely related to the material's internal damping, hardness, and other mechanical properties. The first, second, and third feature sets extracted from the three modalities are concatenated and normalized to form a unified high-dimensional scrap steel feature parameter vector.

[0021] Step S3: Based on the characteristic parameters of the scrap steel, perform hierarchical classification and attribute analysis on the scrap steel to be recycled to obtain classification results and attribute prediction results; Specifically, based on a pre-set set of identification rules and an attribute mapping table, the hierarchical classification process employs a two-level progressive structure. The first-level classification rules, based on core physical indicators in the feature parameters, such as estimated overall density or main geometric shape, initially classify scrap steel into primary material categories such as heavy, light, or specific alloy series. This primary category serves as a filtering condition, activating the corresponding second-level classification rule set from the rule set. This refined classification rule set contains matching conditions, and by comparing patterns of subsets of feature parameter vectors strongly correlated with the material, the scrap steel is further identified as a specific material type. For example, the heavy category is subdivided into scrapped car panels, H-beams, or castings, forming the classification result. The attribute analysis process proceeds simultaneously with the refined classification, matching identical feature parameter vectors with a pre-set attribute mapping table. This mapping table defines the correspondence from specific combinations of feature values ​​to specific chemical compositions (such as carbon and manganese content ranges) or physical properties (such as hardness grades and coating conditions). The entries in the mapping table are traversed; when a feature parameter satisfies all the constraints of an entry, the scrap steel is determined to possess the attribute label specified by that entry. Finally, the system outputs in parallel the precise material type obtained from the fine classification, as well as the attribute prediction results about the composition and physical properties, consisting of a series of attribute labels.

[0022] Step S4: Based on the classification results and the attribute prediction results, perform recycling analysis on the scrap steel to be recycled to obtain a scrap steel recycling report.

[0023] Specifically, the decision-making logic is driven by invoking a pre-built recycling strategy library. The specific material type in the classification results is used as an index to search the recycling strategy library, matching the basic recycling processing path associated with that material (such as specified crushing, packaging, or sorting flow) and the unit weight benchmark value based on market conditions. The attribute prediction results are compared with the quality evaluation standards stored in the recycling strategy library for different materials and uses. For example, the predicted chromium content is compared with the furnace entry standards for stainless steel, and the surface coating status is compared with cleanliness requirements, thereby calculating quality adjustment coefficients item by item. Based on the preset value calculation relationship, the unit weight benchmark value, various quality adjustment coefficients, and the estimated weight of scrap steel volume calculated from 3D data are comprehensively calculated to obtain the comprehensive recycling value score of the individual scrap steel. According to the determined recycling processing path and value score range, a suitable structured report template is selected from the strategy library, and the classification results, key attribute parameters, calculated weight, value score, and processing suggestions are automatically filled into the corresponding fields of the template to generate a scrap steel recycling report.

[0024] This invention provides a multimodal data analysis method for scrap steel recycling. By simultaneously acquiring visual, 3D geometric, and impact acoustic data, a comprehensive digital characterization system for scrap steel is constructed, overcoming the limitations of insufficient information from a single sensor. Through automated feature extraction and parametric integration, multi-source heterogeneous data is transformed into standard feature parameters, enabling an objective quantitative description of the surface morphology, internal structure, and material properties of scrap steel, significantly improving the consistency and repeatability of identification. Based on the feature parameters, hierarchical material classification and simultaneous prediction of component properties are performed, realizing an automated process from coarse sorting to fine identification, greatly improving processing efficiency. By automatically linking the analysis results with recycling strategies and value standards, a structured recycling decision report is generated, achieving a closed loop from identification and assessment to decision-making.

[0025] In one embodiment, the multimodal data acquisition of the scrap steel to be recycled to obtain visual data, three-dimensional geometric data, and impact acoustic data includes: The surface image sequence of the scrap steel to be recycled is acquired by a visual acquisition device, and the surface image sequence is combined into the visual data. The surface point cloud coordinates of the scrap steel to be recycled are obtained by a 3D scanning device, and the 3D geometric data is constructed based on the surface point cloud coordinates. Among them, surface point cloud coordinates refer to a set of coordinates (X, Y, Z) obtained by a 3D scanning device, representing the geometric position of the scrap steel surface in three-dimensional space. Each coordinate point represents an actual spatial position on the scrap steel surface.

[0026] Specifically, when the same synchronous trigger signal activates visual acquisition, the projection module of the 3D scanning device projects a set of grating stripe patterns with unique coded features onto the scrap steel surface. Simultaneously, the camera module on the other side captures the stripe images deformed by the height variations of the scrap steel surface. The captured deformed stripe images are decoded and phase-calculated. Combining precisely calibrated internal system parameters (such as the relative position of the projector and camera, and focal length) and external parameters, a triangulation algorithm maps each pixel in the image to 3D space, calculating its precise (X, Y, Z) coordinates in the world coordinate system. This process continues as the scrap steel passes through, generating a dense set of surface point cloud coordinates covering the upper surface and part of the sides of the scrap steel. This raw point cloud is preprocessed, including removing outliers caused by noise and downsampling the point cloud to balance detail and data volume. The preprocessed point cloud is then connected into a continuous triangular mesh surface using a triangulation algorithm. This mesh surface, composed of millions of triangles, together with its vertices (i.e., point cloud coordinates) and surface topology, constitutes three-dimensional geometric data that can characterize the precise shape, size, and volume of scrap steel.

[0027] Acoustic sensors are used to acquire the acoustic signal sequence generated when a standard impact force is applied to the scrap steel to be recycled, and the acoustic signal sequence is recorded as the impact acoustic data. Specifically, impact acoustic data refers to a structured dataset formed by standardizing and encapsulating the original acoustic signal sequence. This data includes the original waveforms from multiple channels, as well as metadata such as trigger time, sampling rate, sensor gain, and ambient background noise samples.

[0028] When an occluded area is detected in the surface image sequence, the three-dimensional scanning device is controlled to perform supplementary scanning of the occluded area, and the supplementary point cloud coordinates obtained from the supplementary scanning are merged into the surface point cloud coordinate sequence, and the three-dimensional geometric data is updated.

[0029] The method provided in this embodiment acquires surface image sequences through a visual acquisition device, providing a data source of scrap steel appearance features containing rich spectral and texture information for subsequent analysis, overcoming the limitations of insufficient information dimensions in single visible light imaging. Simultaneously, a 3D scanning device acquires surface point cloud coordinates and constructs 3D geometric data, accurately quantifying the physical dimensions, volume, and morphological characteristics of the scrap steel, providing a reliable geometric basis for morphology-based automatic classification and weight estimation. Finally, an acoustic sensor acquires sound wave signal sequences under standard impact force, transforming the internal mechanical properties of the scrap steel into analyzable digital acoustic features, enabling non-destructive testing of non-visual attributes such as material hardness and internal structural consistency.

[0030] In one embodiment, the step of extracting scrap steel feature parameters from the visual data, the three-dimensional geometric data, and the impact acoustic data includes: The color distribution histogram and surface texture gradient sequence are extracted and integrated from the visual data to obtain the first feature set; Specifically, the hyperspectral image cube is subjected to dimensionality reduction and enhancement. Principal component analysis (PCA) is used to compress hundreds of spectral bands, selecting the top few principal component images containing the majority of information. For color feature extraction, specific narrow-band images related to the absorption peaks of substances such as iron oxides, paint, and zinc layers are selected. On these images, a global color / grayscale histogram is calculated, and a set of statistical moments describing the distribution pattern is extracted as quantization features. For texture feature extraction, this is performed on the principal component images with the clearest spatial details. By traversing each pixel of the image, the brightness difference in the horizontal and vertical directions is calculated, thus obtaining the gradient magnitude and direction of that pixel. After completing the full image traversal, two matrices of the same size as the source image are generated, storing the gradient magnitude and direction of each pixel, respectively. Statistical analysis is performed on the entire magnitude and direction matrices, calculating their mean, standard deviation, and consistency. Multiple statistical features extracted from the color histogram and multiple statistical features calculated from the texture gradient are normalized to eliminate the influence of dimensions, and then combined into a coherent one-dimensional numerical array according to a predefined index order. This array is the first feature set, which comprehensively encodes the spectral reflectance characteristics and spatial texture structure information of the scrap steel surface.

[0031] Extract the point cloud coordinates from the three-dimensional geometric data, calculate the contour morphology parameters of the scrap steel to be recycled based on the point cloud coordinates, and integrate all the contour geometric parameters to obtain the second feature set; The time-domain energy envelope and frequency-domain peak distribution are extracted and integrated from the impact acoustic data to obtain a third feature set; Specifically, for time-domain analysis, the entire signal is divided into a series of short-time overlapping time windows. The sum of the squares of the amplitudes of all sampling points within each time window is calculated as the short-time energy of that window. The energy values ​​of all windows are arranged in chronological order and smoothly connected to form the time-domain energy envelope curve. From this envelope curve, dynamic parameters such as the time it takes for the envelope to reach its maximum value, the maximum energy value, the time required for the peak to decay to a certain proportion (e.g., 50%) (attenuation constant), and the slope of the fitted curve for the entire attenuation segment are extracted. For frequency-domain analysis, a typical segment of the time-domain envelope with stable energy that represents free decay is extracted. Windowing and Fast Fourier Transform are applied to this segment of signal to transform it into the frequency domain, obtaining the amplitude spectrum. In the amplitude spectrum, all peak points that satisfy the condition of amplitude exceeding a preset threshold and being local maxima are identified. The frequency and amplitude values ​​corresponding to the first few most important peaks are recorded, and statistical measures such as the interval between peak frequencies, the ratio of the main peak frequency to the amplitude, and the concentration of peak frequencies are calculated. The dynamic parameter set extracted from the time domain envelope and the spectral parameter set extracted from the frequency domain peak values ​​are normalized separately. Then, following a predetermined order of time domain first, then frequency domain, and global parameters first, then detailed parameters, they are integrated into a one-dimensional numerical array. This array is the third feature set, which encodes the unique vibration decay behavior and spectral fingerprint exhibited by scrap steel under impact excitation.

[0032] The first feature set, the second feature set, and the third feature set are merged to obtain the scrap steel feature parameters.

[0033] The method provided in this embodiment extracts and integrates color distribution histograms and surface texture gradient sequences from visual data, transforming the spectral reflectance characteristics and microstructure of scrap steel surfaces into a standardized first feature set. By processing point cloud coordinates in three-dimensional geometric data to calculate contour morphology parameters and integrating them into a second feature set, the macroscopic physical dimensions, volume, shape complexity, and other geometric properties of scrap steel are refined into a set of computable mathematical descriptions. By extracting and integrating temporal energy envelopes and frequency peak distributions from impact acoustic data to form a third feature set, the vibration attenuation dynamics and resonance spectrum modes of the material after impact are transformed into digital features, achieving non-destructive detection and indirect quantification of non-visual attributes such as internal hardness, damping characteristics, and structural integrity.

[0034] In one embodiment, the step of extracting the point cloud coordinate set from the three-dimensional geometric data, calculating the contour morphology parameters of the scrap steel to be recycled based on the point cloud coordinate set, and integrating all the contour geometric parameters to obtain a second feature set includes: Traverse the point cloud coordinates in the point cloud coordinate set and calculate the spatial distance between each point cloud coordinate and all its adjacent coordinates. Each spatial distance value is compared with a preset distance threshold. When a spatial distance value is greater than the preset distance threshold, both point cloud coordinate points corresponding to the spatial distance value are marked as contour boundary points. Connect all the boundary points of the contour to obtain a two-dimensional contour polygon; Specifically, all contour boundary points are projected onto a selected 2D reference plane using a projection matrix. A horizontal plane perpendicular to the direction of gravity is chosen, resulting in a set of 2D points. Projection eliminates Z-axis height information, focusing on the object's top-view contour. The projected 2D point set is then processed using computational geometry algorithms. The convex hull of these points is calculated, which is the smallest convex polygon containing all points. A radius parameter, alpha, is introduced to generate a shape from the point set that includes concavities and more closely approximates the actual point distribution. The final output is an ordered sequence of vertices, defining the connection paths of the polygon edges. Finally, this initially generated polygon undergoes post-processing, such as applying the Douglas-Puk algorithm to simplify the polygon edges, removing redundant collinear or closely spaced vertices, reducing the data volume while preserving the basic shape characteristics. The resulting ordered list of vertices and the closed regions they define constitute the 2D contour polygon.

[0035] The ratio between the area and perimeter of the two-dimensional contour polygon is calculated to obtain the contour compactness parameter. The contour compactness parameter is then integrated with the contour boundary points to obtain the contour geometric parameters. Specifically, the process iterates through all vertices of the 2D polygon outline, multiplying the X-coordinate of each vertex by the Y-coordinate of the next vertex in a fixed order, and vice versa. This alternating addition and subtraction is performed according to the formula rules, and the area is obtained by taking half of the absolute value. Perimeter calculation is performed simultaneously. For each edge of the polygon, for the edge connecting vertex i and vertex j, the distance between the two points in the 2D plane is calculated by squaring the difference in X-coordinates and the difference in Y-coordinates, summing them, and then taking the square root. This process is repeated for all edges, and the calculated edge lengths are accumulated to obtain the perimeter value. After obtaining the two basic quantities, area and perimeter, compactness calculation is performed. The area value is divided by the square of the perimeter value to obtain the contour compactness parameter. This parameter is a shape factor. At the same time, some statistical information is extracted from the original contour boundary point subset as supplementary information. For example, the standard deviation of the Z-coordinate (height) of all boundary points in 3D space is calculated to assess the height variation on the contour, or the total number of boundary points is counted to reflect the richness of detail in the contour. Finally, the core contour compactness parameters are combined with these supplementary boundary point statistical features according to a predetermined data structure (such as a one-dimensional array or a feature dictionary) to form the final set of contour geometric parameters.

[0036] Integrate all the aforementioned contour morphology parameters to obtain the second feature set.

[0037] The method provided in this embodiment accurately quantifies the local geometric continuity of the scrap steel surface by traversing the point cloud coordinate set and calculating the spatial distance to all adjacent points, providing an objective metric based on spatial distribution density for reliable identification of real physical edges. By comparing each spatial distance value with a preset threshold and marking boundary points, the discrete data points implicit in the 3D point cloud, representing abrupt changes in shape, are accurately transformed into explicit contour boundary indicators, effectively distinguishing between continuous surfaces and contour edges. By connecting all marked contour boundary points, a 2D contour polygon is generated, realizing the reconstruction from discrete 3D boundary points to continuous 2D geometry. By calculating the ratio of the area to the perimeter of the 2D contour polygon, a contour compactness parameter is obtained, refining complex shape information into a dimensionless index characterizing its filling efficiency and regularity. By integrating the compactness parameter and boundary point information into contour geometric parameters, and finally aggregating all contour morphology parameters to form a second feature set, a multi-level digital description of the macroscopic shape of scrap steel from local edges to global morphology is constructed.

[0038] In one embodiment, the hierarchical classification and attribute analysis of the scrap steel to be recycled based on the scrap steel characteristic parameters to obtain classification results and attribute prediction results includes: The scrap steel feature parameters are identified according to the first-level classification rules of the preset identification rule set to obtain the primary material category. The first-level classification rules refer to the rules in the identification rule set specifically used for the highest-level, coarsest material classification. The design goal of the first-level classification rules is to use a few of the most distinguishing key features to categorize scrap steel into several non-overlapping major categories, such as "heavy scrap steel," "light scrap steel," "stainless steel," or "non-ferrous metals."

[0039] Specifically, a pre-defined set of identification rules is loaded, and the first-level classification rule is located. The first-level classification rule is represented by a series of conditional statements. For example, one rule might state: if the overall estimated density of the feature parameters is greater than threshold A and the profile compactness is less than threshold B, then it is classified as heavy scrap steel; another rule might state: if the average reflectance of a specific spectrum is higher than threshold C, then it is preferentially classified as stainless steel. The relevant dimensional values ​​in the scrap steel feature parameter vector are sequentially evaluated and compared with the thresholds or ranges set in these rules. When a feature parameter satisfies all the conditions set for a rule, that rule is triggered, and the category label pointed to by its conclusion is determined as the primary material category. In some designs, multiple rules may be partially satisfied; in this case, the final category may be determined by the priority ranking of the rules or additional arbitration logic (such as calculating the sum of the weights of the rules that satisfy the conditions). Once determined, the primary material category is output and fixed.

[0040] Based on the primary material category, the corresponding secondary classification rule is selected from the preset identification rule set. The secondary material identification is performed on the scrap steel feature parameters and the primary material category according to the secondary classification rule to obtain the classification result. Specifically, the second-level classification rules refer to a subset of rules in the identification rule set that are associated with a specific primary material category and are used for further fine-tuning within that category. For example, the second-level rules associated with the heavy scrap steel category are responsible for distinguishing between plates, profiles, castings, and scrap automotive materials; while the rules associated with stainless steel may distinguish between austenitic, ferritic, and other series.

[0041] Specifically, a subset of second-level classification rules specific to the primary category is retrieved and loaded. The secondary material identification process is initiated. Compared to the first level, the second-level rules examine more and more refined feature dimensions. For example, for heavy scrap steel, the rules comprehensively consider the attenuation constant from acoustic features (related to carbon content), the surface area-to-volume ratio from geometric features (related to shape), and the intensity of specific oxidation peaks from spectral features. The scrap steel feature parameter vector is input into the activated subset of second-level rules, and logical matching is performed again. The matching process traverses all relevant rules, aiming to find the rule entry that most accurately matches the current combination of scrap steel features. When a match is successful, the specific material type pointed to by that rule is determined as a candidate result. In some implementations, multiple rules are allowed to be satisfied with different confidence levels, outputting a primary type and several alternative types. Finally, the specific material type determined as the most reliable or unique match is officially output as the classification result.

[0042] The scrap steel feature parameters are labeled based on a pre-set attribute mapping table to obtain attribute prediction labels. The attribute prediction label is integrated with the attribute prediction label to obtain the attribute prediction result.

[0043] The method provided in this embodiment rapidly matches and judges the characteristic parameters of scrap steel by applying first-level classification rules from a preset identification rule set, classifying scrap steel into a few non-overlapping primary material categories, thus achieving early and effective diversion of the analysis path. Based on the primary material categories, it dynamically filters from the rule set and applies corresponding second-level classification rules for secondary material identification, ensuring the high specificity of the fine-grained classification logic and avoiding misjudgments caused by interfering features between different major material categories. By performing label recognition of scrap steel characteristic parameters based on a preset attribute mapping table, independent of the classification process, the prediction of multi-dimensional attributes can be performed in parallel.

[0044] In one embodiment, the scrap steel feature parameters are labeled based on a preset attribute mapping table to obtain attribute prediction labels, including: Traverse each mapping rule in the attribute mapping table, wherein the mapping rule includes rule feature conditions and target attribute labels; The scrap steel feature parameters are matched sequentially with each of the mapping rules. When it is detected that the scrap steel feature parameters satisfy the rule feature conditions of the current mapping rule, the corresponding target attribute label is marked as a candidate label. Specifically, following a strict sequential matching logic, starting with the first mapping rule, the program reads its pre-parsed set of rule feature conditions. For each condition unit in this set, the program performs the following operations: Based on the feature dimension index specified in the condition unit, it locates and extracts the corresponding specific value from the scrap steel feature parameter vector. This actual value is then logically compared with the comparison value or value range defined in the condition unit using specified operators (such as "greater than", "within the range", "equal to") to obtain a judgment result. After completing the independent judgment of all condition units under a single rule, a logical AND operation is performed on these judgment results. Only when the comparison results of all condition units are true is it determined that the current scrap steel feature parameter satisfies all rule feature conditions of this mapping rule. Once the conditions are met, one or more associated target attribute tags are extracted from this rule. Each extracted tag is marked as a candidate tag, and the source rule identifier that generated this candidate tag is recorded. Regardless of whether the match is successful or not, after the current rule is processed, it automatically moves to the next mapping rule in the queue, repeating the complete matching and judgment process described above.

[0045] When there are multiple candidate labels, all candidate labels are sorted by priority according to the attribute mapping table, and the candidate label with the highest priority is selected as the attribute prediction label. If no candidate labels are generated after the traversal is completed, the predefined default attribute label will be output as the attribute prediction label.

[0046] The method provided in this embodiment ensures exhaustive access to the pre-set knowledge base by traversing each mapping rule in the attribute mapping table. This provides a foundation for comprehensively matching various attribute patterns that scrap steel feature parameters may conform to, avoiding omissions in attribute recognition. By precisely matching the scrap steel feature parameters with the rule feature conditions of each mapping rule one by one, candidate labels are activated only when all constraints are met. When multiple candidate labels are generated, they are sorted according to the preset priority of the attribute mapping table, and the highest priority is selected as the output. This mechanism effectively resolves inference conflicts that may arise between different rules. If no candidate labels are generated after traversal, a predefined default attribute label is output. This fault-tolerant design ensures that the attribute recognition process can produce a definite output under any circumstances.

[0047] In one embodiment, the step of constructing a scrap steel recycling report by performing recycling analysis on the scrap steel to be recycled based on the classification results and the attribute prediction results includes: The classification results are matched with a preset recycling strategy library to obtain a basic recycling processing path and a unit weight benchmark value. The basic recycling process refers to the recommended follow-up treatment procedures and destinations for specific identified material types. For example, for low-carbon steel plates, the process might involve shearing, packaging, and transporting to a converter for steelmaking; for copper-containing motors, the process might involve dismantling, copper component sorting, and non-ferrous metal recycling.

[0048] Benchmark value per unit weight: This refers to the reference price per unit weight (e.g., per ton) corresponding to the material type in the classification results, mapped from the recycling strategy database. This value is a basic figure that reflects the fair market value of this type of material under standard quality conditions, and does not yet take into account the specific differences in composition and quality of individual scrap steel pieces.

[0049] The attribute prediction results are evaluated based on the recycling evaluation criteria in the recycling strategy library to obtain a quality evaluation coefficient. The recycling value is calculated based on the unit weight benchmark value and the quality evaluation coefficient to obtain a recycling value score; Specifically, the base value per unit weight is used as the calculation base. If the quality assessment coefficient is a single composite coefficient, the base value is directly multiplied by that coefficient. If multiple independent quality assessment coefficients exist (e.g., one for composition, one for surface condition), these coefficients are multiplied sequentially by the base value. This multiplication operation simulates the cumulative adjustment effect of various quality attributes on the base value. The adjusted unit value, after quality adjustment, is multiplied by the estimated weight of the scrap steel. The estimated weight is obtained by multiplying the volume calculated from three-dimensional geometric data by a pre-stored average density estimate based on the material type. This multiplication extends the unit value to the entire physical entity of the scrap steel, yielding a preliminary total value. Depending on the strategy library settings, fixed processing cost parameters (such as estimated handling and cutting costs) are added or subtracted from this result, or an additional coefficient related to the recycling process is applied. Finally, the value processed by all the calculation rules is determined as the recycling value score of the scrap steel.

[0050] Based on the basic recycling processing path and the recycling value score, the corresponding report template is extracted from the recycling strategy library. The classification results, attribute prediction results, recycling value score, and basic recycling processing path are filled into the report template to obtain the scrap steel recycling report.

[0051] The method provided in this embodiment directly obtains the corresponding basic recycling processing path and unit weight benchmark value by matching the classification results with a preset recycling strategy library. This achieves seamless integration from material identification to specific processing suggestions and economic benchmarks, significantly improving the targeting and response speed of recycling decisions. Based on the evaluation criteria in the recycling strategy library, the attribute prediction results are quality-evaluated and adjustment coefficients are generated, transforming qualitative physicochemical attributes into quantitative value influencing factors, enabling the final pricing to objectively reflect the intrinsic quality differences of individual scrap steel. A recycling value score is derived by comprehensively calculating the unit weight benchmark value, quality evaluation coefficient, and estimated weight, integrating the three elements of material, attributes, and physical scale into an authoritative economic value indicator. Based on the processing path and value score, a standardized report template is dynamically extracted and populated, automatically generating a structured scrap steel recycling report. This report integrates all analytical conclusions, value assessments, and processing guidance, achieving a digital closed loop for the entire recycling analysis process and a high degree of standardization in information output.

[0052] Reference Figure 2 As shown, the present invention also provides a multimodal data analysis device for scrap steel recycling, applied to the multimodal data analysis method for scrap steel recycling described in any one of the above-mentioned methods, comprising: The acquisition module is used to acquire multimodal data of the scrap steel to be recycled, and obtain visual data, three-dimensional geometric data and impact acoustic data; The analysis module is used to extract scrap steel features from the visual data, the three-dimensional geometric data and the impact acoustic data to obtain scrap steel feature parameters; The association module is used to perform hierarchical classification and attribute analysis on the scrap steel to be recycled based on the scrap steel characteristic parameters, and to obtain classification results and attribute prediction results. The processing module is used to perform recycling analysis on the scrap steel to be recycled based on the classification results and the attribute prediction results, and to obtain a scrap steel recycling report.

[0053] Reference Figure 3 As shown, the present invention also provides a multimodal data analysis system for scrap steel recycling, comprising: Memory, used to store programs; A processor is used to execute the program to implement the various steps of the multimodal data analysis method for scrap steel recycling described in any of the above-mentioned embodiments.

[0054] The present invention also provides a storage medium storing computer instructions for causing a computer to perform any of the methods described above.

[0055] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the system and each module described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0056] In this embodiment, the processor and memory can be connected via a bus or other means. The memory may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive. The processor may be a general-purpose processor, such as a central processing unit, digital signal processor, application-specific integrated circuit, or one or more integrated circuits configured to implement embodiments of the present invention.

[0057] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A multimodal data analysis method for scrap steel recycling, characterized in that, include: Multimodal data acquisition was performed on the scrap steel to be recycled to obtain visual data, three-dimensional geometric data and impact acoustic data; The visual data, the three-dimensional geometric data, and the impact acoustic data are used to extract scrap steel features to obtain scrap steel feature parameters; Based on the characteristic parameters of the scrap steel, the scrap steel to be recycled is classified hierarchically and analyzed for attributes to obtain classification results and attribute prediction results. Based on the classification results and the attribute prediction results, a recycling analysis is performed on the scrap steel to be recycled to obtain a scrap steel recycling report.

2. The multimodal data analysis method for scrap steel recycling according to claim 1, characterized in that, The process involves multimodal data acquisition of the scrap steel to be recycled, yielding visual data, three-dimensional geometric data, and impact acoustic data, including: The surface image sequence of the scrap steel to be recycled is acquired by a visual acquisition device, and the surface image sequence is combined into the visual data. The surface point cloud coordinates of the scrap steel to be recycled are obtained by a 3D scanning device, and the 3D geometric data is constructed based on the surface point cloud coordinates. Acoustic sensors are used to acquire the acoustic signal sequence generated when a standard impact force is applied to the scrap steel to be recycled, and the acoustic signal sequence is recorded as the impact acoustic data. When an occluded area is detected in the surface image sequence, the three-dimensional scanning device is controlled to perform supplementary scanning of the occluded area, and the supplementary point cloud coordinates obtained from the supplementary scanning are merged into the surface point cloud coordinate sequence, and the three-dimensional geometric data is updated.

3. The multimodal data analysis method for scrap steel recycling according to claim 1, characterized in that, The step of extracting scrap steel features from the visual data, the three-dimensional geometric data, and the impact acoustic data to obtain scrap steel feature parameters includes: The color distribution histogram and surface texture gradient sequence are extracted and integrated from the visual data to obtain the first feature set; Extract the point cloud coordinates from the three-dimensional geometric data, calculate the contour morphology parameters of the scrap steel to be recycled based on the point cloud coordinates, and integrate all the contour geometric parameters to obtain the second feature set; The time-domain energy envelope and frequency-domain peak distribution are extracted and integrated from the impact acoustic data to obtain a third feature set; The first feature set, the second feature set, and the third feature set are merged to obtain the scrap steel feature parameters.

4. The multimodal data analysis method for scrap steel recycling according to claim 3, characterized in that, The point cloud coordinate set is extracted from the three-dimensional geometric data, and the contour morphology parameters of the scrap steel to be recycled are calculated based on the point cloud coordinate set. All the contour geometric parameters are then integrated to obtain a second feature set, including: Traverse the point cloud coordinates in the point cloud coordinate set and calculate the spatial distance between each point cloud coordinate and all its adjacent coordinates. Each spatial distance value is compared with a preset distance threshold. When a spatial distance value is greater than the preset distance threshold, both point cloud coordinate points corresponding to the spatial distance value are marked as contour boundary points. Connect all the boundary points of the contour to obtain a two-dimensional contour polygon; The ratio between the area and perimeter of the two-dimensional contour polygon is calculated to obtain the contour compactness parameter. The contour compactness parameter is then integrated with the contour boundary points to obtain the contour geometric parameters. Integrate all the aforementioned contour morphology parameters to obtain the second feature set.

5. The multimodal data analysis method for scrap steel recycling according to claim 1, characterized in that, The process involves hierarchical classification and attribute analysis of the scrap steel to be recycled based on the characteristic parameters of the scrap steel, yielding classification results and attribute prediction results, including: The scrap steel feature parameters are identified according to the first-level classification rules of the preset identification rule set to obtain the primary material category. Based on the primary material category, the corresponding secondary classification rule is selected from the preset identification rule set. The secondary material identification is performed on the scrap steel feature parameters and the primary material category according to the secondary classification rule to obtain the classification result. The scrap steel feature parameters are labeled based on a pre-set attribute mapping table to obtain attribute prediction labels. The attribute prediction label is integrated with the attribute prediction label to obtain the attribute prediction result.

6. The multimodal data analysis method for scrap steel recycling according to claim 5, characterized in that, Based on a pre-set attribute mapping table, the characteristic parameters of the scrap steel are labeled to obtain attribute prediction labels, including: Traverse each mapping rule in the attribute mapping table, wherein the mapping rule includes rule feature conditions and target attribute labels; The scrap steel feature parameters are matched sequentially with each of the mapping rules. When it is detected that the scrap steel feature parameters satisfy the rule feature conditions of the current mapping rule, the corresponding target attribute label is marked as a candidate label. When there are multiple candidate labels, all candidate labels are sorted by priority according to the attribute mapping table, and the candidate label with the highest priority is selected as the attribute prediction label. If no candidate labels are generated after the traversal is completed, the predefined default attribute label will be output as the attribute prediction label.

7. The multimodal data analysis method for scrap steel recycling according to claim 1, characterized in that, The process involves performing a recycling analysis on the scrap steel to be recycled based on the classification results and the attribute prediction results, resulting in a scrap steel recycling report, including: The classification results are matched with a preset recycling strategy library to obtain a basic recycling processing path and a unit weight benchmark value. The attribute prediction results are evaluated based on the recycling evaluation criteria in the recycling strategy library to obtain a quality evaluation coefficient. The recycling value is calculated based on the unit weight benchmark value and the quality evaluation coefficient to obtain a recycling value score; Based on the basic recycling processing path and the recycling value score, the corresponding report template is extracted from the recycling strategy library. The classification results, attribute prediction results, recycling value score, and basic recycling processing path are filled into the report template to obtain the scrap steel recycling report.

8. A multimodal data analysis device for scrap steel recycling, characterized in that, The multimodal data analysis method for scrap steel recycling according to any one of claims 1-7 includes: The acquisition module is used to acquire multimodal data of the scrap steel to be recycled, and obtain visual data, three-dimensional geometric data and impact acoustic data; The analysis module is used to extract scrap steel features from the visual data, the three-dimensional geometric data and the impact acoustic data to obtain scrap steel feature parameters; The association module is used to perform hierarchical classification and attribute analysis on the scrap steel to be recycled based on the scrap steel characteristic parameters, and to obtain classification results and attribute prediction results. The processing module is used to perform recycling analysis on the scrap steel to be recycled based on the classification results and the attribute prediction results, and to obtain a scrap steel recycling report.

9. A multimodal data analysis system for scrap steel recycling, characterized in that, include: Memory, used to store programs; A processor is configured to execute the program to implement the various steps of the multimodal data analysis method for scrap steel recycling as described in any one of claims 1-7.

10. A storage medium, characterized in that, The computer contains computer instructions for causing the computer to perform the method according to any one of claims 1 to 7.

Images