A waste pretreatment material classification method and system based on visual detection

Abstract

This application relates to the field of waste sorting technology, specifically disclosing a method and system for classifying waste pretreatment materials based on visual inspection. The method includes the following steps: acquiring the three-dimensional information and surface appearance information of the material; adjusting the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's posture, and acquiring multiple consecutive frames of spectral images of the material; performing spatial calibration on the multiple consecutive frames of spectral images based on the three-dimensional spatial boundary and posture; performing time-series fusion processing on the spatially calibrated multiple consecutive frames of spectral images to obtain stable spectral feature data of the material; identifying the material's material type based on the stable spectral feature data, three-dimensional information, and surface appearance information; and generating sorting instructions for the material based on the material type, three-dimensional information, and surface appearance information. This application can comprehensively assess the recycling value of materials, improving resource recovery rate and economic value.

Description

Technical Field

[0001] This application relates to the field of waste sorting technology, and in particular to a method and system for classifying waste pretreatment materials based on visual detection. Background Technology

[0002] In modern industrial production and environmental protection, efficient and accurate automated sorting of mixed waste is crucial for achieving resource recycling. Traditional sorting methods struggle to handle complex and varied materials, leading to the development of automated systems based on visual inspection. Initially, these systems relied on analyzing the color, shape, and other features of two-dimensional images to identify common recyclables, such as PET bottles and cardboard boxes, which performed reasonably well under ideal conditions. However, relying solely on macroscopic visual features cannot distinguish between materials that look similar but are made of vastly different materials, such as transparent PET, PP, and PVC containers. This necessitated a technological upgrade to incorporate near-infrared spectroscopy. This technology identifies chemical components by analyzing the unique near-infrared absorption and reflection spectra of materials, significantly improving the ability to differentiate between similar-looking materials.

[0003] However, near-infrared spectroscopy technology still faces multiple severe challenges in actual waste sorting environments. First, dark or black plastics strongly absorb near-infrared light, resulting in weak signals that are difficult to identify. Second, contaminants such as oil, dirt, and moisture adhering to the material surface can severely interfere with and mask its intrinsic spectrum, leading to misjudgments. Furthermore, thin, easily shaken plastic films or packaging bags on high-speed conveyor belts produce weak and unstable reflected signals, making accurate data collection difficult. More complexly, for multi-layered composite materials such as milk cartons and potato chip bags, the spectrum often only identifies the outermost layer and cannot reflect the actual internal composition, leading to decreased sorting purity. Simultaneously, materials that have undergone aging and degradation will experience spectral shifts, increasing the deviation from matching standard spectral libraries. Ultimately, the core objective of the system is not only to identify the "material" but also to achieve high-purity recycling. Even if a material is accurately identified as PET, if it is severely contaminated or tightly bound to foreign labels or bottle caps and difficult to separate, it should not enter the high-purity PET recycling stream. Summary of the Invention

[0004] This application proposes a visual detection-based waste pretreatment material classification method and system, aiming to solve the technical problems in existing waste sorting technologies, such as limited material identification accuracy, difficulty in dealing with complex and variable material states, susceptibility to contamination and interference from composite materials, and inability to comprehensively assess recyclability.

[0005] In a first aspect, this application provides a visual detection-based method for classifying waste pretreatment materials, used to sort materials on a waste sorting line, the method comprising the following steps: The three-dimensional information and surface appearance information of the material are obtained, wherein the three-dimensional information includes the three-dimensional spatial boundary, orientation and thickness of the material; Based on the orientation of the material, the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material is adjusted to optimize the incident and reception conditions of the spectral signals and acquire continuous multi-frame spectral images of the material. Based on the three-dimensional spatial boundary and the pose, spatial calibration is performed on the continuous multi-frame spectral images to align the pixel positions of image regions representing the same material in different frames. The material's stable spectral characteristic data are obtained by performing time-series fusion processing on a series of consecutive spectral images after spatial calibration. The material is identified based on the stable spectral characteristic data, the three-dimensional information, and the surface appearance information; Based on the material, the three-dimensional information, and the surface appearance information, a sorting instruction is generated for the material.

[0006] As some embodiments of this application, the steps of obtaining the three-dimensional information and surface appearance information of the material include: The surface appearance information of the material is acquired using a visible light camera, and the surface appearance information includes at least one of the material's color, texture, and surface contamination information; A structured light pattern is projected onto the surface of the material in the detection area located on the waste sorting line using a structured light projection device. The image acquisition device works synchronously with the structured light projection device to acquire the deformation image of the structured light pattern on the material surface. The deformation image is processed, and the three-dimensional coordinates of each point on the surface of the material are calculated by using the triangulation principle or the coded structured light decoding algorithm to generate the three-dimensional point cloud data of the material. Based on the three-dimensional point cloud data, the three-dimensional spatial boundary, orientation, and thickness of the material are determined as the three-dimensional information of the material.

[0007] As some embodiments of this application, the step of determining the three-dimensional spatial boundary, orientation, and thickness of the material based on the three-dimensional point cloud data includes: Applying a point cloud segmentation algorithm to the three-dimensional point cloud data will segment the point clouds corresponding to visually overlapping materials into independent sets of material point clouds; By analyzing the changes in the normal vectors in the point clouds of independent material point cloud sets, the three-dimensional spatial boundaries of the corresponding materials can be identified. By fitting the surface point cloud of an independent set of material point clouds, the normal vector of the surface of the set of material point clouds is calculated to determine the pose of the corresponding material. The thickness of the corresponding material is estimated by calculating the coordinate span of the point cloud in the depth direction of an independent set of material point clouds.

[0008] As some embodiments of this application, the step of spatially calibrating the continuous multi-frame spectral images based on the three-dimensional spatial boundary and the pose to align the pixel positions of image regions representing the same material in different frames includes: Based on the three-dimensional spatial boundary and orientation of the material, the reference pose of the material is determined; Based on the three-dimensional spatial boundary and the pose, calculate the pose transformation parameters of the image region representing the same material relative to the reference pose in each frame of the continuous multi-frame spectral images; Based on the pose transformation parameters, a geometric transformation is performed on the image region representing the same material in each frame of the continuous multi-frame spectral images, so that the pixel positions of the image regions representing the same material in different frames are aligned.

[0009] As some embodiments of this application, the step of performing time-series fusion processing on consecutive multi-frame spectral images after spatial calibration to obtain stable spectral characteristic data of the material includes: In the spatially calibrated multi-frame spectral images, the spectral data of pixels representing the same material are calculated by performing an arithmetic average or weighted average in the time dimension to obtain stable spectral feature data of the material.

[0010] As some embodiments of this application, the step of adjusting the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's orientation to optimize the incident and received conditions of the spectral signals, and acquiring continuous multi-frame spectral images of the material, further includes: Within a single exposure time window of acquiring multiple consecutive spectral images, a high-speed visible light camera array deployed at different angles is used to simultaneously capture the deformation process of the material surface and quantify the local deformation parameters of the material within the exposure time window. Before the step of performing time-series fusion processing on the spatially calibrated consecutive multi-frame spectral images to obtain stable spectral characteristic data of the material, the method further includes: Based on the local deformation parameters and the preset optical transmission model, predicted distortion data that quantitatively describes the interference of the current deformation state on the spectral signal is calculated. Based on the predicted distortion data, inverse compensation is performed on the spatially calibrated consecutive multi-frame spectral images to restore the intrinsic spectral characteristics of the material in its undeformed state, resulting in compensated consecutive multi-frame spectral images, which serve as the basis for calculating the stable spectral characteristic data of the material.

[0011] As some embodiments of this application, the step of identifying the material of the material based on the stable spectral feature data, the three-dimensional information, and the surface appearance information includes: The stable spectral feature data is matched with a preset standard material spectral library to calculate at least one candidate material and its corresponding spectral matching confidence level. Based on the three-dimensional information, the geometric integrity of the material and the spatial relationship between the material and the boundaries of other materials in the field of view are analyzed to obtain the physical state analysis results. Based on the surface appearance information, the contamination coverage of the material surface is analyzed to obtain the contamination level analysis results; Based on the preset material verification rules, and combined with the spectral matching confidence level, the physical state analysis results, and the pollution level analysis results, the candidate materials are confirmed or modified to determine the material of the material.

[0012] As some embodiments of this application, the step of adjusting the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's orientation to optimize the incident and received conditions of the spectral signals, and acquiring continuous multi-frame spectral images of the material, further includes: The material is irradiated with a polarized near-infrared light source in at least two orthogonal polarization directions, and the polarization spectral data of the material is collected. After the step of confirming or modifying the candidate materials according to preset material verification rules, combined with the spectral matching confidence level, the physical state analysis results, and the contamination level analysis results, and determining the material of the material, the method further includes: Based on the polarization spectral data, polarization spectral features characterizing the microscopic chemical structure of the material are extracted. The polarization spectral features are matched with a preset chemical residue polarization fingerprint database to identify whether chemical residues exist in the material and the type of chemical residues, which serves as the basis for adjusting the sorting instructions of the material in subsequent steps.

[0013] As some embodiments of this application, the step of generating a sorting instruction for the material based on the material, the three-dimensional information, and the surface appearance information includes: Based on the material, the physical state analysis results obtained from the three-dimensional information, and the pollution degree analysis results obtained from the surface appearance information, and in conjunction with the preset recycling purity standard, the recyclability of the material is comprehensively evaluated to obtain the evaluation result. Based on the evaluation results, a sorting instruction is generated to sort the material into the corresponding recycling category or waste category.

[0014] Secondly, this application also provides a visual detection-based waste pretreatment material sorting system for sorting materials on a waste sorting line, the system comprising: The information acquisition module is used to acquire the three-dimensional information and surface appearance information of the material, wherein the three-dimensional information includes the three-dimensional spatial boundary, orientation and thickness of the material; The parameter adjustment module is used to adjust the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's posture, so as to optimize the incident and reception conditions of the spectral signals and acquire continuous multi-frame spectral images of the material. The spectral acquisition and calibration module is used to perform spatial calibration on the continuous multi-frame spectral images based on the three-dimensional spatial boundary and the orientation, so that the pixel positions of the image regions representing the same material in different frames are aligned. The spectral fusion module is used to perform time-series fusion processing on multiple consecutive frames of spatially calibrated spectral images to obtain stable spectral feature data of the material. The material identification module is used to identify the material of the material based on the stable spectral feature data, the three-dimensional information, and the surface appearance information. The instruction generation module is used to generate sorting instructions for the material based on the material, the three-dimensional information, and the surface appearance information.

[0015] The technical solution according to the embodiments of this application has at least the following beneficial effects: To address the challenge of distinguishing materials with similar appearances, accurate chemical material identification is achieved through stable and high-quality intrinsic spectral characteristics. For dark materials with strong absorption and thin, flaky materials with weak or unstable signals due to significant vibration, proactive optimization of acquisition conditions and multi-frame spatiotemporal fusion technology significantly enhances the signal-to-noise ratio and stability. Facing the challenges of identifying surface contamination, moisture interference, and multilayer composite materials, the fusion of three-dimensional geometric information and surface appearance information effectively identifies contamination, determines material integrity and composite state, and avoids misjudgments. Finally, by introducing a comprehensive recyclability assessment mechanism based on three-dimensional information, appearance information, and recycling standards, it transcends simple material identification, enabling a comprehensive assessment of the material's recycling value and improving resource recovery rates and economic value.

[0016] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0017] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0018] Figure 1 This is a flowchart illustrating a visual detection-based waste pretreatment material classification method provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the architecture of a visual detection-based waste pretreatment material classification system provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] In modern industrial production and environmental protection, efficient and accurate automated sorting of mixed waste is a crucial step in achieving resource recycling and reducing environmental burden. Especially in the waste pretreatment stage, the ability to quickly identify and sort materials of different materials directly impacts the purity and economic efficiency of subsequent recycling processes. Traditional sorting methods often struggle to handle the complex and varied types and states of materials. Therefore, automated sorting systems based on visual inspection have emerged and improved sorting efficiency to some extent. However, these systems still face many challenges in practical applications, particularly when dealing with materials that are similar in appearance, complex in state, or affected by environmental factors. Their recognition accuracy and decision-making capabilities are often limited. For example, many wastes exhibit high similarity in appearance, making accurate differentiation difficult based solely on macroscopic visual features such as color, shape, and texture. Furthermore, oil stains, dirt, food residue, or high moisture content on the material surface, as well as vibrations, deformations, and overlaps that may occur during high-speed movement, can severely interfere with the acquisition and analysis of spectral signals, leading to a decrease in recognition accuracy. Furthermore, existing systems, when processing composite materials, often only identify the outermost or thickest layer, ignoring other internal materials. This results in sorting results that fail to meet the stringent purity requirements of downstream recycling companies. Ultimately, even if the system can identify the material relatively accurately, the ultimate goal of sorting is to achieve high-purity recycling. This means that the system needs to make a comprehensive judgment on the "recyclability" of the identified materials based on the stringent purity requirements of downstream recycling processes, rather than simply determining "what material it is."

[0022] In this regard, such as Figure 1 As shown, this application discloses a visual detection-based method for classifying waste pretreatment materials, used to sort materials on a waste sorting line. The method includes the following steps: S110, acquire the three-dimensional information and surface appearance information of the material, wherein the three-dimensional information includes the three-dimensional spatial boundary, orientation and thickness of the material; S120, according to the orientation of the material, adjust the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material, so as to optimize the incident and reception conditions of the spectral signals, and acquire continuous multi-frame spectral images of the material. S130, based on the three-dimensional spatial boundary and the pose, perform spatial calibration on the continuous multi-frame spectral images to align the pixel positions of image regions representing the same material in different frames; S140, perform time-series fusion processing on the continuous multi-frame spectral images after spatial calibration to obtain stable spectral characteristic data of the material; S150, based on the stable spectral characteristic data, the three-dimensional information, and the surface appearance information, the material of the material is identified; S160, Based on the material, the three-dimensional information, and the surface appearance information, generate a sorting instruction for the material.

[0023] "Three-dimensional information" refers to the geometric properties of a material in three-dimensional space. The three-dimensional spatial boundary defines the material's outline and extent in space; the orientation describes the material's position and tilt angle relative to a reference coordinate system; and the thickness refers to the material's dimensions in the depth direction. This three-dimensional information is crucial for accurately understanding the material's physical morphology. "Surface appearance information" encompasses the material's visual characteristics in the visible light band, such as color, texture, and surface contamination. This information helps distinguish materials with similar appearances and assess their surface cleanliness. "Spectral signal" refers to the material's absorption, reflection, or transmission characteristics of light within a preset wavelength range. Analyzing the spectral signal reveals the material's chemical composition and internal structure. "Continuous multi-frame spectral images" refers to multiple spectral images of the same material acquired continuously over a period of time, used to capture spectral changes in the material during dynamic processes. "Spatial calibration" refers to aligning the pixel positions representing the same material in different frames of images through geometric transformation, eliminating positional deviations caused by material movement or camera shake. "Time-series fusion processing" refers to integrating calibrated multi-frame spectral data to extract more stable and representative spectral features. "Stable spectral characteristic data" refers to spectral information that accurately reflects the essential chemical composition of materials after fusion processing, reducing the impact of noise and interference. "Sorting instructions" are specific operational instructions issued based on a comprehensive evaluation of the material's material composition, three-dimensional information, and surface appearance information, guiding the sorting equipment to sort the materials into the corresponding recycling or waste categories.

[0024] The three-dimensional information of materials can be obtained in various ways. For example, the surface of the material can be scanned by LiDAR to directly obtain the three-dimensional point cloud data, and then the three-dimensional spatial boundaries, orientation, and thickness of the material can be extracted using point cloud processing algorithms. Alternatively, a binocular vision system can be used to capture images of the material from different angles using two or more cameras, and then the depth information of the material can be calculated using stereo matching algorithms to construct a three-dimensional model and extract the required three-dimensional information. The surface appearance information of the material can be collected using a visible light camera. For example, a high-resolution color camera can be used to capture visible light images of the material, and the color and texture features of the material can be extracted from the images to identify whether there are obvious contamination areas on the surface.

[0025] For acquiring continuous multi-frame spectral images of materials, for example, if the material surface is tilted, the incident angle of the spectrometer can be adjusted to align with the normal direction of the material surface, maximizing the received intensity of the spectral signal. Alternatively, the illumination angle of the light source can be adjusted to avoid excessive specular reflection or shadows, thereby obtaining clearer and more accurate spectral data. When acquiring continuous multi-frame spectral images, a high-speed spectroscopic camera can be used to continuously capture multiple spectral images at a preset frame rate as the material passes through the detection area, ensuring that the spectral information of the material at different times can be captured.

[0026] Spatial calibration is performed on multiple consecutive frames of spectral images. For example, the precise contour of the material can be identified in each frame using its three-dimensional spatial boundary information. Then, based on the material's pose information, pose transformation parameters of the material relative to a reference pose are calculated for each frame. These parameters can include geometric transformations such as translation, rotation, and scaling. Next, using these pose transformation parameters, geometric transformations are performed on the image region of the material in each frame, such as through affine or perspective transformations, to calibrate the image regions of the material in all frames to a unified reference coordinate system, thereby aligning pixels representing the same material in different frames spatially.

[0027] Temporal series fusion processing is performed on consecutive frames of spatially calibrated spectral images. For example, for each pixel representing the same material in the calibrated image, its spectral data in different frames can be extracted. Then, these spectral data are fused along the temporal dimension. For instance, the arithmetic mean of the spectral data of the same pixel across multiple frames can be used to eliminate the effects of random noise and transient fluctuations, resulting in more stable spectral features. Alternatively, a weighted averaging method can be used, assigning different weights based on the quality or confidence level of each frame of the spectral image, thereby obtaining more representative and stable spectral feature data.

[0028] To identify the material type, for example, the obtained stable spectral feature data can be compared with a pre-defined standard material spectral library. Spectral matching algorithms (such as least squares method, correlation coefficient method, etc.) can be used to calculate the spectral similarity between the material and various known materials, thereby identifying the most likely material type. Simultaneously, combining the material's three-dimensional information, such as its thickness and shape integrity, can help determine the material's reliability. For example, if the spectral identification result indicates a thin film material, but the three-dimensional information shows a large material thickness, a reassessment may be necessary. Furthermore, surface appearance information, such as color, texture, and contamination status, can also serve as supplementary criteria. For example, if the material surface is severely contaminated, it may affect the accuracy of spectral identification; in this case, appearance information can be used for correction.

[0029] Finally, sorting instructions for the materials are generated. For example, after identifying the material's composition, the recyclability of the material can be comprehensively assessed based on preset recycling standards and downstream processing requirements, combined with the material's three-dimensional information (such as size and integrity) and surface appearance information (such as degree of contamination). For instance, a bottle identified as PET may not be suitable for entering the high-purity PET recycling stream if it is too small, severely damaged, or covered with difficult-to-remove paint. Based on this comprehensive assessment, corresponding sorting instructions will be generated, such as sorting the material to a PET recycling bin, a mixed plastic recycling bin, or a waste disposal bin.

[0030] The method proposed in this application achieves accurate classification through a collaborative and orderly multi-stage processing flow. The entire process embodies a closed-loop logic from "perception" to "correction," "fusion," and then to "decision-making." The core lies in using three-dimensional information to enhance and stabilize two-dimensional spectral analysis, and improving the practicality of waste material judgment through multi-modal information collaborative decision-making.

[0031] In summary, to address the challenge of distinguishing materials with similar appearances, accurate chemical material identification is achieved through stable and high-quality intrinsic spectral characteristics. For dark materials with strong absorption and thin, flaky materials with weak or unstable signals due to significant vibration, proactive optimization of acquisition conditions and multi-frame spatiotemporal fusion technology significantly enhance the signal-to-noise ratio and stability. Facing the challenges of identifying surface contamination, moisture interference, and multi-layered composite materials, fusing three-dimensional geometric information with surface appearance information effectively identifies contamination, determines material integrity and composite state, and avoids misjudgments. Finally, by introducing a comprehensive recyclability assessment mechanism based on three-dimensional information, appearance information, and recycling standards, it transcends simple material identification, enabling a comprehensive assessment of the material's recycling value and improving resource recovery rates and economic value.

[0032] In some embodiments of this application, the step of obtaining the three-dimensional information and surface appearance information of the material preferably includes: The surface appearance information of the material is acquired using a visible light camera, and the surface appearance information includes at least one of the material's color, texture, and surface contamination information; A structured light pattern is projected onto the surface of the material in the detection area located on the waste sorting line using a structured light projection device. The image acquisition device works synchronously with the structured light projection device to acquire the deformation image of the structured light pattern on the material surface. The deformation image is processed, and the three-dimensional coordinates of each point on the surface of the material are calculated by using the triangulation principle or the coded structured light decoding algorithm to generate the three-dimensional point cloud data of the material. Based on the three-dimensional point cloud data, the three-dimensional spatial boundary, orientation, and thickness of the material are determined as the three-dimensional information of the material.

[0033] Acquiring surface appearance information of materials refers to capturing two-dimensional image data of the material surface using imaging equipment within the standard visible spectrum. This surface appearance information may specifically include the material's inherent color characteristics, surface texture details, and any surface contamination such as stains or attachments. This information plays a crucial role in subsequent material identification and sorting instruction generation.

[0034] Projecting structured light patterns onto material surfaces involves using equipment such as laser projectors or LED arrays to project pre-defined geometric optical patterns (e.g., stripes, dot matrix, or coded patterns) onto the surfaces of materials on a waste sorting line. Simultaneously, an image acquisition device, such as an industrial camera, works synchronously with the structured light projection device to capture images of the deformation of the structured light pattern on the material surface from different angles or positions. This synchronous acquisition ensures temporal consistency between the deformed image and the projected pattern, providing a foundation for accurate 3D reconstruction.

[0035] The deformation image is processed using image processing algorithms to analyze the deformation produced by the structured light pattern on the material surface. For example, triangulation can be used to calculate the three-dimensional coordinates of each point on the material surface using known light source, camera position, and the geometric relationship of the deformation pattern. Alternatively, when using coded structured light, a decoding algorithm can identify the coded information corresponding to each pixel, thereby accurately calculating its three-dimensional position. This generates three-dimensional point cloud data describing the geometry of the material surface, containing a set of three-dimensional coordinates of discrete points on the material surface.

[0036] The three-dimensional spatial boundary of a material refers to its contour or shape in three-dimensional space, which can be obtained through edge detection or contour extraction algorithms of point clouds. The material's orientation refers to its direction and tilt angle in space, which can be determined by performing principal component analysis (PCA) on the point cloud data or fitting geometric primitives (such as planes or spheres) to determine its normal direction or principal axis direction. The material's thickness can be estimated by analyzing the maximum span in the depth direction of the point cloud data or by fitting the upper and lower surfaces. This determined three-dimensional information is crucial for subsequent spectral calibration, material identification, and sorting decisions.

[0037] The proposed solution utilizes surface appearance information acquired through a visible light camera to provide intuitive data for preliminary material identification and contamination assessment. Meanwhile, the 3D point cloud data obtained through structured light technology can precisely quantify the material's 3D spatial boundaries, orientation, and thickness. This comprehensive perception capability, combining 2D appearance information and 3D geometric information, improves the accuracy and completeness of material information acquisition, laying a solid foundation for subsequent spectral acquisition optimization, spatial calibration, generation of stable spectral feature data, and ultimately, material identification and sorting instruction generation.

[0038] In a further embodiment of this application, the step of determining the three-dimensional spatial boundary, orientation, and thickness of the material based on the three-dimensional point cloud data preferably includes: Applying a point cloud segmentation algorithm to the three-dimensional point cloud data will segment the point clouds corresponding to visually overlapping materials into independent sets of material point clouds; By analyzing the changes in the normal vectors in the point clouds of independent material point cloud sets, the three-dimensional spatial boundaries of the corresponding materials can be identified. By fitting the surface point cloud of an independent set of material point clouds, the normal vector of the surface of the set of material point clouds is calculated to determine the pose of the corresponding material. The thickness of the corresponding material is estimated by calculating the coordinate span of the point cloud in the depth direction of an independent set of material point clouds.

[0039] Point cloud segmentation algorithms aim to distinguish different materials within raw 3D point cloud data. On waste sorting lines, materials are often stacked or closely packed, potentially causing visual overlap. By applying point cloud segmentation algorithms, such as those based on region growing, clustering, or deep learning, these overlapping material point cloud data can be effectively separated into independent material point cloud sets, ensuring that subsequent analysis of individual materials is not interfered with by other materials.

[0040] For each individual material point cloud set, the changes in the normal vectors in the point cloud are usually manifested at the edges of the object or where the surface curvature changes significantly. Therefore, by detecting these changes, the actual outline and boundary of the material can be accurately delineated.

[0041] The orientation of a material refers to its direction and tilt angle in three-dimensional space, which is crucial for subsequent geometric adjustments in spectral acquisition and the generation of sorting instructions. For example, point clouds can be fitted to planes or surfaces using methods such as principal component analysis (PCA) or least squares to obtain their principal axis directions or surface normals, thereby characterizing the material's orientation.

[0042] By calculating the coordinate span of the point cloud in the depth direction of an independent set of material point clouds, the dimensions of the material perpendicular to the sorting line plane are directly reflected. This is of great significance for assessing the volume and weight of the material, as well as the handling difficulty during the sorting process. For example, the maximum and minimum values ​​of the depth coordinates in the point cloud can be found, and the difference between them is the estimated thickness of the material.

[0043] The proposed solution effectively avoids misidentification and information confusion caused by material overlap through refined point cloud segmentation; it ensures accurate acquisition of the material's three-dimensional spatial boundaries and orientation through normal vector analysis and surface fitting, which is crucial for subsequent spectral acquisition optimization and sorting instruction generation; and accurate thickness estimation further refines the description of the material's physical properties. This three-dimensional information provides a solid data foundation for subsequent material identification and sorting decisions, improving the overall efficiency and accuracy of waste pretreatment material classification.

[0044] In a specific embodiment of this application, the step of spatially calibrating multiple consecutive frames of spectral images based on the three-dimensional spatial boundary and orientation of the material, so as to align the pixel positions of image regions representing the same material in different frames, preferably includes: Based on the three-dimensional spatial boundary and orientation of the material, the reference pose of the material is determined; Based on the three-dimensional spatial boundary and the pose, calculate the pose transformation parameters of the image region representing the same material relative to the reference pose in each frame of the continuous multi-frame spectral images; Based on the pose transformation parameters, a geometric transformation is performed on the image region representing the same material in each frame of the continuous multi-frame spectral images, so that the pixel positions of the image regions representing the same material in different frames are aligned.

[0045] Determining the reference pose of a material refers to selecting a base frame or calculating an average pose from a multi-frame image sequence as a reference for calibration of all subsequent frames. For example, the pose of the material in the first frame can be set as the reference pose, or a representative and stable reference pose can be obtained through statistical analysis of the material's poses across multiple frames. This reference pose provides a unified coordinate system for subsequent pose transformation calculations.

[0046] Calculating pose transformation parameters involves quantifying the difference between the actual pose of the material in each frame and a preset reference pose. This can be achieved through various geometric transformation models, such as rigid body transformations (including translation and rotation), affine transformations, or perspective transformations. These parameters describe how to map the image region of the material in the current frame to the position corresponding to the reference pose.

[0047] Geometric transformations of the image region can be performed based on pose transformation parameters. For example, image registration techniques can be used, and image pixels can be resampled using interpolation algorithms (such as bilinear interpolation or bicubic interpolation) to achieve translation, rotation, scaling, or more complex deformation correction of the image region. The purpose is to eliminate image region misalignment caused by factors such as material movement and tumbling on the sorting line or changes in camera perspective, ensuring that the pixel coordinates of the same physical point in different frames are as consistent as possible.

[0048] The proposed solution first establishes a unified reference pose, providing a stable benchmark for subsequent calibration operations. Then, for each frame of the image, the required pose transformation parameters are calculated based on the difference between the actual pose of the material and this reference pose. Because these parameters quantify the geometric deviation of the material relative to the reference pose in each frame, subsequent geometric transformations can be specifically used to correct image regions. This precise calibration based on 3D information can compensate for image misalignment caused by material movement, ensuring accurate alignment of pixels representing the same material in different frames before time-series fusion.

[0049] In a specific embodiment of this application, the step of performing time-series fusion processing on consecutive multi-frame spectral images after spatial calibration to obtain stable spectral characteristic data of the material preferably includes: In the spatially calibrated multi-frame spectral images, the spectral data of pixels representing the same material are calculated by performing an arithmetic average or weighted average in the time dimension to obtain stable spectral feature data of the material.

[0050] The spatially calibrated consecutive multi-frame spectral images refer to multi-frame spectral images that have undergone geometric transformation to align the pixel positions of image regions representing the same material in different frames. The spectral data of pixels representing the same material refers to the sequence of spectral information collected from pixels corresponding to the same physical location on the material in these calibrated consecutive multi-frame spectral images.

[0051] When performing arithmetic averaging over time, the spectral values ​​of the same pixel collected in different frames are typically summed and then divided by the number of frames to obtain the average spectral value of that pixel. As a preferred implementation, weighted averaging can also be used. Weighted averaging involves assigning different weights to the spectral data of different frames and then summing them. These weights can be determined based on various factors, such as the frame's signal-to-noise ratio, image sharpness, acquisition time, or correlation with the material's motion state. Weighted averaging can highlight data from high-quality frames or suppress the influence of interfering frames, thereby obtaining a more accurate fusion result. Thus, after fusion processing, stable spectral characteristic data of the material can be obtained, which more accurately reflects the intrinsic spectral properties of the material.

[0052] The proposed solution improves the signal-to-noise ratio and stability of the acquired material spectral feature data. Compared to analysis using only single-frame spectral data, the stable spectral feature data after time-series fusion processing more accurately reflects the true material properties, effectively reducing the risk of misidentification due to factors such as instantaneous noise, material vibration, or changes in ambient light. This provides a more reliable input for subsequent material identification steps.

[0053] In a more specific embodiment of this application, the step of adjusting the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's orientation to optimize the incident and received conditions of the spectral signals, and acquiring continuous multi-frame spectral images of the material, further includes: Within a single exposure time window of acquiring multiple consecutive spectral images, a high-speed visible light camera array deployed at different angles is used to simultaneously capture the deformation process of the material surface and quantify the local deformation parameters of the material within the exposure time window. Before the step of performing time-series fusion processing on the spatially calibrated consecutive multi-frame spectral images to obtain stable spectral characteristic data of the material, the method further includes: Based on the local deformation parameters and the preset optical transmission model, predicted distortion data that quantitatively describes the interference of the current deformation state on the spectral signal is calculated. Based on the predicted distortion data, inverse compensation is performed on the spatially calibrated consecutive multi-frame spectral images to restore the intrinsic spectral characteristics of the material in its undeformed state, resulting in compensated consecutive multi-frame spectral images, which serve as the basis for calculating the stable spectral characteristic data of the material.

[0054] A high-speed visible light camera array is a system composed of multiple visible light cameras deployed at different angles to simultaneously capture dynamic changes on a material surface from multiple perspectives. Due to its high-speed characteristics, this camera array can capture minute deformation processes on the material surface within an extremely short exposure time window—the time it takes to acquire a single frame of spectral image. Through collaborative processing of multi-view images, the local deformation parameters of the material at that instant can be precisely quantified, such as displacement of surface points, deformation gradient, and curvature changes. The purpose is to provide a precise physical basis for subsequent spectral distortion compensation.

[0055] The optical transmission model can be understood as a mathematical or physical model used to describe how the surface deformation state of a material (characterized by local deformation parameters) affects the reflection, scattering, and absorption of incident light, thereby leading to distortion of the spectral signal. This model can be constructed based on physical optics principles, empirical data, or machine learning methods, with the aim of mapping the material's deformation parameters to specific interference patterns on the spectral signal. Based on the local deformation parameters and the preset optical transmission model, predicted distortion data that quantitatively describes the interference of the current deformation state on the spectral signal can be calculated. This data can include deviations in spectral intensity at specific wavelengths, changes in spectral shape, etc., with the aim of accurately predicting the degree and direction of the spectral signal deviating from the intrinsic spectrum caused by deformation.

[0056] The inverse compensation refers to the reverse correction of multiple consecutive frames of spatially calibrated spectral images based on the calculated predicted distortion data. For example, if the predicted distortion data shows that the spectral intensity of a certain region is enhanced due to deformation, corresponding attenuation compensation is performed; conversely, if it is weakened due to deformation, enhancement compensation is performed. The aim is to eliminate or significantly reduce the influence of material deformation on the spectral signal, thereby restoring the intrinsic spectral characteristics of the material in its undeformed state. This allows the compensated multiple consecutive frames of spectral images to more realistically reflect the material properties, providing a purer data source for subsequent time-series fusion.

[0057] The proposed solution improves the accuracy of spectral detection for flexible or irregular materials in dynamic sorting environments. By precisely quantifying and inversely compensating for spectral distortion caused by material deformation, the resulting stable spectral feature data more accurately reflects the material's properties rather than its instantaneous physical state. This not only enhances the reliability of material identification and reduces missorting rates but also enables the system to adapt to a wider range of material types and more complex operating environments, thereby improving the overall performance and robustness of waste pretreatment material sorting methods.

[0058] The following is a specific example to illustrate this.

[0059] Imagine a plastic film moving along a waste sorting line. Due to vibrations or airflow, the film's surface may experience slight wrinkles or jitters during the tens of milliseconds of exposure time required for a single frame image to be captured by a spectral camera. Meanwhile, an array of multiple high-speed visible light cameras deployed above and to the sides of the film simultaneously captures images of its surface at hundreds or even thousands of frames per second. By performing 3D reconstruction and motion analysis on these high-speed images, local deformation parameters such as displacement vectors and local curvature changes at various points on the film's surface can be precisely calculated.

[0060] These local deformation parameters are then input into a pre-defined optical transmission model. This model may have been trained with experimental data to understand the impact of different degrees of wrinkles or surface tilt on near-infrared spectral reflectance. For example, the model might predict that the spectral signal of a wrinkled region might be 5% lower than that of a flat region or exhibit a redshift at a specific wavelength. Based on these predictions, the system generates corresponding prediction distortion data.

[0061] After spatial calibration of consecutive multi-frame spectral images of the thin film, the predicted distortion data is used for inverse compensation of each frame. For example, if a pixel exhibits low reflectance in the original spectral image due to deformation, the compensation algorithm adjusts its gain appropriately based on the predicted distortion data to restore its intrinsic spectral value in a flat, undeformed state. After compensation, these consecutive multi-frame spectral images are fed into a time-series fusion module for arithmetic or weighted averaging calculations, ultimately yielding more accurate and stable thin film spectral feature data, thus ensuring the accuracy of subsequent material identification.

[0062] Based on the above embodiments, the step of identifying the material of the material according to the stable spectral feature data, the three-dimensional information, and the surface appearance information preferably includes: The stable spectral feature data is matched with a preset standard material spectral library to calculate at least one candidate material and its corresponding spectral matching confidence level. Based on the three-dimensional information, the geometric integrity of the material and the spatial relationship between the material and the boundaries of other materials in the field of view are analyzed to obtain the physical state analysis results. Based on the surface appearance information, the contamination coverage of the material surface is analyzed to obtain the contamination level analysis results; Based on the preset material verification rules, and combined with the spectral matching confidence level, the physical state analysis results, and the pollution level analysis results, the candidate materials are confirmed or modified to determine the material of the material.

[0063] The similarity between the collected material spectral characteristics and the typical spectral fingerprints of known materials is quantified. The standard material spectral library stores standard spectral characteristic data for various common recyclable materials (such as PET, HDPE, PP, PS, etc.) and non-recyclable materials. The matching process can employ various algorithms such as spectral angle matching, correlation coefficient methods, or machine learning classifiers to calculate one or more most likely candidate materials, and provide a spectral matching confidence score for each candidate material, reflecting the reliability of the match.

[0064] Analyzing the geometric integrity of the material and its spatial relationship with the boundaries of other materials within the field of view involves assessing the material's physical state using information such as its three-dimensional spatial boundaries, orientation, and thickness. For example, it can determine whether the material is in a complete form (e.g., a complete bottle or box), whether it has any broken or missing parts, or whether it overlaps or adheres to other materials. By analyzing the material's geometric characteristics, it's possible to avoid misclassifying broken, incomplete, or obscured materials as complete materials, thereby improving the accuracy of identification.

[0065] Analyzing the contamination coverage of material surfaces involves assessing the cleanliness of the material surface using its color, texture, and surface contamination information. For example, it can identify the presence of contaminants such as labels, oil, dirt, and liquid residues on the material surface, and quantify their coverage area or degree of contamination. Surface contamination affects the purity of spectral signals, causing shifts in spectral characteristics; therefore, analyzing the degree of contamination helps correct or adjust spectral matching results.

[0066] Material verification rules are a pre-defined logical judgment or decision-making model that comprehensively considers the preliminary results of spectral matching, the physical integrity of the material, and the surface contamination status. For example, if the spectral matching confidence is high, and the physical state analysis results show that the material is intact, has no obvious overlap, and has a low degree of contamination, the candidate material obtained from spectral matching can be confirmed. Conversely, if the spectral matching confidence is low, or the physical state is abnormal (such as severe damage or obstruction), or the degree of contamination is too high, it may be necessary to modify the candidate material, or even classify it as an "unknown" or "pending" material, to avoid misclassification. For example, a confidence threshold (such as 0.85) can be set, and only candidate materials exceeding this threshold will be further verified. For materials with high confidence, independent physical state, and clean surface, the rule will directly "confirm" its material. However, when it is detected that the material is tightly bonded with a different material, or that there is heavy contamination on the surface, even if its spectral characteristics clearly point to PET, the rule will "correct" the judgment result: outputting it as "PET-PVC composite material" or "contaminated PET," instead of simply "PET." This rule-based verification mechanism essentially elevates simple "chemical material identification" to a comprehensive material identification that includes both "physical state" and "cleanliness state." This allows the identification results to directly and accurately reflect whether the material has the potential for high-purity recycling, thus providing reliable and sufficient input for subsequent precise sorting decisions based on recyclable value.

[0067] This application's solution comprehensively utilizes stable spectral characteristic data, three-dimensional information, and surface appearance information of materials, and introduces a multi-dimensional analysis and verification mechanism to make the material identification process more refined and intelligent. This significantly improves the accuracy and robustness of material identification on the waste pre-processing sorting line, reduces mis-sorting, thereby increasing the purity of recycled materials, reducing subsequent processing costs, and ultimately improving the efficiency and economic benefits of the entire waste resource utilization process.

[0068] The following is a specific example to illustrate this.

[0069] Suppose a crushed plastic bottle with oil stains is detected on the waste sorting line.

[0070] First, stable spectral characteristic data of the plastic bottle were obtained and matched with a standard material spectral library. Preliminary matching results may show an 85% confidence level for PET material, but due to the presence of oil, a 10% confidence level for PP material is also possible.

[0071] Secondly, the physical state of the plastic bottle was analyzed. Although it was flattened, the three-dimensional spatial boundary analysis showed that it still maintained the basic structure of the bottle and had no obvious overlap with other surrounding materials, thus obtaining the analysis result of "physical state: intact but deformed".

[0072] Next, the contamination level of the plastic bottle was analyzed. Visible light camera images showed obvious oil stains on the bottle, and the contamination level analysis result was "Contamination level: Moderate oil stains".

[0073] Finally, a comprehensive judgment is made based on preset material verification rules. The rules might be set as follows: if the PET spectral matching confidence level is higher than 80%, the physical state is intact, and the contamination level is moderate or lower, then the material is confirmed as PET. In this case, despite the presence of oil stains, the system ultimately confirms the material as PET because of its high spectral confidence level and intact physical form. If the oil stains are so severe that the PET confidence level falls below a threshold, or if the 3D information shows the bottle is severely broken, it may be identified as "unknown" or "non-recyclable" according to the rules, avoiding incorrect PET sorting.

[0074] In a further embodiment of this application, the step of adjusting the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's orientation to optimize the incident and received conditions of the spectral signals, and acquiring continuous multi-frame spectral images of the material, further includes: The material is irradiated with a polarized near-infrared light source in at least two orthogonal polarization directions, and the polarization spectral data of the material is collected. After the step of confirming or modifying the candidate materials according to preset material verification rules, combined with the spectral matching confidence level, the physical state analysis results, and the contamination level analysis results, and determining the material of the material, the method further includes: Based on the polarization spectral data, polarization spectral features characterizing the microscopic chemical structure of the material are extracted. The polarization spectral features are matched with a preset chemical residue polarization fingerprint database to identify whether chemical residues exist in the material and the type of chemical residues, which serves as the basis for adjusting the sorting instructions of the material in subsequent steps.

[0075] Polarized near-infrared light sources are light sources capable of emitting near-infrared light with specific polarization states. For example, they can be achieved by combining polarizing filters with a broadband near-infrared light source, or by using laser diode arrays with inherent polarization characteristics. By irradiating a material with at least two orthogonal polarization directions (e.g., 0° and 90°, or 45° and 135°), the reflection, transmission, or absorption spectra of the material under different polarized light irradiation can be obtained. This multi-polarization irradiation helps to capture the unique response of the material's surface or internal microstructure to polarized light, thus providing richer structural information than unpolarized light.

[0076] Polarization spectral data can be understood as spectral information of materials collected under illumination from different polarization directions. This data includes the response of microscopic features such as molecular structure, crystal orientation, surface roughness, and chemical bonding states to polarized light. By analyzing this polarization spectral data, polarization spectral features can be extracted. Polarization spectral features refer to numerical values ​​or patterns extracted from polarization spectral data that characterize the microscopic chemical structure of the material or the presence of specific chemical residues. For example, polarization spectral features can be formed by calculating parameters such as the difference, ratio, or degree of polarization of spectra from different polarization directions.

[0077] A chemical residue polarization fingerprint database is a pre-established collection of characteristic spectral fingerprints of various common or target chemical residues under different polarized light irradiation. This database is constructed by performing polarization spectral measurements and feature extraction on known chemical residues. By matching the extracted polarization spectral features of the material with this database, the presence and type of specific chemical residues in the material can be identified. For example, using machine learning algorithms or spectral similarity analysis, the polarization spectral features of the material to be tested are compared with the fingerprints in the database to determine the presence of a certain chemical residue, such as plasticizers, flame retardants, pesticide residues, or specific food residues. The identified chemical residue types serve as an important basis for adjusting sorting instructions, ensuring the accuracy and safety of sorting.

[0078] This application introduces polarization spectroscopy technology. By capturing the unique response of materials to polarized light, it can extract polarization spectral features characterizing the microscopic chemical structure of materials and match them with a chemical residue polarization fingerprint database to accurately identify the presence and type of specific chemical residues in the material. This capability enables the sorting system to not only identify the macroscopic material composition of materials but also gain a deeper understanding of their microscopic chemical composition, such as detecting plasticizers and flame retardants in plastics or ink residues in paper. Therefore, sorting instructions can be adjusted based on the presence and type of chemical residues. For example, materials containing hazardous chemical residues can be sorted into special treatment categories instead of ordinary recycling categories, or materials with excessive levels of specific chemical residues can be directly classified as waste. This not only helps improve the purity of recycled materials and the quality of reused products, reducing the risk of secondary pollution, but also ensures the environmental safety of the sorting process and subsequent processing, thereby achieving higher-value resource recycling.

[0079] In this embodiment of the application, the step of generating a sorting instruction for the material based on the material, the three-dimensional information, and the surface appearance information preferably includes: Based on the material, the physical state analysis results obtained from the three-dimensional information, and the pollution degree analysis results obtained from the surface appearance information, and in conjunction with the preset recycling purity standard, the recyclability of the material is comprehensively evaluated to obtain the evaluation result. Based on the evaluation results, a sorting instruction is generated to sort the material into the corresponding recycling category or waste category.

[0080] "Physical condition analysis results" refer to the process of analyzing a material's three-dimensional information, such as its geometric integrity, presence of damage or deformation, and its spatial relationship with the boundaries of other materials within the field of view, to determine the material's structural integrity and physical usability. For example, even if a plastic bottle is made of the correct material, its recycling value and processing difficulty will change if it is severely flattened or broken.

[0081] "Contamination level analysis results" refer to the quantitative assessment of the cleanliness of a material's surface by analyzing its surface appearance information, such as color, texture, and surface contamination coverage. For example, food residue, oil stains, and label residue can all affect the purity of the recycled material.

[0082] "Preset recycling purity standards" refer to quantitative indicators regarding material purity, physical integrity, and degree of contamination, set in advance according to the requirements of different recycling categories or downstream processing technologies. These standards are key criteria for determining whether materials can be effectively recycled.

[0083] By combining the results of material and physical state analysis, pollution level analysis, and preset recycling purity standards, the "recyclability" of materials can be comprehensively assessed. This assessment is a multi-factor weighted decision-making process aimed at determining whether a material is suitable for entering a specific recycling stream or should be discarded.

[0084] Based on the material's recyclability level or type determined through comprehensive evaluation, the system will generate a clear sorting instruction, directing the sorting facility to send the material to a specific recycling channel (e.g., PET plastic recycling, HDPE plastic recycling, paper recycling, etc.) or directly to the waste disposal channel. For example, the "Grade A PET Recycling" standard may require the material to be pure PET monomers, free of other materials adhering to it, with a surface contamination coverage of <5%, and no irremovable chemical residues. The evaluation process is as follows: the receiving material (e.g., "pure PET", "contaminated PP"), physical state analysis results (whether it is independent, intact, or composite), and contamination level analysis results are compared item by item with the purity standards for the corresponding material category. For example, for a bottle identified as "PET", the system will retrieve the "PET Recycling Standard" and check: whether its physical state shows "independent and intact individual" (no, then it conflicts with the "no adhering" standard); and whether its contamination analysis results are below the contamination threshold (no, then it conflicts with the cleanliness standard). Based on the comparison results, the engine executes a tiered decision: if all Grade A standards are met, a "sort to high-purity PET stream" instruction is generated; if only the main material is met but there is slight contamination or separable accessories, it is downgraded to "sort to general PET or mixed plastic stream" according to the rules; if there is unacceptable composite (such as permanent composite of PET and aluminum foil), severe contamination, or detected harmful chemical residues, it is determined that it will contaminate the entire batch of raw materials, generating a "sort to waste or energy disposal" instruction. The core of this implementation method is to transform abstract recycling quality requirements into automatically executed digital rules directly linked to three-dimensional geometry, surface appearance, and chemical spectral information, thereby ensuring that every sorted material meets the actual requirements of downstream deep processing for compositional consistency and purity.

[0085] This application's solution, by comprehensively evaluating the material composition, physical state, degree of contamination, and recycling purity standards, avoids missorting damaged or heavily contaminated materials into the high-value recycling stream, thereby effectively improving the purity of recycled materials and reducing subsequent processing costs. Simultaneously, materials that, while of correct material composition, fail to meet recycling standards due to physical state or contamination issues can be promptly sorted into the waste category, avoiding ineffective recycling. This refined sorting instruction generation mechanism enables more rational resource utilization, reduces waste, and enhances the economic and environmental benefits of the entire waste sorting system.

[0086] like Figure 2As shown, this application also discloses a visual detection-based waste pretreatment material sorting system for sorting materials on a waste sorting line. The system includes: The information acquisition module 210 is used to acquire the three-dimensional information and surface appearance information of the material, wherein the three-dimensional information includes the three-dimensional spatial boundary, orientation and thickness of the material; The parameter adjustment module 220 is used to adjust the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's posture, so as to optimize the incident and reception conditions of the spectral signals and acquire continuous multi-frame spectral images of the material. The spectral acquisition and calibration module 230 is used to perform spatial calibration on the continuous multi-frame spectral images based on the three-dimensional spatial boundary and the attitude, so that the pixel positions of the image regions representing the same material in different frames are aligned. The spectral fusion module 240 is used to perform time-series fusion processing on a series of consecutive spectral images after spatial calibration to obtain stable spectral feature data of the material. The material identification module 250 is used to identify the material of the material based on the stable spectral feature data, the three-dimensional information, and the surface appearance information. The instruction generation module 260 is used to generate sorting instructions for the material based on the material, the three-dimensional information, and the surface appearance information.

[0087] The information acquisition module 210 comprises a visible light camera for acquiring surface appearance information, a structured light projection device (such as a line laser scanner or DLP projector), and one or more high-speed industrial cameras synchronized with it. This module also integrates a high-performance edge computing unit that runs 3D reconstruction and point cloud processing algorithms to generate 3D point cloud data of the material and extract 3D spatial boundaries, orientation, and thickness information from it. Alternatively, this module may include a LiDAR scanner or a multi-view stereo vision system.

[0088] The parameter adjustment module 220 includes a programmable logic controller, a mechanical actuator (such as a servo motor gimbal) for driving the spectral camera and / or illumination source, and a high-speed hyperspectral camera. The control unit drives the actuator based on received attitude information to adjust the geometric orientation of the optics and triggers the hyperspectral camera to acquire multiple consecutive frames of spectral images.

[0089] The spectral acquisition and calibration module 230 includes an image processor (such as an FPGA accelerator card or a GPU industrial computer) that is programmed to execute real-time image geometric correction algorithms, calculate pose transformation parameters using three-dimensional spatial boundary and pose data, and perform spatial calibration on multi-frame spectral images.

[0090] The spectral fusion module 240 includes a digital signal processor or a software algorithm module integrated in a central processing unit. This module performs a time-series fusion algorithm (such as arithmetic mean or weighted average) on the spatially calibrated spectral image sequence to obtain stable spectral feature data.

[0091] The material identification module 250 includes a spectral matching unit that integrates a standard material spectral database, and a decision analysis unit that integrates a physical state analysis subroutine, a contamination analysis subroutine, and a rule engine. This module comprehensively processes stable spectral feature data, 3D point clouds, and visible light images, and outputs material determination results with status labels.

[0092] The instruction generation module 260 includes an evaluation unit with an embedded knowledge base of recycling purity standards and an instruction output unit. This module receives material determination results, physical integrity and contamination data, compares and grades them according to preset standards, and generates instructions to control the sorting actuator (such as a pneumatic nozzle array).

[0093] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0094] The preferred embodiments of this application have been described in detail above, but this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application.

Claims

1. A method for classifying waste pretreatment materials based on visual detection, used for sorting materials on a waste sorting line, characterized in that, The method includes the following steps: The three-dimensional information and surface appearance information of the material are obtained, wherein the three-dimensional information includes the three-dimensional spatial boundary, orientation and thickness of the material; Based on the orientation of the material, the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material is adjusted to optimize the incident and reception conditions of the spectral signals and acquire continuous multi-frame spectral images of the material. Based on the three-dimensional spatial boundary and the pose, spatial calibration is performed on the continuous multi-frame spectral images to align the pixel positions of image regions representing the same material in different frames. The material's stable spectral characteristic data are obtained by performing time-series fusion processing on a series of consecutive spectral images after spatial calibration. The material is identified based on the stable spectral characteristic data, the three-dimensional information, and the surface appearance information; Based on the material, the three-dimensional information, and the surface appearance information, a sorting instruction is generated for the material.

2. The method for classifying waste pretreatment materials based on visual detection according to claim 1, characterized in that, The steps for obtaining the three-dimensional information and surface appearance information of the material include: The surface appearance information of the material is acquired using a visible light camera, and the surface appearance information includes at least one of the material's color, texture, and surface contamination information; A structured light pattern is projected onto the surface of the material in the detection area located on the waste sorting line using a structured light projection device. The image acquisition device works synchronously with the structured light projection device to acquire the deformation image of the structured light pattern on the material surface. The deformation image is processed, and the three-dimensional coordinates of each point on the surface of the material are calculated by using the triangulation principle or the coded structured light decoding algorithm to generate the three-dimensional point cloud data of the material. Based on the three-dimensional point cloud data, the three-dimensional spatial boundary, orientation, and thickness of the material are determined as the three-dimensional information of the material.

3. The method for classifying waste pretreatment materials based on visual detection according to claim 2, characterized in that, The step of determining the three-dimensional spatial boundary, orientation, and thickness of the material based on the three-dimensional point cloud data includes: Applying a point cloud segmentation algorithm to the three-dimensional point cloud data will segment the point clouds corresponding to visually overlapping materials into independent sets of material point clouds; By analyzing the changes in the normal vectors in the point clouds of independent material point cloud sets, the three-dimensional spatial boundaries of the corresponding materials can be identified. By fitting the surface point cloud of an independent set of material point clouds, the normal vector of the surface of the set of material point clouds is calculated to determine the pose of the corresponding material. The thickness of the corresponding material is estimated by calculating the coordinate span of the point cloud in the depth direction of an independent set of material point clouds.

4. The method for classifying waste pretreatment materials based on visual detection according to claim 1, characterized in that, The step of spatially calibrating the consecutive multi-frame spectral images based on the three-dimensional spatial boundary and the pose, so that the pixel positions of image regions representing the same material in different frames are aligned, includes: Based on the three-dimensional spatial boundary and orientation of the material, the reference pose of the material is determined; Based on the three-dimensional spatial boundary and the pose, calculate the pose transformation parameters of the image region representing the same material relative to the reference pose in each frame of the continuous multi-frame spectral images; Based on the pose transformation parameters, a geometric transformation is performed on the image region representing the same material in each frame of the continuous multi-frame spectral images, so that the pixel positions of the image regions representing the same material in different frames are aligned.

5. The method for classifying waste pretreatment materials based on visual detection according to claim 1, characterized in that, The step of performing time-series fusion processing on consecutive multi-frame spectral images after spatial calibration to obtain stable spectral feature data of the material includes: In the spatially calibrated multi-frame spectral images, the spectral data of pixels representing the same material are calculated by performing an arithmetic average or weighted average in the time dimension to obtain stable spectral feature data of the material.

6. The method for classifying waste pretreatment materials based on visual detection according to claim 5, characterized in that, The step of adjusting the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's orientation to optimize the incident and received conditions of the spectral signals, and acquiring continuous multi-frame spectral images of the material, further includes: Within a single exposure time window of acquiring multiple consecutive spectral images, a high-speed visible light camera array deployed at different angles is used to simultaneously capture the deformation process of the material surface and quantify the local deformation parameters of the material within the exposure time window. Before the step of performing time-series fusion processing on the spatially calibrated consecutive multi-frame spectral images to obtain stable spectral characteristic data of the material, the method further includes: Based on the local deformation parameters and the preset optical transmission model, predicted distortion data that quantitatively describes the interference of the current deformation state on the spectral signal is calculated. Based on the predicted distortion data, inverse compensation is performed on the spatially calibrated consecutive multi-frame spectral images to restore the intrinsic spectral characteristics of the material in its undeformed state, resulting in compensated consecutive multi-frame spectral images, which serve as the basis for calculating the stable spectral characteristic data of the material.

7. The method for classifying waste pretreatment materials based on visual detection according to claim 1, characterized in that, The step of identifying the material's composition based on the stable spectral feature data, the three-dimensional information, and the surface appearance information includes: The stable spectral feature data is matched with a preset standard material spectral library to calculate at least one candidate material and its corresponding spectral matching confidence level. Based on the three-dimensional information, the geometric integrity of the material and the spatial relationship between the material and the boundaries of other materials in the field of view are analyzed to obtain the physical state analysis results. Based on the surface appearance information, the contamination coverage of the material surface is analyzed to obtain the contamination level analysis results; Based on the preset material verification rules, and combined with the spectral matching confidence level, the physical state analysis results, and the pollution level analysis results, the candidate materials are confirmed or modified to determine the material of the material.

8. The method for classifying waste pretreatment materials based on visual detection according to claim 7, characterized in that, The step of adjusting the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's orientation to optimize the incident and received conditions of the spectral signals, and acquiring continuous multi-frame spectral images of the material, further includes: The material is irradiated with a polarized near-infrared light source in at least two orthogonal polarization directions, and the polarization spectral data of the material is collected. After the step of confirming or modifying the candidate materials according to the preset material verification rules, combined with the spectral matching confidence level, the physical state analysis results, and the contamination level analysis results, and determining the material of the material, the method further includes: Based on the polarization spectral data, polarization spectral features characterizing the microscopic chemical structure of the material are extracted. The polarization spectral features are matched with a preset chemical residue polarization fingerprint database to identify whether chemical residues exist in the material and the type of chemical residues, which serves as the basis for adjusting the sorting instructions of the material in subsequent steps.

9. The method for classifying waste pretreatment materials based on visual detection according to claim 1, characterized in that, The step of generating a sorting instruction for the material based on the material, the three-dimensional information, and the surface appearance information includes: Based on the material, the physical state analysis results obtained from the three-dimensional information, and the pollution degree analysis results obtained from the surface appearance information, and in conjunction with the preset recycling purity standard, the recyclability of the material is comprehensively evaluated to obtain the evaluation result. Based on the evaluation results, a sorting instruction is generated to sort the material into the corresponding recycling category or waste category.

10. A visual detection-based waste pretreatment material sorting system for sorting materials on a waste sorting line, characterized in that, The system includes: The information acquisition module is used to acquire the three-dimensional information and surface appearance information of the material, wherein the three-dimensional information includes the three-dimensional spatial boundary, orientation and thickness of the material; The parameter adjustment module is used to adjust the geometric acquisition relationship of the optical device used to acquire spectral signals relative to the material according to the material's posture, so as to optimize the incident and reception conditions of the spectral signals and acquire continuous multi-frame spectral images of the material. The spectral acquisition and calibration module is used to perform spatial calibration on the continuous multi-frame spectral images based on the three-dimensional spatial boundary and the orientation, so that the pixel positions of the image regions representing the same material in different frames are aligned. The spectral fusion module is used to perform time-series fusion processing on multiple consecutive frames of spatially calibrated spectral images to obtain stable spectral feature data of the material. The material identification module is used to identify the material of the material based on the stable spectral feature data, the three-dimensional information, and the surface appearance information. The instruction generation module is used to generate sorting instructions for the material based on the material, the three-dimensional information, and the surface appearance information.

Images