Method, system, electronic device and storage medium for identifying plastic packaging

Abstract

The application provides a plastic packaging identification method and system, electronic equipment and storage medium. The plastic packaging identification method comprises the following steps: obtaining hyperspectral data of plastic packaging, image data of plastic packaging and conveying speed of plastic packaging; constructing the hyperspectral data into a hyperspectral data cube according to the conveying speed, and performing spectral dimension standardization processing on the hyperspectral data cube to generate a target spectrum vector; correcting the image data to generate visual model data; performing spectral feature matching on the target spectrum vector to generate a material category label; performing geometric feature solving on the visual model data to generate a geometric attribute label; mapping the geometric center coordinates corresponding to the geometric attribute label to a coordinate system in which the hyperspectral data cube is located; and generating identification data packets under the condition that the material category label and the geometric attribute label are spatially consistent.

Description

Technical Field

[0001] This application relates to the technical field of plastic packaging identification, and more particularly to a method, system, electronic device, and storage medium for identifying plastic packaging. Background Technology

[0002] Visual sorting technology in the field of plastic packaging sorting is a key method for the efficient identification and classification of various types of plastic packaging through advanced image processing and computer vision algorithms. This technology utilizes high-resolution cameras to capture the surface features of the packaging, including visual information such as color, texture, shape, and markings, and then combines this with an intelligent analysis system to achieve precise sorting. It not only significantly improves the automation level and accuracy of sorting but also significantly reduces the cost and error of manual sorting. It is widely used in recycling, resource reuse, and the environmental protection industry, providing important technical support for sustainable development and the circular economy.

[0003] In current visual analysis technology for plastic packaging, factors such as the reflective properties of the material surface, similar colors, and diverse textures often make it difficult to accurately distinguish different types of plastic materials. Summary of the Invention

[0004] This application provides a method, system, electronic device, and storage medium for identifying plastic packaging, in order to solve the problems existing in related technologies. The technical solution is as follows: In a first aspect, embodiments of this application provide a method for identifying plastic packaging, including: Acquire hyperspectral data, image data, and conveying speed of the plastic packaging; Based on the transmission speed, the hyperspectral data is constructed into a hyperspectral data cube, and the hyperspectral data cube is subjected to spectral dimension normalization to generate the target spectral vector. The image data is corrected to generate visual model data; Perform spectral feature matching on the target spectral vector to generate material category labels; Perform geometric feature calculations on the visual model data to generate geometric attribute labels; The geometric center coordinates corresponding to the geometric attribute labels are mapped to the coordinate system of the hyperspectral data cube. When the space of the material category label and the geometric attribute label is consistent, an identification data package is generated.

[0005] In one embodiment of this application, it further includes: When the similarity coefficient between the target spectral vector and any standard derivative spectral vector is less than the dynamic classification threshold, principal component analysis is performed on the spectral data of the region to obtain the first k principal component feature vectors. The first k principal component feature vectors are input into the trained random forest classification model to obtain stacked state labels; Write the stack status label into the identification data packet.

[0006] In one embodiment of this application, converting raw byte data to obtain a floating-point array includes: In the case where no valid material category label or non-stacked label was generated after performing spectral analysis on the preprocessed hyperspectral data and comparing it with the standard spectral library, the image data was reconstructed to obtain a three-dimensional point cloud model of the plastic packaging to be identified. Calculate the volume of the 3D point cloud model and the estimated mass of the plastic packaging to obtain the spatial packing density; By inputting the spatial packing density into the transfer learning model, the predicted potential material type is obtained.

[0007] In one embodiment of this application, constructing a hyperspectral data cube from hyperspectral data according to the transmission speed includes: The spatial displacement is calculated by using the transport speed and the row sampling frequency of the hyperspectral imaging device to analyze two adjacent hyperspectral frames of data. Based on the spatial displacement, multiple consecutively acquired hyperspectral frames are stitched together in the spatial dimension to obtain a hyperspectral data cube with both spatial and spectral dimensions.

[0008] In one embodiment of this application, spectral dimension normalization is performed on the hyperspectral data cube to generate a target spectral vector, including: Extract each specified band from the hyperspectral data cube to obtain a single-band two-dimensional image; Performing a two-dimensional Fourier transform on a single-band two-dimensional image will yield frequency domain data. High-frequency components with concentrated energy and vertical distribution are detected in frequency domain data to obtain the strip noise frequency; A band-stop filter is constructed to set the amplitude corresponding to the frequency of strip noise to zero, thus obtaining the filtered frequency domain data. Perform a two-dimensional inverse Fourier transform on the filtered frequency domain data to generate a denoised hyperspectral data cube; The spectral dimension of the denoised hyperspectral data cube is normalized to generate the target spectral vector.

[0009] In one embodiment of this application, spectral dimension normalization is performed on the denoised hyperspectral data cube to generate a target spectral vector, including: Each pixel in the denoised hyperspectral data cube is extracted to obtain the original spectral curve; The original spectral curve is transformed using a standard normal variable to obtain the spectral intensity of the original spectral curve; The spectral intensity of the original spectral curve is calculated to obtain the mean and standard deviation; The normalized spectral curve is obtained by subtracting the mean spectral intensity from the intensity value of each wavelength point in the original spectral curve and then dividing by the standard deviation. The first derivative of the fitted polynomial is obtained by performing a convolution operation on the normalized spectral curve using a filtering algorithm. The target spectral vector is generated based on the first derivative.

[0010] In one embodiment of this application, the geometric center coordinates corresponding to the geometric attribute labels are mapped to the coordinate system where the hyperspectral data cube is located. When the material category labels and geometric attribute labels are in the same spatial location, generating the identification data package includes: The closed contours in the visual model data are extracted to obtain the geometric center coordinates corresponding to the geometric attribute labels; Using a preset coordinate transformation matrix, the geometric center coordinates corresponding to the geometric attribute labels are transformed to the pixel coordinate system of the hyperspectral data cube to obtain the projection center coordinates; Based on the hyperspectral data cube, generate the effective spectral pixel region for the material category label; The effective spectral pixel region of the material category label is calculated to obtain the spectral weighted center coordinates of the effective spectral pixel region; The Euclidean distance between the coordinates of the projected centroid and the coordinates of the spectral weighting center is calculated to obtain the Euclidean distance; If the Euclidean distance is less than the preset tolerance radius, the material category label and the geometric attribute label will satisfy spatial consistency and generate an identification data package.

[0011] Secondly, embodiments of this application provide a plastic packaging identification system, including: The first determining module is used to acquire hyperspectral data of the plastic packaging, image data of the plastic packaging, and the conveying speed of the plastic packaging; The first generation module is used to construct a hyperspectral data cube from the hyperspectral data according to the transmission speed, and to perform spectral dimension normalization on the hyperspectral data cube to generate the target spectral vector. The second generation module is used to correct the image data and generate visual model data; The third generation module is used to perform spectral feature matching on the target spectral vector and generate material category labels; The fourth generation module is used to perform geometric feature calculations on the visual model data and generate geometric attribute labels; The fifth generation module is used to map the geometric center coordinates corresponding to the geometric attribute labels to the coordinate system where the hyperspectral data cube is located, and generate an identification data package when the space of the material category label and the geometric attribute label are consistent.

[0012] Thirdly, embodiments of this application provide an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the aforementioned plastic packaging identification method.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium that stores computer instructions, wherein when the computer instructions are executed on a computer, the methods in any of the above-described embodiments are performed.

[0014] The advantages or beneficial effects of the above technical solutions include at least the following: In this embodiment, the method for identifying plastic packaging includes: constructing a hyperspectral data cube based on the transport speed from hyperspectral data; standardizing the spectral dimensions of the hyperspectral data cube to generate a target spectral vector; correcting the image data to generate visual model data; performing spectral feature matching on the target spectral vector to generate a material category label; performing geometric feature calculation on the visual model data to generate a geometric attribute label; mapping the geometric center coordinates corresponding to the geometric attribute label to the coordinate system of the hyperspectral data cube; and generating an identification data package when the space of the material category label and the geometric attribute label is consistent. This solves the problems of single vision not being able to identify materials and single spectrum not being able to distinguish shapes. By comparing geometric attributes and material types at the same spatial location, it effectively eliminates the problem of misidentification of plastic packaging due to reflections or ghosting, thus improving the identification accuracy of plastic packaging. It effectively solves the problem in current plastic packaging visual analysis technology that the reflective characteristics of the material surface, similar colors, and diverse textures often make it difficult to accurately distinguish different types of plastic materials.

[0015] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0016] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0017] Figure 1 This is a flowchart of a method for identifying plastic packaging according to an embodiment of this application.

[0018] Figure 2 This is a block diagram of an electronic device according to an embodiment of the present application. Detailed Implementation

[0019] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0020] Figure 1 A flowchart illustrating a method for identifying plastic packaging according to an embodiment of this application is shown. Figure 1 As shown, a method for identifying plastic packaging includes: S110: Acquire hyperspectral data of the plastic packaging, image data of the plastic packaging, and the conveying speed of the plastic packaging; S120: Based on the transmission speed, the hyperspectral data is constructed into a hyperspectral data cube, and the hyperspectral data cube is subjected to spectral dimension normalization to generate the target spectral vector; S130: Correct the image data and generate visual model data; S140: Perform spectral feature matching on the target spectral vector to generate material category labels; S150: Perform geometric feature calculations on the visual model data to generate geometric attribute labels; S160: Map the geometric center coordinates corresponding to the geometric attribute labels to the coordinate system where the hyperspectral data cube is located. If the space of the material category label and the geometric attribute label is consistent, generate an identification data package.

[0021] In this embodiment, the method for identifying plastic packaging includes: constructing a hyperspectral data cube based on the transport speed from hyperspectral data; standardizing the spectral dimensions of the hyperspectral data cube to generate a target spectral vector; correcting the image data to generate visual model data; performing spectral feature matching on the target spectral vector to generate a material category label; performing geometric feature calculation on the visual model data to generate a geometric attribute label; mapping the geometric center coordinates corresponding to the geometric attribute label to the coordinate system of the hyperspectral data cube; and generating an identification data package when the space of the material category label and the geometric attribute label is consistent. This solves the problems of single vision not being able to identify materials and single spectrum not being able to distinguish shapes. By comparing geometric attributes and material types at the same spatial location, it effectively eliminates the problem of misidentification of plastic packaging due to reflections or ghosting, thus improving the identification accuracy of plastic packaging. It effectively solves the problem in current plastic packaging visual analysis technology that the reflective characteristics of the material surface, similar colors, and diverse textures often make it difficult to accurately distinguish different types of plastic materials.

[0022] In step S110, hyperspectral data of the plastic packaging, image data of the plastic packaging, and conveying speed of the plastic packaging are acquired.

[0023] In this embodiment, a hyperspectral imager mounted above the belt conveyor is used for data acquisition. Considering the vibrational absorption characteristics of functional groups such as CH bonds and OH bonds in the molecular structures of different plastic materials (such as PE, PET, PVC, etc.), this embodiment configures the imager to cover the spectral range of 400-2500nm, with a focus on acquiring data in the 900-1700nm (near-infrared NIR region), as this band can significantly enhance material differentiation. The hyperspectral data not only contains the two-dimensional spatial information (x, y) of the plastic packaging, but also a set of one-dimensional spectral information λ for each pixel in continuous bands, typically presented in a "combined image and spectrum" format.

[0024] During the data acquisition process, to balance the reflectivity signal of the particle surface and suppress background interference, the imaging angle needs to be calibrated so that the camera's pitch angle forms a preset angle with the conveyor belt surface (preferably 15°-30°). Simultaneously, a uniform light source composed of halogen lamps and LEDs is used to avoid the influence of natural light fluctuations on spectral intensity. The spectral information of each row of plastic packaging on the conveyor belt is recorded using a pushbroom imaging method.

[0025] High-resolution industrial cameras (RGB cameras) are used to acquire visible light image data of plastic packaging. The image data is mainly used to obtain high-precision texture, edge contours, and surface stain information of the plastic packaging, compensating for the relatively low spatial resolution of hyperspectral imaging.

[0026] Because the belt speed is not constant during actual operation due to the load, this embodiment uses an encoder mounted on the belt drive pulley to monitor the belt's operating status in real time. The pulse signal output by the encoder is received by the controller and converted into real-time conveyor speed. Conveyor speed refers to the distance the belt carrying the plastic packaging travels per unit time.

[0027] In step S120, the hyperspectral data is constructed into a hyperspectral data cube according to the transmission speed, and the hyperspectral data cube is subjected to spectral dimension normalization to generate the target spectral vector.

[0028] In the embodiments of this application, the hyperspectral imager typically employs a pushbroom acquisition mode, meaning that as the belt moves, the hyperspectral imager acquires data for one line at a time (including the spatial X-axis and wavelength λ-axis). To reconstruct the complete two-dimensional shape of the plastic packaging, it is necessary to stitch together the continuously acquired multi-frame line data along the direction of motion (Y-axis).

[0029] In this process, the conveyor speed plays a decisive role in spatial calibration. If data is stitched together only at fixed time intervals, the stitched image will be stretched (when the speed decreases) or compressed (when the speed increases) in the Y-axis direction when the belt speed changes, resulting in distortion of the plastic packaging shape.

[0030] In this embodiment, the stacking interval or resampling ratio of data rows is dynamically adjusted according to the transmission speed. Specifically, the spatial-wavelength two-dimensional data collected in each frame are stacked sequentially in time series according to the physical displacement represented by the transmission speed, thereby constructing a hyperspectral data cube containing two spatial dimensions (X, Y) and one spectral dimension (λ). The hyperspectral data cube accurately reproduces the aspect ratio of the plastic packaging in the physical world. The hyperspectral data cube is a three-dimensional data cube structure, where the X and Y axes represent the planar image information of the plastic packaging, and the Z axis (spectral axis) represents the light intensity response of each pixel at different wavelengths.

[0031] Because of the complex surface morphology of plastic packaging (such as flakes and films), including wrinkles and varying particle sizes, light scattering at different degrees occurs on the surface. This means that even for the same material (such as PET), the intensity of its original spectrum can vary significantly due to differences in surface roughness (i.e., baseline drift or multiplicative interference), directly affecting recognition accuracy.

[0032] To eliminate this physical interference, spectral normalization is performed on the spectral curve corresponding to each pixel in the hyperspectral data cube. In this embodiment, spectral normalization is performed using the Standard Normal Variable Transform (SNV) algorithm. Specifically, for the original spectral data of any pixel in the hyperspectral data cube, the mean and standard deviation of all bands of the spectral curve are calculated; then, the spectral value of each band is subtracted from the mean, and then divided by the standard deviation. The calculation formula is: Where: X snv : represents the standardized spectral data; x: represents the original acquired spectral data; : represents the average value of the spectrum curve; σ: represents the standard deviation of the spectrum curve.

[0033] After processing the hyperspectral data cube using the standard normal variable transformation algorithm described above, the spectral curves of plastic packaging of the same material with different surface conditions will be normalized to the same baseline, resulting in normalized spectral curve data. This normalized spectral curve data is then used to generate a target spectral vector for subsequent feature matching. The target spectral vector refers to a one-dimensional data sequence extracted from the hyperspectral data cube, which has undergone mathematical transformation to eliminate physical noise and represents only the chemical composition characteristics of the plastic packaging.

[0034] In this embodiment, the standard normal variable transformation algorithm effectively solves the problem of varying light scattering on the surface of plastic packaging (such as bottles and films) due to complex surface morphology, wrinkles, and inconsistent particle sizes. This results in significant differences in the intensity of the original spectrum of the same material (such as PET) due to variations in surface roughness (i.e., baseline drift or multiplicative interference), directly affecting recognition accuracy. Even for PET plastic with dirty, wrinkled, or broken surfaces, after standardization using the standard normal variable transformation algorithm, the target spectral vector can be highly consistent with the spectrum of standard clean PET, thus significantly improving the material recognition accuracy under harsh conditions.

[0035] In step S130, the image data is corrected to generate visual model data; In this embodiment, to avoid the industrial camera (RGB camera) obstructing the optical path of the hyperspectral imager or to obtain a larger field of view, the industrial camera is often installed at a certain tilt angle (e.g., at an angle of 15°-30° to the belt normal). This non-perpendicular shooting causes trapezoidal distortion (objects appear larger when closer and smaller when farther away) and perspective distortion in the image data. Furthermore, the industrial camera lens itself may have optical barrel or pincushion distortion. Without correction, the coordinates of the plastic packaging in the image will not correspond linearly to the actual physical coordinates on the belt, leading to errors in subsequent air jet position calculations.

[0036] In this embodiment, the camera's intrinsic parameter matrix and distortion coefficients are obtained through a pre-calibration board. The image data, based on the intrinsic parameter matrix and distortion coefficients, is then resampled and geometrically transformed (e.g., by perspective transformation) to correct the tilted image into an orthophoto image from a vertical top-down view, ensuring that the distance between pixels in the image is linearly proportional to the actual physical distance.

[0037] The corrected image still contains the conveyor belt background, typically black or another single color. To extract the plastic packaging itself, background segmentation is performed. Specifically, a grayscale threshold or color threshold is set, for example, defining the hue range of the background in the HSV color space. Pixels belonging to the conveyor belt background are set to zero (black), while pixels belonging to the plastic packaging are retained. Subsequently, morphological operations (such as opening) are performed on the retained plastic packaging area to remove edge burrs and noise, resulting in a sharp binarized mask.

[0038] The visual model data in this embodiment is a parametric data set describing the geometric attributes and spatial location of plastic packaging, abstracted from the original pixel matrix after preprocessing. It is based on a structured feature set extracted from a binary mask. Using edge detection algorithms (such as the Canny operator) or contour finding algorithms, the closed contour information of each individual plastic package is extracted from the processed binary mask. This generates the visual model data. The visual model data includes a sequence of contour points, centroid coordinates, and the minimum bounding rectangle.

[0039] Among them, the contour point set sequence is the coordinate set describing the edge shape of the plastic packaging; the centroid coordinates are the geometric center position of the plastic packaging in the calibrated image coordinate system; and the minimum bounding rectangle is the minimum bounding box parameters (length, width, angle) that enclose the plastic packaging.

[0040] In this embodiment, trapezoidal distortion of the image data is eliminated through geometric and perspective correction, enabling the coordinate system in the visual model data to establish an accurate linear mapping relationship with the coordinate system of the hyperspectral data cube.

[0041] In step S140, spectral feature matching is performed on the target spectral vector to generate a material category label.

[0042] In this embodiment, the standard spectral feature library is a set of data pre-stored in memory. It contains the standard spectral vectors of various known materials (such as PET, PP, PE, PVC, PC, etc.) under ideal conditions.

[0043] The target spectral vector is input into the standard spectral feature library for matching, that is, the target spectral vector is compared with each standard spectral vector in the standard spectral feature library one by one.

[0044] The Spectral Angle Mapper (SAM) algorithm can be used to measure the similarity between a target spectral vector and each standard spectral vector in a standard spectral feature library. The SAM algorithm treats spectral data across N bands as two vectors in N-dimensional space and determines similarity by calculating the generalized angle between the two vectors. A smaller angle indicates a more consistent shape of the spectral curves and a more similar material composition.

[0045] The formula for calculating similarity (usually expressed as a cosine value) is as follows: in: : Represents the spectral similarity value (ranging from 0 to 1, with values ​​closer to 1 indicating greater similarity); n: Represents the total number of spectral bands; t i : Represents the spectral value of the target spectral vector in the i-th band; r i : Represents the spectral value of the standard spectral vector in the i-th band in the database.

[0046] The spectral angle mapping algorithm can ignore the differences in overall spectral brightness (amplitude) and focus only on the geometric features of the spectral curve, thereby accurately identifying chemical components.

[0047] In traditional static recognition, a fixed judgment threshold (e.g., 0.95) is usually set. However, in this embodiment, the conveyor belt is in a high-speed motion state, and the conveying speed affects the signal-to-noise ratio of the spectral acquisition. The faster the speed, the slightly higher the possibility of a single pixel mixing with background noise, which may cause a slight decrease in the calculated similarity value.

[0048] To prevent missed detections at high conveying speeds, a dynamic threshold strategy can be employed, adjusting the judgment threshold based on a preset linear or non-linear relationship. As the conveying speed V increases, the judgment threshold is automatically and finely lowered within a safe range.

[0049] The specific determination method is as follows: If the calculated similarity If the value is greater than or equal to the current dynamic threshold, the plastic packaging is determined to belong to the corresponding material, and a material category label containing the specific material name (such as PET) is generated.

[0050] If the calculated similarity If the value is less than the dynamic threshold, the plastic packaging is determined to be an impurity or an unknown type, and other corresponding labels are generated.

[0051] The above judgments are used to generate material category labels.

[0052] In the embodiments of this application, the target spectral vector is compared using a spectral angle mapping algorithm, and identification is performed using the vibrational absorption characteristics of molecular bonds. This allows for the accurate differentiation of plastics with identical appearance and transparency but different chemical compositions (e.g., transparent PET bottles and transparent PVC bottles). Furthermore, the introduction of a dynamic threshold related to the conveying speed effectively offsets the slight signal quality attenuation caused by belt acceleration. This ensures that the conveying and sorting equipment does not mis-grab items during low-speed operation and does not miss items during high-speed operation, significantly improving the industrial applicability of the conveying and sorting equipment.

[0053] In step S150, geometric feature calculations are performed on the visual model data to generate geometric attribute labels.

[0054] In this embodiment, the closed contour of the plastic packaging is read from the visual model data. Based on the closed contour of the plastic packaging, the minimum bounding rectangle of the plastic packaging is calculated, where the minimum bounding rectangle refers to the rectangle with the smallest area that can completely enclose the two-dimensional contour of the plastic packaging. From this, two basic physical quantities are extracted: L (length): The length of the longest side of the smallest bounding rectangle; W (width): The length of the shorter side of the smallest bounding rectangle.

[0055] The actual projected area of ​​the plastic packaging is calculated by counting the number of pixels contained within the outline.

[0056] To numerically describe the morphological characteristics of plastic packaging, this embodiment calculates the following two core shape factors: Aspect Ratio (AR): Used to measure whether plastic packaging is elongated or rectangular, or square-shaped. The formula is: The larger the AR value, the thinner and longer the plastic packaging.

[0057] Fill Rate (FR): Used to measure whether plastic packaging is solid and regular or hollow / irregular.

[0058] The calculation formula is: Where L×W represents the area of ​​the smallest bounding rectangle, This represents the actual projected area of ​​the plastic packaging.

[0059] The closer to 1, the closer the plastic packaging is to a rectangle or circle (such as a complete bottle cap or a regular sheet). The smaller the value, the more broken the edges of the plastic packaging or the more holes (such as wrinkled film or damaged mesh) are inside.

[0060] A shape classification rule base is obtained based on training with a large number of samples. The calculated AR and FR are compared with the thresholds in the shape classification rule base to generate geometric attribute labels. The determination process can be as follows: Film Classification: If AR is in a high range (e.g., >3.0) and FR is low (e.g., <0.6), it indicates that the plastic packaging is slender and irregularly shaped, thus classifying it as a film and generating the corresponding geometric attribute label. This type of plastic packaging requires a smaller air volume during subsequent air jetting to prevent it from being blown away.

[0061] Bottle / Bottle Flake Classification: If AR is in a low range (e.g., 1.0) 2.5) If the FR is high (e.g., >0.8), it indicates that the plastic packaging has a full and regular shape, and is therefore determined to be a rigid bottle or a whole bottle, generating the corresponding geometric attribute label. This type of plastic packaging has a large mass, requiring more valves to be opened or higher air pressure to be applied.

[0062] Impurity determination: If the actual projected area of ​​the plastic packaging is less than the preset minimum area threshold, it is determined to be a tiny dust or noise and marked as an object to be ignored.

[0063] After the above judgment, geometric attribute labels can be generated.

[0064] In this embodiment, by distinguishing between the film and the bottle, the problem of the traditional one-size-fits-all cutting method is solved. For plastic packaging of the same material but different shapes, different air jet intensities can be matched accordingly, avoiding the phenomenon of blowing the film everywhere or not blowing heavy bottle pieces. Moreover, by using aspect ratio and fill rate, two factors with minimal computational cost but significant features, the use of complex deep learning image segmentation networks is avoided, ensuring real-time response capability on high-speed production lines.

[0065] In step S160, the geometric center coordinates corresponding to the geometric attribute labels are mapped to the coordinate system where the hyperspectral data cube is located. If the material category label and the geometric attribute label are in the same space, an identification data package is generated.

[0066] In this embodiment, industrial cameras (RGB) typically have high spatial resolution, while hyperspectral imagers have relatively low spatial resolution. Furthermore, there is a physical installation distance ΔD between the two in the belt running direction.

[0067] The coordinate system containing the hyperspectral data cube is used as the reference coordinate system. The geometric center coordinates of the plastic packaging are denoted as P. rgb (x, y). Spatiotemporal compensation is performed on this coordinate using the transport speed V. The compensation formula is: Where: Y hsi : The vertical coordinate mapped to the hyperspectral coordinate system; Y rgb : Vertical coordinate in the original visual model; ΔD: Physical distance between the two cameras; V: Current conveyor speed of the belt; The ratio of the resolution of the two cameras.

[0068] This mapping projected the high-precision visual contour center onto the pixel grid of the hyperspectral data, thus finding its corresponding spectral position.

[0069] In this embodiment, to avoid mismatching the shape of the foreground plastic packaging with the material signal in the background, it is necessary to perform spatial consistency verification on the material category label and the geometric attribute label.

[0070] Check whether the generated material category label is valid within the preset neighborhood around the mapped geometric center point, i.e., whether it is a known material and not the background or impurities.

[0071] If at the mapped coordinate point (X hsi Y hsi If a material category label exists within the area and its neighborhood, for example, if a spectral signal of PET is detected and the overlap (IoU) between the area covered by the spectral signal and the mapped geometric contour exceeds a preset threshold, then it is determined that the space of the material category label and the geometric attribute label is consistent.

[0072] If the mapped location is the background (without a material label), or the material label is displayed as impurities, it is determined that the space of the material category label and the geometry attribute label is inconsistent, and the object will be ignored or marked as an anomaly.

[0073] Spatial consistency verification, where the material category label and geometric attribute label are spatially consistent, confirms that the physical object possesses both specific geometric attributes (e.g., bottle flakes) and a specific material category (e.g., PET). The scattered information is then packaged to generate the final identification data packet. This identification data packet is a structured instruction set containing the following fields: Object ID: A unique tracking number for the object; Material Type: Material category (e.g., PET); Geometry Type: Geometry attribute (e.g., Film / Bottle); Center Pos: Precise position coordinates in a unified coordinate system; Time Stamp: The estimated time stamp of arrival at the jet array after velocity compensation.

[0074] Finally, a complete identification data package was generated, which contains a dataset of complete feature attributes (identity + location + shape) of plastic packaging, so as to facilitate subsequent sorting or other operations.

[0075] In this embodiment, by introducing the conveyor speed parameter into the coordinate mapping, the data asynchrony problem caused by belt speed fluctuations or camera installation distance is dynamically compensated. Even if the belt speed fluctuates, the geometric properties and materials can be accurately aligned, avoiding misalignment recognition.

[0076] The dual verification mechanism based on spatial consistency filters out noise from a single sensor. For example, if an industrial camera mistakenly identifies a reflective metal sheet as a bottle shape, but the hyperspectral camera does not detect the spectrum of plastic packaging at the corresponding location, it will intercept the image and not generate an identification data packet, thus preventing subsequent malfunctions of the gas valve.

[0077] In one embodiment of this application, it further includes: When the similarity coefficient between the target spectral vector and any standard derivative spectral vector is less than the dynamic classification threshold, principal component analysis is performed on the spectral data of the region to obtain the first k principal component feature vectors. The first k principal component feature vectors are input into the trained random forest classification model to obtain stacked state labels; Write the stack status label into the identification data packet.

[0078] In real-world industrial sorting scenarios, plastic packages transported on conveyor belts are not necessarily isolated; often, bottle fragments overlap or labels cover the fragments. In this case, the collected hyperspectral data is not the spectrum of a single material, but rather a mixture and superposition of the spectra of two or more materials, resulting in distorted spectral curves. If only step-by-step spectral angle mapping algorithms are used to measure the similarity between the target spectral vector and each standard spectral vector in the standard spectral feature library, the similarity of this mixed spectrum to the standard spectrum of any single material will be low (i.e., less than the dynamic classification threshold), making it easily misclassified as impurities.

[0079] In this embodiment, when the similarity coefficient between the target spectral vector and any standard derivative spectral vector in the database is less than the current dynamic classification threshold, the region is determined to be a difficult region. At this point, the spectrum is no longer treated as a simple curve, but rather as a high-dimensional data point (dimension equal to the number of bands). To extract potential material combination features from the chaotic mixed signal, principal component analysis (PCA) is performed on the spectral data of this region.

[0080] The core of principal component analysis lies in dimensionality reduction and noise reduction. Due to the massive amount of hyperspectral data (e.g., 500 bands), directly inputting it into a classifier can lead to the curse of dimensionality.

[0081] Calculate the covariance among the bands of the spectral data for this region to construct a covariance matrix. Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. Based on the eigenvalues ​​in descending order, select the first k eigenvectors (i.e., principal component directions). Project the original high-dimensional spectral data onto these k directions to generate the first k principal component eigenvectors.

[0082] Typically, selecting the first 3-5 principal components can contribute more than 95% of the variance information. These principal components reflect the most drastically changing features in the mixed spectrum (such as the main absorption peak combination patterns) and eliminate minor random noise.

[0083] Although the extracted first k principal component feature vectors contain key information, their physical meaning is abstract and cannot be directly understood. Therefore, machine learning models are needed for decoding.

[0084] This embodiment employs a Random Forest classification model. Specifically, it collects a large amount of known historical stacking sample data (e.g., PET on top of PVC, PP on top of PE, label paper on top of bottle flakes) and unknown impurity data. Using the principal components of these samples as input and manually labeled stacking states as output, a Random Forest model composed of multiple decision trees is trained and generated.

[0085] The first k principal component feature vectors are input into the trained random forest classification model. Each decision tree in the random forest classification model votes, and the class with the most votes is output as the stacking state label. The specific content of this label may include: PET / PVC stack, label / PET stack, or unknown impurities.

[0086] The inferred stacking status label is written into the identification data packet. Specifically: if the label indicates a stack containing the target material (such as a PET / PVC stack), special strategies may be adopted during subsequent air jet control (e.g., increasing the blowing force to blow both out simultaneously, or performing secondary sorting at the end). If the label indicates an unknown impurity, it is marked as ignored in the data packet, and the air jet device will not take any action, allowing it to fall naturally.

[0087] Traditional techniques often result in errors or misclassifications when encountering mixed spectra caused by overlapping materials. In this embodiment, principal component analysis (PCA) is used to extract mixed features, combined with the powerful nonlinear classification capabilities of the random forest distribution model. This effectively identifies the composition of overlapping materials (e.g., recognizing PET and PVC stacked together), significantly improving sorting recall. PCA is used for dimensionality reduction, inputting only the top k most critical features into the random forest distribution model, greatly reducing the computational load on the machine learning model. This allows complex AI inference processes to be completed in real-time on high-speed production lines (milliseconds), meeting the pace requirements of industrial production. Furthermore, conventional linear matching often fails to identify bottles covered with labels or slightly contaminated surfaces. The random forest distribution model in this embodiment can mitigate errors caused by stacking, improving resource recycling rates.

[0088] In one embodiment of this application, converting raw byte data to obtain a floating-point array includes: In the case where no valid material category label or non-stacked label was generated after performing spectral analysis on the preprocessed hyperspectral data and comparing it with the standard spectral library, the image data was reconstructed to obtain a three-dimensional point cloud model of the plastic packaging to be identified. Calculate the volume of the 3D point cloud model and the estimated mass of the plastic packaging to obtain the spatial packing density; By inputting the spatial packing density into the transfer learning model, the predicted potential material type is obtained.

[0089] In this embodiment, spectral features are extracted from the preprocessed hyperspectral data, and the extracted feature curves are matched with benchmark data in a standard spectral library using Euclidean distance or spectral angle. The effective material category label is the material result output when the spectral matching similarity is higher than a preset first confidence threshold (e.g., 95%). The non-stacked state label is generated by analyzing the plastic packaging outline through image morphological processing (such as edge detection or instance segmentation algorithms). When the plastic packaging outline is identified as closed and without pixel adhesion or occlusion to other plastic packaging, an identifier indicating that the plastic packaging is in a separate, individual state is generated.

[0090] When a valid material category label is not generated (i.e., the spectral characteristics are fuzzy or it belongs to an unknown new material), and a non-stacked label is detected at the same time, it is determined that the conditions for performing geometric and physical analysis are met, and the 3D reconstruction process is automatically triggered.

[0091] In this embodiment, by using a dual-condition triggering mechanism such as material category label analysis and stacking state analysis, we can avoid performing high-computation 3D reconstruction on all plastic packaging (saving resources) and ensure that auxiliary recognition is activated only when the spectrum is ineffective and physical conditions permit (no obstruction), thereby improving recognition efficiency and accuracy.

[0092] After triggering the reconstruction command, the image data undergoes depth analysis. In this embodiment, the image data preferably includes structured light data or binocular vision data synchronized with the position of the hyperspectral imaging unit. Using a motion recovery structure algorithm or a stereo matching algorithm based on the parallax principle, the pixel coordinates of the two-dimensional image are transformed into three-dimensional spatial coordinates, thereby reconstructing the surface geometry of the plastic packaging to be identified, resulting in a three-dimensional point cloud model. This three-dimensional point cloud model is represented by the set P = {p1, p2, ..., pn}, where each point pi contains spatial coordinate information (x, y, pn). i y i , z i This expands the original two-dimensional data, which only contained spectral information, into three-dimensional data that includes spatial geometric information, providing a basis for calculating the physical volume of plastic packaging that cannot be identified by optical surface features.

[0093] The 3D point cloud model is voxelized or a Delaunay triangulation is constructed to form a closed convex hull. The volume of the closed geometry is calculated and denoted as V. cloud It should be noted that the volume V of a closed geometric solid... cloud It characterizes the spatial envelope volume of plastic packaging.

[0094] Obtain the mass estimate M of the plastic packaging to be identified. est By deploying a sequence of dynamic weighing sensors at the bottom of the transmission device, gravity signals are synchronously collected and converted into mass estimates as the plastic package passes through the imaging area.

[0095] Calculate the spatial packing density ρ. The calculation formula is as follows: The calculated spatial packing density differs from the theoretical microscopic density of a material. It comprehensively reflects the material density and macroscopic structure of an object (such as hollow or porous). This physical property has extremely high discernibility when the spectral characteristics are contaminated (such as plastic bottles and mud lumps covered with soil), and can effectively distinguish objects that look similar but have different textures.

[0096] The calculated spatial packing density ρ is used as the core feature and input into the transfer learning model to obtain the predicted potential material type.

[0097] Since the number of samples of unknown or unusual materials in industrial scenarios is usually small, directly training deep neural networks can easily lead to overfitting. This embodiment employs a transfer learning strategy: using a large amount of existing labeled data of standard materials (such as standard plastics, metals, and wood) to train the base model, learning the physical mapping between density, volume, and material category. For the small number of unknown materials in the current identification task, some parameters of the base model are frozen, and the output layer is fine-tuned using a small number of samples. A feature vector F=[ρ, S] is constructed, where S is the secondary spectral features remaining from the spectral analysis stage (i.e., spectral features that do not reach the threshold but still contain some information). The vector F is input into the model, and the material category with the highest probability is output. Transfer learning solves the cold-start identification problem in the case of small samples or zero samples.

[0098] In one embodiment of this application, constructing a hyperspectral data cube from hyperspectral data according to the transmission speed includes: The spatial displacement is calculated by using the transport speed and the row sampling frequency of the hyperspectral imaging device to analyze two adjacent hyperspectral frames of data. Based on the spatial displacement, multiple consecutively acquired hyperspectral frames are stitched together in the spatial dimension to obtain a hyperspectral data cube with both spatial and spectral dimensions.

[0099] In the embodiments of this application, the hyperspectral imaging device preferably employs a pushbroom imaging mode. In this mode, the device does not capture the entire object at once, but rather continuously acquires local linear regions of the object in a time sequence as the object moves.

[0100] Wherein: the conveying speed *v* refers to the physical speed at which the object to be identified moves along a predetermined direction within the imaging area by the conveying device (such as a conveyor belt), typically measured in millimeters per second (mm / s). The line sampling frequency *f* refers to the number of image frames (i.e., single-line spectral images) acquired by the hyperspectral imaging device per unit time, measured in Hertz (Hz). Based on the above conveying speed *v* and sampling frequency *f*, the spatial displacement Δd between two adjacent hyperspectral frame data is calculated. Hyperspectral frame data refers to the data acquired at a certain moment when the slit of the imaging device is aligned with a linear region on the surface of the object. The hyperspectral frame data contains the spatial position information (i.e., X-axis information perpendicular to the direction of movement) and spectral intensity information of this linear region. Since the object physically moves within the time interval t = 1 / f between two adjacent sampling intervals, this movement distance is the spatial displacement. The calculation formula is as follows: Where Δd is the physical resolution (or pixel equivalent length) of the reconstructed image in the transport direction (Y-axis).

[0101] By calculating the spatial displacement, the actual physical length represented by each frame of the image can be quantified. This is the basis for subsequent image reconstruction, ensuring that a pixel in the digital image accurately corresponds to a physical region of a specific length on the object's surface, preventing geometric distortions such as stretching or compression in the longitudinal direction caused by changes in transport speed.

[0102] After obtaining multiple consecutively acquired hyperspectral frame data and their corresponding spatial displacements, a data reconstruction process is performed to obtain a hyperspectral data cube.

[0103] The specific steps are as follows: Each frame of hyperspectral data is essentially a two-dimensional matrix (x, λ), where x represents the spatial dimension perpendicular to the transmission direction (i.e., the scan line width), and λ represents the spectral band dimension.

[0104] Using the transmission direction as the second spatial dimension (i.e., the Y-axis), the continuously acquired hyperspectral data in time sequence are arranged and stacked in an orderly manner in space based on the calculated spatial displacement Δd.

[0105] If the transmission speed is constant, the data from each frame can be stitched together sequentially according to the time sequence to form a continuous two-dimensional spatial image.

[0106] If the conveying speed v fluctuates, the data is interpolated or resampled using the real-time calculated Δd to ensure that the scale of the generated image is consistent in the Y-axis direction.

[0107] After the above stitching process, a three-dimensional data structure (x, y, λ) is finally formed, which is the hyperspectral data cube.

[0108] x-axis (row dimension): corresponds to the physical width of the slit in the hyperspectral camera.

[0109] y-axis (column dimension): corresponds to the longitudinal length of the object as it moves along the conveyor belt, and is composed of multiple frames of data stitched together.

[0110] λ axis (spectral dimension): corresponds to the spectral response value of each spatial pixel at different wavelengths.

[0111] The method in this embodiment transforms the originally discrete time series data into spatial entity data with strict topological relationships. It can not only analyze the spectral composition of a single point, but also analyze the spatial features such as texture and shape of the object surface. Moreover, it provides rich basic data for subsequent algorithms (such as convolutional neural networks), enabling the model to simultaneously utilize the physical and chemical properties (spectral features) of the object material and the geometric properties (spatial features) of the appearance for high-precision classification and recognition.

[0112] In one embodiment of this application, spectral dimension normalization is performed on the hyperspectral data cube to generate a target spectral vector, including: Extract each specified band from the hyperspectral data cube to obtain a single-band two-dimensional image; Performing a two-dimensional Fourier transform on a single-band two-dimensional image will yield frequency domain data. High-frequency components with concentrated energy and vertical distribution are detected in frequency domain data to obtain the strip noise frequency; A band-stop filter is constructed to set the amplitude corresponding to the frequency of strip noise to zero, thus obtaining the filtered frequency domain data. Perform a two-dimensional inverse Fourier transform on the filtered frequency domain data to generate a denoised hyperspectral data cube; The spectral dimension of the denoised hyperspectral data cube is normalized to generate the target spectral vector.

[0113] Since the hyperspectral data cube contains a spatial dimension (x, y) and a spectral dimension vector (λ), it is first sliced ​​for dimensionality reduction to utilize two-dimensional image processing algorithms. The slices are arranged according to the order of the spectral bands (λ1, λ2, ..., λ...). n The data planes under each specified band are locked and extracted one by one to obtain a series of single-band two-dimensional images. Each single-band two-dimensional image represents the spatial reflection intensity distribution of the object to be identified at a specific wavelength.

[0114] To separate effective texture information from periodic noise information in an image, a two-dimensional Fourier transform (2D-FFT) is performed on each extracted single-band two-dimensional image. Using the principle of Fourier transform, the image is mapped from the spatial domain to the frequency domain. In the transformed frequency domain data, the low-frequency components of the image (representing contours and background) are typically concentrated in the center of the spectrum, while the high-frequency components (representing edges, details, and noise) are distributed around the periphery of the spectrum. This transforms spatially indistinguishable superimposed noise into specific frequency points with identifiable locations in the frequency domain, providing a foundation for accurate noise reduction.

[0115] In frequency domain data, noise feature identification is crucial. Striped noise is typically caused by inconsistent detector element responses in pushbroom hyperspectral cameras, manifesting as bright and dark fringes along the scanning direction in the spatial domain. Based on Fourier transform properties, periodic fringes in the spatial domain are mapped to bright spots or lines with extremely high energy and a specific directional distribution in the frequency domain. High-frequency components with concentrated energy and vertical distribution are detected. These high-frequency components appear as vertical line segments or discrete point sequences with significantly high amplitudes, deviating from the central origin, on the spectrum; these are labeled as striped noise frequencies.

[0116] A band-stop filter is dynamically constructed based on the location coordinates of the detected stripe noise frequencies. Essentially, the band-stop filter is a mask matrix with the same dimension as the frequency domain data. In the mask matrix, the positions corresponding to the stripe noise frequency coordinates are assigned extremely small weight coefficients (e.g., 0), while other positions are assigned weight coefficients of 1. A dot product operation is performed between the frequency domain data and the band-stop filter, setting the amplitude corresponding to the stripe noise frequency to zero or significantly attenuating it, thus obtaining the filtered frequency domain data. This eliminates specific frequency noise caused by instrument system errors while preserving the edge and texture information of the object itself to the greatest extent possible.

[0117] A two-dimensional inverse Fourier transform (2D-IFFT) is performed on the filtered frequency domain data to restore it to the spatial domain, generating a denoised single-band image. This process is repeated for all bands, and all processed single-band images are stacked and reassembled along the spectral dimension according to their original spectral order, ultimately generating a denoised hyperspectral data cube. At this point, the bright and dark stripes in the image have been eliminated, and the image quality has been significantly improved.

[0118] To eliminate differences in reflected light intensity caused by the angle of illumination from the light source and the unevenness of the object's surface (i.e., to eliminate brightness interference in the phenomenon of different spectra of the same object), the denoised hyperspectral data cube is subjected to spectral dimension normalization. Specifically, for each spatial pixel (x, y) in the cube, its corresponding spectral curve data S=[s1, s2, ..., s...] is extracted. n The vector S is normalized (e.g., L2 norm normalization) so that the magnitude of the processed spectral vector is 1, or it is mapped to the interval [0, 1].

[0119] Standardization transforms the absolute intensity differences in spectral data into relative waveform differences, allowing subsequent classification models to focus more on the spectral fingerprint (waveform features) of the material itself rather than brightness features, thereby improving the robustness of recognition in complex lighting environments.

[0120] In one embodiment of this application, spectral dimension normalization is performed on the denoised hyperspectral data cube to generate a target spectral vector, including: Each pixel in the denoised hyperspectral data cube is extracted to obtain the original spectral curve; The original spectral curve is transformed using a standard normal variable to obtain the spectral intensity of the original spectral curve; The spectral intensity of the original spectral curve is calculated to obtain the mean and standard deviation; The normalized spectral curve is obtained by subtracting the mean spectral intensity from the intensity value of each wavelength point in the original spectral curve and then dividing by the standard deviation. The first derivative of the fitted polynomial is obtained by performing a convolution operation on the normalized spectral curve using a filtering algorithm. The target spectral vector is generated based on the first derivative.

[0121] In this embodiment, the denoised hyperspectral data cube is analyzed pixel by pixel. For any spatial pixel (x, y) in the hyperspectral data cube, its response values ​​in all bands are extracted along the spectral dimension and combined to form the original spectral curve. This curve is usually represented as a vector R = [r1, r2, ..., r...]. n ], where r i This represents the spectral reflectance or absorbance of the pixel at the i-th wavelength.

[0122] Scattering correction based on Standard Normal Variation (SNV) aims to eliminate spectral tilt or shift caused by varying scattering efficiencies of object surfaces. Statistical analysis is performed on the spectral intensity of the original spectral curve. Specifically, the mean μ and standard deviation σ of the intensity values ​​of all bands in the spectral vector R corresponding to the pixel are calculated. The mean μ reflects the overall brightness level (baseline shift) of the spectral curve. The standard deviation σ reflects the overall fluctuation range (gain variation) of the spectral curve.

[0123] Based on the above statistics, the intensity value r at each wavelength point in the original spectral curve i Perform standardization calculations. Specifically, subtract the mean spectral intensity from the wavelength intensity value, divide the difference by the standard deviation, and obtain the standard normal variable transformation value. The calculation formula is as follows: After this step, the normalized spectral curve is obtained.

[0124] By employing the aforementioned standard normal variable transformation, spectral multiplicative and additive interferences caused by inconsistent particle sizes, different scattering angles, or changes in optical path on the surface of the object to be identified are effectively corrected. This ensures that the processed spectral features are only related to the chemical composition of the object, and are decoupled from the physical surface state of the object.

[0125] To further eliminate low-frequency background noise (such as baseline drift) and sharpen weak peak features in the spectrum, a filtering algorithm is used to process the normalized spectral curve. In this embodiment, the Savitzky-Golay (SG) filtering algorithm is preferably used. The specific operation is as follows: A fixed-width sliding window is defined, which moves point by point along the normalized spectral curve. Within each window, the spectral data points are fitted to a low-order polynomial (e.g., a quadratic or cubic polynomial) using the least squares method. This process is achieved by convolving pre-calculated convolution coefficients with the data within the window, eliminating the need for explicit equation solving and thus ensuring computational speed. Based on the fitted local polynomial mathematical expression, the first derivative of the polynomial at the center point of the window is directly calculated analytically. After traversing the entire curve, a new curve is output, composed of the derivative values ​​at all center points. The polynomial fitting process utilizes neighborhood information, effectively smoothing out random high-frequency noise. Calculating the first derivative effectively eliminates constant baseline shifts and linear background interference in the spectrum (because the derivative of a constant is 0, and the derivative of a linear term is constant), thereby highlighting the feature peaks and valleys with the largest rates of change in the spectral curve and significantly improving feature resolution.

[0126] The first-order derivative data sequence is determined as the target spectral vector. The target spectral vector is a collection of high-dimensional features after denoising, scattering correction and differential enhancement (SG derivative) of standard normal variables, which serves as the standard input data for the subsequent transfer learning model, greatly improving the convergence speed and classification accuracy of the material recognition model.

[0127] In one embodiment of this application, the geometric center coordinates corresponding to the geometric attribute labels are mapped to the coordinate system where the hyperspectral data cube is located. When the material category labels and geometric attribute labels are in the same spatial location, generating the identification data package includes: The closed contours in the visual model data are extracted to obtain the geometric center coordinates corresponding to the geometric attribute labels; Using a preset coordinate transformation matrix, the geometric center coordinates corresponding to the geometric attribute labels are transformed to the pixel coordinate system of the hyperspectral data cube to obtain the projection center coordinates; Based on the hyperspectral data cube, generate the effective spectral pixel region for the material category label; The effective spectral pixel region of the material category label is calculated to obtain the spectral weighted center coordinates of the effective spectral pixel region; The Euclidean distance between the coordinates of the projected centroid and the coordinates of the spectral weighting center is calculated to obtain the Euclidean distance; If the Euclidean distance is less than the preset tolerance radius, the material category label and the geometric attribute label will satisfy spatial consistency and generate an identification data package.

[0128] In this embodiment, the visual model data is parsed. This visual model data originates from a visual sensor (such as a high-resolution industrial camera or a 3D profilometer) that acquires data synchronously with a hyperspectral imaging device. The processing unit performs edge detection or connected component analysis on the visual model data to extract the closed contour of the object to be identified. This closed contour represents the physical boundary of the object. The geometric centroid or geometric center of this closed contour is calculated to obtain the geometric center coordinates (x, y, y) corresponding to the geometric attribute label. vis y vis Geometric attribute labels are identifiers generated by industrial cameras after preliminary inspection of objects based on their appearance and shape. They are associated with the object's shape, size, and location information.

[0129] Due to spatial differences in physical installation (such as positional offset and angular rotation) between the industrial camera and the hyperspectral imager, their coordinate systems do not coincide. A preset coordinate transformation matrix is ​​used to map and transform the geometric center coordinates. This preset coordinate transformation matrix, obtained beforehand through calibration on a calibration target, describes the mapping relationship between the view planes of the hyperspectral imager and the industrial camera. Through the preset coordinate transformation matrix operation, the geometric center coordinates of the industrial camera in the visual coordinate system are transformed to the pixel coordinate system of the hyperspectral data cube, obtaining the projection center coordinates (x...). proj y proj This process maps the physical location of objects to the spectral data space, providing a unified metric for subsequent location comparisons. Based on the material category labels (e.g., PE plastic or 304 stainless steel) generated during the hyperspectral analysis phase, a reverse index is performed on the spatial plane of the hyperspectral data cube. All pixels classified under that material category label are selected; these pixels form a connected or disconnected set in space, thus generating the effective spectral pixel region. This region reflects the actual spatial distribution range of the object's material as perceived by the hyperspectral imager.

[0130] To more accurately characterize the core location of the material distribution, instead of simply calculating the geometric center of the effective spectral pixel region, we calculate the spectral weighted center coordinates (x... spec y spec For each pixel within the region, its classification confidence score or spectral response intensity is obtained as a weight value w. i Calculate the center using the weighted average formula: Among them, (x i y i ) represents the coordinates of the i-th pixel within the region.

[0131] Compared to ordinary geometric centers, spectral weighted centers are more resistant to interference. For example, when spectral mixing (mixed pixels) at the object's edges leads to inaccurate classification, the weight of edge pixels is reduced, and the calculated center is more likely to be the core of the object with pure material, thus improving the robustness of localization.

[0132] For the projection center coordinates (x proj y proj ) and the coordinates of the spectral weighting center (x spec y spec The Euclidean distance D is obtained by calculating the straight-line distance between the two points. The calculation formula is: This Euclidean distance quantifies the degree of deviation between the position of an object in the image and the position of the material as measured by the spectrum.

[0133] The calculated Euclidean distance is compared with a preset tolerance radius. If the Euclidean distance is less than the preset tolerance radius, the material category label (spectral result) and the geometric attribute label (visual result) are determined to have spatial consistency. This means that the plastic packaging in the image is the same plastic packaging identified by hyperspectral analysis, and the two are successfully matched.

[0134] Visual geometric information (volume, shape) and hyperspectral material information (composition, category) are encapsulated to generate the final identification data packet, which is then output to the downstream sorting execution mechanism.

[0135] When objects slide on the conveyor belt, images are misaligned, or foreign objects interfere (such as stains on the belt being mistakenly identified as material), the distance between the projection center and the spectral center becomes too large (i.e., D > tolerance radius). In this case, data packets will be refused to be generated, thus avoiding incorrect sorting and significantly improving the purity and accuracy of industrial sorting.

[0136] Secondly, embodiments of this application provide a plastic packaging identification system, including: The first determining module is used to acquire hyperspectral data of the plastic packaging, image data of the plastic packaging, and the conveying speed of the plastic packaging; The first generation module is used to construct a hyperspectral data cube from the hyperspectral data according to the transmission speed, and to perform spectral dimension normalization on the hyperspectral data cube to generate the target spectral vector. The second generation module is used to correct the image data and generate visual model data; The third generation module is used to perform spectral feature matching on the target spectral vector and generate material category labels; The fourth generation module is used to perform geometric feature calculations on the visual model data and generate geometric attribute labels; The fifth generation module is used to map the geometric center coordinates corresponding to the geometric attribute labels to the coordinate system where the hyperspectral data cube is located, and generate an identification data package when the space of the material category label and the geometric attribute label are consistent.

[0137] In this embodiment, the method for identifying plastic packaging includes: constructing a hyperspectral data cube based on the transport speed from hyperspectral data; standardizing the spectral dimensions of the hyperspectral data cube to generate a target spectral vector; correcting the image data to generate visual model data; performing spectral feature matching on the target spectral vector to generate a material category label; performing geometric feature calculation on the visual model data to generate a geometric attribute label; mapping the geometric center coordinates corresponding to the geometric attribute label to the coordinate system of the hyperspectral data cube; and generating an identification data package when the space of the material category label and the geometric attribute label is consistent. This solves the problems of single vision not being able to identify materials and single spectrum not being able to distinguish shapes. By comparing geometric attributes and material types at the same spatial location, it effectively eliminates the problem of misidentification of plastic packaging due to reflections or ghosting, thus improving the identification accuracy of plastic packaging. It effectively solves the problem in current plastic packaging visual analysis technology that the reflective characteristics of the material surface, similar colors, and diverse textures often make it difficult to accurately distinguish different types of plastic materials.

[0138] In one embodiment of this application, it further includes: When the similarity coefficient between the target spectral vector and any standard derivative spectral vector is less than the dynamic classification threshold, principal component analysis is performed on the spectral data of the region to obtain the first k principal component feature vectors. The first k principal component feature vectors are input into the trained random forest classification model to obtain stacked state labels; Write the stack status label into the identification data packet.

[0139] In one embodiment of this application, converting raw byte data to obtain a floating-point array includes: In the case where no valid material category label or non-stacked label was generated after performing spectral analysis on the preprocessed hyperspectral data and comparing it with the standard spectral library, the image data was reconstructed to obtain a three-dimensional point cloud model of the plastic packaging to be identified. Calculate the volume of the 3D point cloud model and the estimated mass of the plastic packaging to obtain the spatial packing density; By inputting the spatial packing density into the transfer learning model, the predicted potential material type is obtained.

[0140] In one embodiment of this application, constructing a hyperspectral data cube from hyperspectral data according to the transmission speed includes: The spatial displacement is calculated by using the transport speed and the row sampling frequency of the hyperspectral imaging device to analyze two adjacent hyperspectral frames of data. Based on the spatial displacement, multiple consecutively acquired hyperspectral frames are stitched together in the spatial dimension to obtain a hyperspectral data cube with both spatial and spectral dimensions.

[0141] In one embodiment of this application, spectral dimension normalization is performed on the hyperspectral data cube to generate a target spectral vector, including: Extract each specified band from the hyperspectral data cube to obtain a single-band two-dimensional image; Performing a two-dimensional Fourier transform on a single-band two-dimensional image will yield frequency domain data. High-frequency components with concentrated energy and vertical distribution are detected in frequency domain data to obtain the strip noise frequency; A band-stop filter is constructed to set the amplitude corresponding to the frequency of strip noise to zero, thus obtaining the filtered frequency domain data. Perform a two-dimensional inverse Fourier transform on the filtered frequency domain data to generate a denoised hyperspectral data cube; The spectral dimension of the denoised hyperspectral data cube is normalized to generate the target spectral vector.

[0142] In one embodiment of this application, spectral dimension normalization is performed on the denoised hyperspectral data cube to generate a target spectral vector, including: Each pixel in the denoised hyperspectral data cube is extracted to obtain the original spectral curve; The original spectral curve is transformed using a standard normal variable to obtain the spectral intensity of the original spectral curve; The spectral intensity of the original spectral curve is calculated to obtain the mean and standard deviation; The normalized spectral curve is obtained by subtracting the mean spectral intensity from the intensity value of each wavelength point in the original spectral curve and then dividing by the standard deviation. The first derivative of the fitted polynomial is obtained by performing a convolution operation on the normalized spectral curve using a filtering algorithm. The target spectral vector is generated based on the first derivative.

[0143] In one embodiment of this application, the geometric center coordinates corresponding to the geometric attribute labels are mapped to the coordinate system where the hyperspectral data cube is located. When the material category labels and geometric attribute labels are in the same spatial location, generating the identification data package includes: The closed contours in the visual model data are extracted to obtain the geometric center coordinates corresponding to the geometric attribute labels; Using a preset coordinate transformation matrix, the geometric center coordinates corresponding to the geometric attribute labels are transformed to the pixel coordinate system of the hyperspectral data cube to obtain the projection center coordinates; Based on the hyperspectral data cube, generate the effective spectral pixel region for the material category label; The effective spectral pixel region of the material category label is calculated to obtain the spectral weighted center coordinates of the effective spectral pixel region; The Euclidean distance between the coordinates of the projected centroid and the coordinates of the spectral weighting center is calculated to obtain the Euclidean distance; If the Euclidean distance is less than the preset tolerance radius, the material category label and the geometric attribute label will satisfy spatial consistency and generate an identification data package.

[0144] The functions of each module in each device in the embodiments of this application can be found in the corresponding descriptions in the above methods, and will not be repeated here.

[0145] Figure 2 A structural block diagram of an electronic device according to an embodiment of this application is shown. Figure 2 As shown, the electronic device includes a memory 410 and a processor 420. The memory 410 stores instructions that can be executed on the processor 420. When the processor 420 executes the instructions, it implements the plastic packaging identification method in the above embodiments. The number of memories 410 and processors 420 can be one or more. This electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0146] The electronic device may also include a communication interface 430 for communicating with external devices and exchanging data. The devices are interconnected using different buses and can be mounted on a common motherboard or otherwise as needed. The processor 420 can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). The bus can be divided into address buses, data buses, control buses, etc. For ease of illustration, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0147] Optionally, in a specific implementation, if the memory 410, processor 420 and communication interface 430 are integrated on a single chip, the memory 410, processor 420 and communication interface 430 can communicate with each other through an internal interface.

[0148] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0149] This application provides a computer-readable storage medium (such as the memory 410 described above) that stores computer instructions, which, when executed by a processor, implement the method provided in this application.

[0150] Optionally, memory 410 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. Furthermore, memory 410 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 410 may optionally include memory remotely located relative to processor 420, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0151] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0152] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0153] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more (two or more) executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0154] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0155] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0156] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0157] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for identifying plastic packaging, characterized in that, include: Acquire hyperspectral data, image data, and conveying speed of the plastic packaging; Based on the transmission speed, the hyperspectral data is constructed into a hyperspectral data cube, and the hyperspectral data cube is subjected to spectral dimension normalization to generate a target spectral vector. The image data is corrected to generate visual model data; Perform spectral feature matching on the target spectral vector to generate material category labels; Geometric feature calculations are performed on the visual model data to generate geometric attribute labels; The geometric center coordinates corresponding to the geometric attribute labels are mapped to the coordinate system where the hyperspectral data cube is located. When the space of the material category label and the geometric attribute label is consistent, an identification data package is generated.

2. The method according to claim 1, characterized in that, Also includes: When the similarity coefficient between the target spectral vector and any standard derivative spectral vector is less than the dynamic classification threshold, principal component analysis is performed on the spectral data of the region to obtain the first k principal component feature vectors. The first k principal component feature vectors are input into the trained random forest classification model to obtain stacked state labels; Write the stack status label into the identification data packet.

3. The method according to claim 2, characterized in that, Also includes: If the preprocessed hyperspectral data is subjected to spectral analysis and compared with a standard spectral library without generating a valid material category label and the non-stacked state label, the image data is reconstructed to obtain a three-dimensional point cloud model of the plastic packaging to be identified. The volume of the three-dimensional point cloud model and the estimated mass of the plastic packaging are calculated to obtain the spatial packing density; The spatial packing density is input into the transfer learning model to obtain the predicted potential material type.

4. The method according to claim 3, characterized in that, The step of constructing a hyperspectral data cube from the hyperspectral data according to the transmission speed includes: The spatial displacement is calculated by using the transport speed and the row sampling frequency of the hyperspectral imaging device to measure the data of two adjacent hyperspectral frames. Based on the spatial displacement, multiple consecutively acquired hyperspectral frames are stitched together in the spatial dimension to obtain a hyperspectral data cube with spatial and spectral dimensions.

5. The method according to claim 4, characterized in that, The step of performing spectral dimension normalization on the hyperspectral data cube to generate the target spectral vector includes: Extract each specified band from the hyperspectral data cube to obtain a single-band two-dimensional image; Performing a two-dimensional Fourier transform on the single-band two-dimensional image will yield frequency domain data; High-frequency components with concentrated energy and vertical distribution are detected in the frequency domain data to obtain the strip noise frequency; A band-stop filter is constructed to set the amplitude corresponding to the frequency of the strip noise to zero, thereby obtaining the filtered frequency domain data. Perform a two-dimensional inverse Fourier transform on the filtered frequency domain data to generate a denoised hyperspectral data cube; The denoised hyperspectral data cube is subjected to spectral dimension normalization to generate the target spectral vector.

6. The method according to claim 5, characterized in that, The step of performing spectral dimension normalization on the denoised hyperspectral data cube to generate the target spectral vector includes: Each pixel in the denoised hyperspectral data cube is extracted to obtain the original spectral curve; The original spectral curve is subjected to a standard normal transformation to obtain the spectral intensity of the original spectral curve; The spectral intensity of the original spectral curve is calculated to obtain the mean and standard deviation; The normalized spectral curve is obtained by subtracting the mean spectral intensity from the intensity value of each wavelength point in the original spectral curve and then dividing by the standard deviation. The normalized spectral curve is convolved using a filtering algorithm to obtain the first derivative of the fitted polynomial. The target spectral vector is generated based on the first derivative.

7. The method according to claim 6, characterized in that, The step of mapping the geometric center coordinates corresponding to the geometric attribute labels to the coordinate system where the hyperspectral data cube is located, and generating the identification data package when the space of the material category label and the geometric attribute label is consistent, includes: The closed contours in the visual model data are extracted to obtain the geometric center coordinates corresponding to the geometric attribute labels; Using a preset coordinate transformation matrix, the geometric center coordinates corresponding to the geometric attribute labels are transformed to the pixel coordinate system of the hyperspectral data cube to obtain the projection center coordinates; Based on the hyperspectral data cube, generate the effective spectral pixel region for the material category label; The effective spectral pixel region of the material category label is calculated to obtain the spectral weighted center coordinates of the effective spectral pixel region; The Euclidean distance between the projected centroid coordinates and the spectral weighting center coordinates is calculated to obtain the Euclidean distance; If the Euclidean distance is less than the preset tolerance radius, the material category label and the geometric attribute label satisfy spatial consistency, and an identification data packet is generated.

8. A plastic packaging identification system, characterized in that, include: The first determining module is used to acquire hyperspectral data of the plastic packaging, image data of the plastic packaging, and the conveying speed of the plastic packaging; The first generation module is used to construct a hyperspectral data cube from the hyperspectral data according to the transmission speed, and to perform spectral dimension normalization on the hyperspectral data cube to generate a target spectral vector. The second generation module is used to correct the image data and generate visual model data; The third generation module is used to perform spectral feature matching on the target spectral vector and generate a material category label; The fourth generation module is used to perform geometric feature calculation on the visual model data and generate geometric attribute labels; The fifth generation module is used to map the geometric center coordinates corresponding to the geometric attribute labels to the coordinate system where the hyperspectral data cube is located, and generate an identification data package when the space of the material category label and the geometric attribute label is consistent.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-7.

Images