PET waste plastic intelligent sorting method and system based on multi-feature fusion

Abstract

The invention discloses a PET waste plastic intelligent sorting method and system based on multi-feature fusion, and the method comprises the following steps: collecting an RGB-HSI original data set of PET waste plastic, carrying out the dimension reduction processing of the HSI data set based on a genetic algorithm to generate MSI data, and forming a feature fusion data set with RGB data; constructing a feature fusion network based on a YOLACT model, and training by adopting a feature fusion data set; the feature fusion network comprises an RGB branch, an MSI branch and a fusion prediction module, RGB is used for extracting features of RGB data, the MSI branch is used for extracting features of MSI data, and the fusion prediction module is used for combining output of the RGB branch and output of the MSI branch to make comprehensive prediction; and installing the trained feature fusion network to an industrial personal computer to realize intelligent sorting of the PET waste plastics. Compared with a traditional recognition algorithm, the method can improve the recognition precision.

Description

Technical Field

[0001] The present invention relates to the field of hyperspectral imaging technology and multimodal fusion, and in particular to a method and system for intelligently sorting PET waste plastics based on multi-feature fusion. Background Art

[0002] With the continuous growth of global plastic consumption, the recycling and treatment of plastic waste has become an important issue in environmental protection. Among urban domestic waste, PET (polyethylene terephthalate) waste plastic has attracted much attention due to its wide application and recycling value. However, existing PET waste plastic recycling technologies mainly rely on manual sorting, simple mechanical screening or sensor sorting based on a single feature. These methods have problems such as low manual sorting efficiency, insufficient mechanical screening accuracy, and high sensor sorting cost.

[0003] Technological development trends show that with the development of artificial intelligence and machine learning technologies, the field of intelligent detection and sorting equipment for PET waste plastics is gradually transforming towards automation and intelligence. However, the existing technology has not yet provided an efficient, accurate and cost-effective solution, especially for the accurate identification and sorting of PET waste plastics in complex environments. Based on the deficiencies of the existing technology and market demand, the present invention proposes a PET waste plastic identification method based on multi-feature fusion to improve the identification and sorting efficiency and accuracy of PET waste plastics, achieve high-value recycling, and meet the growing needs of environmental protection and resource recovery. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the prior art.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a method for intelligent sorting of PET waste plastics based on multi-feature fusion, comprising the following steps:

[0006] Collect RGB data and HSI data of PET waste plastics, perform dimensionality reduction processing on HSI data based on genetic algorithm to generate MSI data, and generate pixel-aligned RGB-MSI multi-feature data sets;

[0007] A PET waste plastic intelligent sorting model is constructed based on the YOLACT model and trained using an RGB-MSI multi-feature data set; the PET waste plastic intelligent sorting model includes an RGB branch, an MSI branch, and a fusion prediction module, wherein RGB is used to process RGB data, the MSI branch is used to process MSI data, and the fusion prediction module is used to combine the processing results of the RGB branch and the processing results of the MSI branch to make a comprehensive prediction;

[0008] The trained PET waste plastic intelligent sorting model is used to realize the intelligent sorting of PET waste plastics.

[0009] Preferably, the dimensionality reduction processing of HSI data based on genetic algorithm to generate MSI data comprises the following steps:

[0010] S11, randomly select multiple independent samples in the HSI dataset by category, and extract n data points from the hyperspectral image of each sample to obtain C categories of hyperspectral datasets to form a C*N*M preliminary hyperspectral dataset, where N represents the number of samples in each category and M represents the number of bands in each category;

[0011] S12, binary encoding is performed on each band of the preliminary hyperspectral data set to form an initial population, and each individual in the initial population is a candidate band subset; the F1 value of the classification model is used as an evaluation index, and individuals with high fitness are selected through the roulette method, and the best individual is finally selected through crossover mutation, and the K bands represented by the individual are the best band features;

[0012] S13, selecting K characteristic bands corresponding to each HSI data according to the optimal band characteristics to generate MSI data.

[0013] Preferably, the HSI data set in S11 has 224 bands in the wavelength range of 935.9 nm to 1722.5 nm, and a spectral resolution of 3.5 nm;

[0014] When the hyperspectral camera collects HSI data, it is performed under the same parameter settings and external conditions as those for collecting standard diffuse reflection surface image data and dark current images, and the raw radiation value data is converted into reflectance data by black and white frame correction. λ :

[0015]

[0016] In the formula, S λ is the sample radiation value at wavelength λ, D λ is the dark current radiation value at wavelength λ, R λ is the diffuse reflection radiation value of the standard white plate surface at wavelength λ. The corrected reflectivity should be between 0% and 100%.

[0017] Preferably, the process of selecting the best band feature in S12 comprises the following steps:

[0018] S121. Express each band of the initially selected hyperspectral dataset using binary coding. When encoding, use a binary number of length r to represent all spectral bands, where the selected bands are represented by "1" and the unselected bands are represented by "0", i.e., the initial population. Denote the size of the initial population as M, and randomly generate M binary codes of length r as the initial population. Among them, the number of "1"s in each binary code is the same as the size of the target band subset K.

[0019] S122. Calculate the fitness of the individuals in the population, which is expressed as:

[0020]

[0021] where fitness i represents the fitness corresponding to individual i, and p i and r i represent the classification accuracy and recall rate corresponding to individual i respectively.

[0022] S123. Adopt the "roulette wheel" strategy to select an individual from the initial population. The probability of each individual being selected is expressed as:

[0023]

[0024] where, fitness i represents the fitness corresponding to individual i, and the individuals with higher fitness have a higher probability of being selected.

[0025] S124. On the premise that m < K, randomly select m dominant nodes from the paternal gene and the maternal gene respectively, and exchange the selected nodes one by one to generate new individuals to form a new population.

[0026] S125. Repeat steps S121 to S124 until the genetic algorithm meets the termination condition, and decode the individuals with high fitness to obtain K best band features.

[0027] Preferably, to avoid the genetic search falling into a local optimal solution, a mutation method of flipping some gene segments is also adopted; the gene segments refer to the bands with binary coding "1" in the individuals of the population.

[0028] Preferably, in S13, based on the selected K best band features, shear and splice the channel direction of the hyperspectral image, and output the MSI dataset in the format of Tif file. This MSI dataset has a one-to-one correspondence with the RGB dataset.

[0029] Preferably, the PET waste plastic intelligent sorting model is constructed based on the YOLACT model, and the single-input single-trunk network of the existing YOLACT target detection model is improved into a dual-input dual-trunk network, thereby constructing a FLF-YOLACT feature-level fusion network as the PET waste plastic intelligent sorting model; the FLF-YOLACT feature-level fusion network includes:

[0030] The RGB branch uses an independent ResNet residual network for feature extraction. After obtaining the RGB features, they are input into the feature pyramid and finally several RGB feature maps are obtained.

[0031] The MSI branch uses an independent ResNet residual network for feature extraction. After obtaining the MSI features, they are input into the feature pyramid and finally several MSI feature maps are obtained.

[0032] The fusion prediction module uses the Concat fusion method to splice the RGB feature map and the MSI feature map, and then uses the detection head to perform classification prediction based on the spliced ​​fusion features; the fusion features are expressed as:

[0033]

[0034] Among them, X i , Y i Represents the data of the i-th layer in different input sources, K i Represents the dimension position of the result in the output data.

[0035] Preferably, the PET waste plastic intelligent sorting model is constructed based on the YOLACT model, and the existing YOLACT target detection model is used as the RGB branch to realize the prediction of the input RGB data in the form of instance segmentation, and then an MSI branch is rebuilt based on the hyperspectral point classification algorithm to realize the pixel-level prediction of the input MSI data, thereby constructing a DF-YOLACT decision-level fusion network as the PET waste plastic intelligent sorting model; the DF-YOLACT decision-level fusion network includes:

[0036] The RGB branch uses the ResNet backbone network to extract RGB features, and then uses the FPN feature pyramid for information interaction, and finally generates RGB feature maps of different scales. A preliminary prediction is made based on the RGB feature map to obtain the prediction results of the RGB branch, including category, position, mask and confidence.

[0037] The MSI branch consists of an extractor and a classifier. The extractor extracts the spectral data corresponding to each target instance one by one based on the mask information of the target instance, and converts it into a point set, where each point represents the corresponding spectral information in the original image. The classifier identifies and classifies the obtained point sets one by one, and each point set obtains a corresponding category result set.

[0038] The fusion prediction module counts the number of categories in each category result set output by the MSI branch, regards the category with the largest number of occurrences as the category of the target instance corresponding to the category result set, and uses the proportion of the number of occurrences of this category as the confidence level. The prediction results generated by the RGB branch are integrated through the category and confidence level to obtain the final prediction result.

[0039] Preferably, the intelligent sorting of PET waste plastics using the trained PET waste plastics intelligent sorting model comprises the following steps:

[0040] Select the trained PET waste plastic intelligent sorting model according to the actual working conditions and deploy it to the industrial computer;

[0041] The on-site RGB camera and the hyperspectral camera jointly photographed the PET waste on the conveyor belt to obtain two sets of pixel-aligned pictures, which were simultaneously input into the industrial computer; the bands collected by the hyperspectral camera were the K best bands obtained based on the genetic algorithm;

[0042] The industrial computer uses the deployed model to perform classification prediction and obtain the coordinates and categories of PET waste plastics;

[0043] The trained PET waste plastic intelligent sorting model is selected according to the actual working conditions, specifically: if high sorting efficiency is required, the FLF-YOLACT feature-level fusion network is selected; if high sorting accuracy is required, the DF-YOLACT decision-level fusion network is selected.

[0044] The present invention also provides a PET waste plastic intelligent sorting system based on multi-feature fusion, comprising:

[0045] The data processing module collects RGB data and HSI data of PET waste plastics, performs dimensionality reduction processing on the HSI data based on the genetic algorithm to generate MSI data, and generates a pixel-aligned RGB-MSI multi-feature data set;

[0046] A model building module, which builds a PET waste plastic intelligent sorting model based on the YOLACT model and uses an RGB-MSI multi-feature data set for training; the PET waste plastic intelligent sorting model includes an RGB branch, an MSI branch, and a fusion prediction module, wherein RGB is used to process RGB data, the MSI branch is used to process MSI data, and the fusion prediction module is used to combine the processing results of the RGB branch and the processing results of the MSI branch to make a comprehensive prediction;

[0047] The waste plastic sorting module uses the trained PET waste plastic intelligent sorting model to realize the intelligent sorting of PET waste plastics.

[0048] The present invention has the following beneficial effects:

[0049] (1) The present invention adopts a characteristic band selection method based on a genetic algorithm for the band selection of PET waste plastic HSI, and uses the F1-score of the model classification as an evaluation index to automatically select the ideal characteristic band, thereby achieving a PET waste plastic classification effect similar to or even better than that of the full band while achieving a smaller amount of data, effectively solving the dimensionality disaster problem and improving data processing efficiency;

[0050] (2) The present invention combines the data of RGB images and hyperspectral images (HSI), giving full play to the information of RGB in image color, texture and space, and combines the accurate material information of the spectrum to achieve high-precision recognition of PET waste plastics, thereby improving the accuracy of recognition;

[0051] (3) The present invention adopts two different fusion models: feature-level fusion and decision-level fusion, which can select a more appropriate model for specific working conditions and improve the robustness of the model;

[0052] (4) Both models mentioned in the present invention are instance segmentation models and single-stage detection models. Compared with the two-stage instance segmentation model, they can achieve better recognition accuracy in a shorter time. Compared with the target detection model, they can provide finer segmentation boundaries, which is beneficial to the subsequent mechanical sorting and improves the recall rate of PET waste plastics.

[0053] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A method step diagram of an embodiment of the present invention;

[0055] Figure 2 It is a dual-camera acquisition platform according to an embodiment of the present invention;

[0056] Figure 3 This is a flow chart of a band selection method based on a genetic algorithm according to an embodiment of the present invention;

[0057] Figure 4 This is a schematic diagram of the network structure of the FLF-YOLACT feature fusion algorithm according to an embodiment of the present invention;

[0058] Figure 5 This is a schematic diagram of the network structure of the DF-YOLACT decision fusion algorithm according to an embodiment of the present invention;

[0059] Figure 6 This is a schematic diagram of RGB branches of the DF-YOLACT network according to an embodiment of the present invention;

[0060] Figure 7 This is a schematic diagram of the MSI branch of the DF-YOLACT network according to an embodiment of the present invention;

[0061] Figure 8 This is a schematic diagram of the DF-YOLACT network fusion network structure according to an embodiment of the present invention;

[0062] Fig. 9 4 is a system structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to solve the shortcomings of color images in material identification, give full play to the unique advantages of spectral analysis in material identification, and solve the problem of hyperspectral imaging information redundancy, this paper hopes to propose two RGB-MSI multi-feature fusion models through feature fusion to deal with the problem of refined sorting of PET waste plastics. Figure 1 FIG. 1 is a diagram showing steps of a method according to an embodiment of the present invention, wherein the method comprises the following steps:

[0064] S101, collecting RGB data and HSI data of PET waste plastics, performing dimensionality reduction processing on the HSI data based on a genetic algorithm to generate MSI data, and generating a pixel-aligned RGB-MSI multi-feature data set;

[0065] S102, constructing a PET waste plastic intelligent sorting model based on the YOLACT model and using an RGB-MSI multi-feature data set for training; the PET waste plastic intelligent sorting model includes an RGB branch, an MSI branch and a fusion prediction module, RGB is used to process RGB data, the MSI branch is used to process MSI data, and the fusion prediction module is used to combine the processing results of the RGB branch and the processing results of the MSI branch to make a comprehensive prediction;

[0066] S103, using the trained PET waste plastic intelligent sorting model to realize the intelligent sorting of PET waste plastic.

[0067] Specifically, the S101 uses an RGB-HSI data acquisition platform to collect data and generate a pixel-aligned RGB-HSI original data set; the embodiment of the present invention adopts Figure 2 The dual-camera acquisition platform shown is composed of an RGB area array camera and a multi-spectral line array camera.

[0068] Specifically, the original HSI collected in S101 has 224 bands in the wavelength range of 935.9nm to 1722.5nm, and the spectral resolution is 3.5nm. Hyperspectral cameras usually use indium gallium arsenide (InGaAs) or CCD arrays for image acquisition. For this type of image sensor, even in the absence of light, electrons will be generated due to the flow of current in the detector array and the thermal motion of the material itself, and these electrons are also read out by the camera, thereby generating dark current data. The impact of dark current on imaging quality is mainly reflected in the background and shot noise caused by the dark current. When collecting sample images, the same parameter settings and external conditions as those for collecting standard diffuse reflection surface image data and dark current images should be used. The following formula can be used to convert raw radiation value data into reflectance data:

[0069]

[0070] In the formula, Reflectance λ The reflectivity at wavelength λ is the sample radiation value at wavelength, is the dark current radiation value at wavelength λ, is the standard diffuse reflection surface radiation value at wavelength, and the corrected reflectivity should be between 0% and 100%.

[0071] Specifically, S101 is as follows: using the RGB-HSI data acquisition platform to collect data, collecting pixel-aligned RGB-HSI original data sets; randomly selecting multiple independent samples of the same material type in the original hyperspectral data set, and extracting n data points from the hyperspectral image of each sample, these data points are set to form a hyperspectral data set of the same material category containing N data points, and the same is true for other categories, to obtain a light spectrum data set composed of C categories of PET waste plastics, the number of samples in each category is N, the number of bands in each hyperspectral image is M, and a C*N*M preliminary hyperspectral data set is formed; binary encoding is performed on each band to form an initial population, each individual is a candidate band subset, the F1 value of the classification model is used as an evaluation index, and individuals with high fitness are selected by the roulette method, and the best individual is finally selected by cross-mutation, and the K bands represented by the individual are the best band features. According to the best feature band, the corresponding K feature bands are selected for each HSI data, and they are re-joined to form a new MSI image and RGB image to form a feature fusion data set.

[0072] For details, see Figure 3 As shown in Figure 3 , the method for selecting characteristic bands of hyperspectral data based on genetic algorithm in the embodiments of the present invention is explained as follows. Since the selection of spectral characteristic bands belongs to a discrete problem, and each alternative band is an independent discrete individual, binary coding is required to represent each band. When coding, a binary number with a length of r represents all spectral bands, where the selected bands are represented by "1" and the unselected bands are represented by "0". Denote the size of the initial population as M, and randomly generate M binary codes with a length of r as the initial population. Among them, it is required that the number of "1"s in each binary code is the same as the size of the target band subset K.

[0073] The suitability of the fitness function will directly affect the convergence speed of the genetic algorithm and whether the optimal solution can be found. In this experiment, the classification effect of the selected characteristic band subset under the specified classifier is an important indicator to evaluate the superiority of the subset. In order to take into account the precision (P) and recall rate (R) of the classification results, the F1-score of the classifier classification result is selected as the evaluation index of fitness. The calculation formula of the fitness function is as follows:

[0074]

[0075] Where fitness i represents the fitness corresponding to individual i, p i and r i respectively represent the classification precision and recall rate corresponding to individual i.

[0076] The most commonly used individual selection strategy in the GA algorithm is to use the "roulette wheel" method for individual selection. In this strategy, the probability of an individual being selected is proportional to its fitness value. In this strategy, the probability of an individual being selected is proportional to its fitness value. The probability P i of each individual being selected is expressed as:

[0077]

[0078] In the formula, fitness i represents the fitness corresponding to individual i. It can also be seen from the calculation formula that the probability of an individual with a higher fitness being selected is also higher.

[0079] In order to improve the richness of the gene characteristics of the offspring and the searchability of the genetic algorithm, a multi-point crossover strategy is adopted in this experiment. The specific implementation method is as follows: on the premise of satisfying m < K, randomly select m dominant nodes in the paternal gene and the maternal gene respectively. After exchanging the selected nodes one by one, a new generation of individuals can be generated. In order to avoid the genetic search falling into a local optimal solution, in this experiment, a mutation method of flipping some gene segments is also adopted.

[0080] Based on the above steps, when the genetic algorithm meets the termination condition, the optimal gene will be output. By decoding the gene, the optimal target band subset, that is, K optimal bands, can be obtained.

[0081] Specifically, the channel direction of the hyperspectral image is cut and spliced ​​based on the K best bands, and the output format is a multispectral dataset in the Tif file format, which has a one-to-one correspondence with the RGB dataset.

[0082] Specifically, there are two types of PET waste plastic intelligent sorting models constructed in S102, including a feature-level fusion network and a decision-level fusion network.

[0083] For details, see Figure 4 As shown, the embodiment of the present invention builds the FLF-YOLACT feature-level fusion network based on the YOLACT model, which consists of two feature extraction networks, one backbone network extracts features from RGB, and the other backbone network extracts features from MSI. Finally, the features are fused through the "Concat" method and input into the detection head for target recognition. The existing YOLACT target detection model is modified to improve the original single-input single-backbone network into a dual-input dual-backbone network, and the RGB-MSI feature-level fusion network FLF-YOLACT and the RGB-MSI decision-level fusion network FLF-YOLACT are respectively constructed.

[0084] The feature-level fusion network FLF-YOLACT is explained as follows: the input RGB data is 640×640×3, and the MSI data is 640×640×K, where K is the preferred band, and they are output to the corresponding feature extraction backbone network. For the backbone network, both branches use independent ResNet residual networks for feature extraction. ResNet is mainly composed of input layer, convolution layer, residual block and pooling layer. Inputting data into the residual network will generate a total of 5 feature maps from C1 to C5. The C3 to C5 stages are used as the input of the feature pyramid (FPN), where the feature layers with lower layers are used to provide more spatial details, which are more important for capturing fine edges and small targets; the feature layers with higher layers are used to provide richer semantic information, which is helpful for understanding the overall structure of the scene and segmenting large targets.

[0085] The RGB features and MSI features obtained from ResNet are input into the feature pyramid respectively, and finally 5 RGB feature maps and 5 MSI feature maps are obtained. The features of the two types of data are concatenated through the "Concat" fusion method, and the method satisfies the formula:

[0086]

[0087] Where X i , Y i Represents the data of the i-th layer in different input sources, K i Represents the dimension position of the result in the output data.

[0088] After feature fusion, the bounding box and category information of the predicted object are generated through the convolutional layer and fully connected layer of the "prediction head", and the mask map of each instance object is generated through the convolutional layer and upsampling layer of the "protonet".

[0089] For details, see Figure 5 As shown, the embodiment of the present invention builds a DF-YOLACT decision-level fusion network based on the YOLACT model, and adds an MSI branch on the basis of the YOLACT network. This branch performs pixel-level spectral data classification independently of the RGB branch and obtains a classification mask of the same size as RGB. The mask is fused with the YOLACT recognition result for decision-making to improve accuracy. The decision-level fusion network DF-YOLACT is explained as follows:

[0090] Without changing the original algorithm, DF-YOLACT uses the original YOLACT algorithm as the RGB branch to predict the input RGB data in the form of instance segmentation; it reconstructs an MSI branch based on the hyperspectral point classification algorithm to achieve pixel-level prediction of the input MSI data. The MSI branch extracts the spectral point classification results of the corresponding area based on the mask information obtained by the RGB branch, and combines this result with the category and confidence of the RGB branch again to obtain the final prediction result of the input image.

[0091] For the RGB detection branch, see Figure 6 As shown in the figure, since the input data is consistent with the original YOLACT network, the original network structure is directly used, and the 640×640×3 three-channel RGB image is used as the network input. After feature extraction through the ResNet backbone network, the FPN feature pyramid is used for information interaction, and finally 5 feature maps of different scales are generated. The feature map is input into the prediction branch and the prototype mask branch for preliminary prediction, and the prediction result of the RGB branch can be obtained. Since the prediction algorithm is instance segmentation, each instance in the result contains four information: category (Class), location (Box), mask (Mask) and confidence (Confidence).

[0092] For the MSI detection branch, see Figure 7As shown in Figure 1, this branch is mainly composed of an extractor and a classifier. Based on the target instance mask information passed in by the RGB detection branch, the extractor extracts the spectral data corresponding to each target instance one by one and converts it into a point set, where each point represents its corresponding spectral information in the original image. The classifier is used to identify and classify the obtained point sets one by one, and each point set will obtain a corresponding detection result set.

[0093] See also Figure 8 As shown in the figure, in the construction idea of ​​the DF-YOLACT algorithm, the fusion process is mainly based on the RGB branch, supplemented by the MSI branch. The RGB branch is used to generate the category, position, mask and confidence information of the instance, and the MSI branch is used to generate a category result set corresponding to the mask area of ​​the RGB branch. By counting the number of categories in each result set, the category with the most occurrences will be regarded as the category of the instance, and the proportion of its occurrences will represent its confidence. The prediction results generated by the RGB branch are fed back and adjusted through the category and confidence to obtain the final prediction result. Specifically, the result predicted by the MSI branch is material information, and the result predicted by the RGB branch is color and texture information. After obtaining the results of the two, they are integrated to obtain the final prediction result.

[0094] Specifically, the arrangement of S103 is shown in Figure 2 As shown in the figure, the PET waste plastics on the conveyor belt are photographed by the on-site RGB camera and the hyperspectral camera. When predicting the existing PET waste plastics, the RGB camera and the multispectral camera are required to capture the RGB image and the multispectral image of the object at the same time; the two sets of pixel-aligned images are input into the industrial computer at the same time, and the FLF-YOLACT feature fusion network is used for prediction to obtain the coordinates and categories of the PET waste plastics, thereby improving the recognition accuracy; or the DF-YOLACT network is used to input the RGB image and the hyperspectral data into the detector and the classifier respectively, and the results obtained by the two networks are fused at the decision level to solve the shortcoming of insufficient classification accuracy of a single sensor. Through the analysis of the FLF-YOLACT algorithm and the DF-YOLACT algorithm, it is found that both algorithms have been greatly improved compared with YOLACT. From the perspective of the recognition effect of various types of PET plastics, the DF-YOLACT algorithm has higher precision and recall rate, and can obtain better recognition results. In terms of the algorithm's operating efficiency, the inference time of FLF-YOLACT is 62ms shorter than that of the DF-YOLACT algorithm, and its efficiency is about twice that of the DF-YOLACT algorithm. In actual working conditions, it is necessary to comprehensively consider the sorting efficiency and sorting accuracy and select a suitable model for engineering deployment.

[0095] For details, see Fig. 9 FIG. 1 is a system structure diagram of an embodiment of the present invention, including:

[0096] The data processing module 901 collects RGB data and HSI data of PET waste plastics, performs dimensionality reduction processing on the HSI data based on a genetic algorithm to generate MSI data, and generates a pixel-aligned RGB-MSI multi-feature data set;

[0097] A model building module 902 builds a PET waste plastic intelligent sorting model based on the YOLACT model and uses an RGB-MSI multi-feature data set for training; the PET waste plastic intelligent sorting model includes an RGB branch, an MSI branch, and a fusion prediction module, wherein the RGB branch is used to process RGB data, the MSI branch is used to process MSI data, and the fusion prediction module is used to combine the processing results of the RGB branch and the processing results of the MSI branch to make a comprehensive prediction;

[0098] The waste plastic sorting module 903 uses the trained PET waste plastic intelligent sorting model to realize the intelligent sorting of PET waste plastic.

[0099] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for intelligent sorting of PET waste plastics based on multi-feature fusion, characterized in that: The steps include the following: Collect the RGB data and HSI data of PET waste plastics, perform dimensionality reduction processing on the HSI data based on the genetic algorithm to generate MSI data, and generate a pixel-aligned RGB-MSI multi-feature dataset. Construct an intelligent sorting model for PET waste plastics based on the YOLACT model and train it using the RGB-MSI multi-feature dataset. The intelligent sorting model for PET waste plastics includes an RGB branch, an MSI branch, and a fusion prediction module. The RGB branch is used to process RGB data, the MSI branch is used to process MSI data, and the fusion prediction module is used to make a comprehensive prediction by combining the processing results of the RGB branch and the MSI branch. Use the trained intelligent sorting model for PET waste plastics to achieve intelligent sorting of PET waste plastics.

2. The intelligent sorting method for PET waste plastics based on multi-feature fusion according to claim 1 is characterized in that: The dimensionality reduction processing of the HSI data based on the genetic algorithm to generate MSI data includes the following steps: S11, Randomly select multiple independent samples from the HSI dataset by category, and extract n data points from the hyperspectral image of each sample to obtain a hyperspectral dataset of C categories, forming a preliminary hyperspectral dataset of C*N*M. N represents the number of samples in each category, and M represents the number of bands in each category. S12, Perform binary encoding on each band of the preliminary hyperspectral dataset to form an initial population. Each individual in the initial population is a subset of alternative bands. Using the F1 value of the classification model as the evaluation index, select individuals with high fitness through the roulette wheel method, and finally select the best individual through crossover and mutation. The K bands represented by this individual are the best band features. S13, Select the corresponding K feature bands for each HSI data according to the best band features to generate MSI data.

3. The intelligent sorting method for PET waste plastics based on multi-feature fusion according to claim 2 is characterized in that: For the HSI dataset in S11, the HSI data has 224 bands in the wavelength range of 935.9nm to 1722.5nm, and the spectral resolution is 3.5nm. When the hyperspectral camera collects HSI data, it is performed under the same parameter settings and external conditions as those for collecting standard diffuse reflection surface image data and dark current images, and the raw radiation value data is converted into reflectance data by black and white frame correction. λ : In the formula, S λ is the sample radiation value at wavelength λ, D λ is the dark current radiation value at wavelength λ, R λ is the diffuse reflection radiation value of the standard white plate surface at wavelength λ. The corrected reflectivity should be between 0% and 100%.

4. The intelligent sorting method for PET waste plastics based on multi-feature fusion according to claim 2 is characterized in that: The process of selecting the best band features in S12 includes the following steps: S121, Use binary encoding to represent each band of the preliminary hyperspectral dataset. When encoding, use a binary number with a length of r to represent all spectral bands. The selected bands are represented by "1", and the unselected bands are represented by "0", that is, the initial population. Denote the size of the initial population as M, and randomly generate M binary encodings with a length of r as the initial population. Among them, the number of "1" in each binary encoding is the same as the size of the target band subset K. S122, Calculate the fitness of the individuals in the population, expressed as: Among them fitness i represents the fitness of individual i, p i and r i Represent the classification accuracy and recall rate corresponding to individual i respectively; S123, Adopt the "roulette wheel" strategy to select an individual from the initial population. The probability of each individual being selected is expressed as: Among them, fitness i Represents the fitness of individual i. Individuals with high fitness have a higher probability of being selected. S124, On the premise that m < K, randomly select m dominant nodes from the paternal gene and the maternal gene respectively, and exchange the selected nodes one by one to generate a new generation of individuals to form a new population. S125, Repeat steps S121 to S124 until the genetic algorithm meets the termination condition, and decode the individuals with high fitness obtained to obtain K best band features.

5. The intelligent sorting method for PET waste plastics based on multi-feature fusion according to claim 4 is characterized in that: In order to avoid the genetic search from falling into the local optimal solution, a mutation method is also used to flip some gene fragments; the gene fragment refers to the band with binary code "1" in the individuals of the population.

6. The intelligent sorting method for PET waste plastics based on multi-feature fusion according to claim 2 is characterized in that: In S13, the channel directions of the hyperspectral image are cut and spliced ​​based on the selected K best band features, and the output format is an MSI data set in a Tif file. The MSI data set is in a one-to-one correspondence with the RGB data set.

7. The intelligent sorting method for PET waste plastics based on multi-feature fusion according to claim 1 is characterized in that: The PET waste plastic intelligent sorting model is constructed based on the YOLACT model, and the single-input single-trunk network of the existing YOLACT target detection model is improved into a dual-input dual-trunk network, thereby constructing a FLF-YOLACT feature-level fusion network as the PET waste plastic intelligent sorting model; the FLF-YOLACT feature-level fusion network includes: The RGB branch uses an independent ResNet residual network for feature extraction. After obtaining the RGB features, they are input into the feature pyramid and finally several RGB feature maps are obtained. The MSI branch uses an independent ResNet residual network for feature extraction. After obtaining the MSI features, they are input into the feature pyramid and finally several MSI feature maps are obtained. The fusion prediction module uses the Concat fusion method to splice the RGB feature map and the MSI feature map, and then uses the detection head to perform classification prediction based on the spliced ​​fusion features; the fusion features are expressed as: Among them, X i , Y i Represents the data of the i-th layer in different input sources, K i Represents the dimension position of the result in the output data.

8. The intelligent sorting method for PET waste plastics based on multi-feature fusion according to claim 1 is characterized in that: The PET waste plastic intelligent sorting model is constructed based on the YOLACT model, and the existing YOLACT target detection model is used as the RGB branch to realize the prediction of the input RGB data in the form of instance segmentation, and then an MSI branch is rebuilt based on the hyperspectral point classification algorithm to realize the pixel-level prediction of the input MSI data, thereby constructing a DF-YOLACT decision-level fusion network as the PET waste plastic intelligent sorting model; The DF-YOLACT decision-level fusion network includes: The RGB branch uses the ResNet backbone network to extract RGB features, and then uses the FPN feature pyramid for information interaction, and finally generates RGB feature maps of different scales. A preliminary prediction is made based on the RGB feature map to obtain the prediction results of the RGB branch, including category, position, mask and confidence. The MSI branch consists of an extractor and a classifier. The extractor extracts the spectral data corresponding to each target instance one by one based on the mask information of the target instance, and converts it into a point set, where each point represents the corresponding spectral information in the original image. The classifier identifies and classifies the obtained point sets one by one, and each point set obtains a corresponding category result set. The fusion prediction module counts the number of categories in each category result set output by the MSI branch, regards the category with the largest number of occurrences as the category of the target instance corresponding to the category result set, and uses the proportion of the number of occurrences of this category as the confidence level. The prediction results generated by the RGB branch are integrated through the category and confidence level to obtain the final prediction result.

9. The intelligent sorting method for PET waste plastics based on multi-feature fusion according to claim 4 is characterized in that: The method of realizing intelligent sorting of PET waste plastics by using the trained PET waste plastics intelligent sorting model comprises the following steps: Select the trained PET waste plastic intelligent sorting model according to the actual working conditions and deploy it to the industrial computer; The on-site RGB camera and the hyperspectral camera jointly photographed the PET waste on the conveyor belt to obtain two sets of pixel-aligned pictures, which were simultaneously input into the industrial computer; the bands collected by the hyperspectral camera were the K best bands obtained based on the genetic algorithm; The industrial computer uses the deployed model to perform classification prediction and obtain the coordinates and categories of PET waste plastics; The trained PET waste plastic intelligent sorting model is selected according to the actual working conditions, specifically: if high sorting efficiency is required, the FLF-YOLACT feature-level fusion network is selected; if high sorting accuracy is required, the DF-YOLACT decision-level fusion network is selected.

10. An intelligent sorting system for PET waste plastics based on multi-feature fusion, characterized in that: include The data processing module collects RGB data and HSI data of PET waste plastics, performs dimensionality reduction processing on the HSI data based on the genetic algorithm to generate MSI data, and generates a pixel-aligned RGB-MSI multi-feature data set; A model building module, which builds a PET waste plastic intelligent sorting model based on the YOLACT model and uses an RGB-MSI multi-feature data set for training; the PET waste plastic intelligent sorting model includes an RGB branch, an MSI branch, and a fusion prediction module, wherein RGB is used to process RGB data, the MSI branch is used to process MSI data, and the fusion prediction module is used to combine the processing results of the RGB branch and the processing results of the MSI branch to make a comprehensive prediction; The waste plastic sorting module uses the trained PET waste plastic intelligent sorting model to realize the intelligent sorting of PET waste plastics.

Images