Waste plastic classification method and system based on visual identification

Abstract

The invention discloses a waste plastic classification method and system based on visual identification, and particularly relates to the technical field of plastic classification. Reflection suppression is realized through an industrial camera equipped with a polarization filter and a multi-angle cross polarization light source, and the structure and color characteristics of a shielded area are effectively restored by combining brightness equalization, reflection area detection and image restoration technologies; and then semantic segmentation and target classification are completed by using an improved deep neural network model, a control signal is generated based on an identification result, and an execution mechanism is driven to realize accurate sorting of multiple types of plastics, so that the identification robustness and the sorting efficiency of the system under a complex illumination condition are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of plastic classification, and particularly to a method and system for classifying waste plastics based on visual recognition. Background Art

[0002] The classification of waste plastics based on visual recognition refers to the process of automatically identifying and classifying different types of waste plastics by using computer vision technology. This method usually uses a camera to collect plastic images, and combines image processing and machine learning algorithms (such as deep learning) to identify features such as the color, shape, and material of the plastics, so as to classify them into different categories such as PET, PE, PVC, etc., and achieve efficient and intelligent garbage recycling and resource reuse.

[0003] The existing technologies have the following deficiencies:

[0004] In the recognition of waste plastics on a high-speed conveyor belt, smooth or wet plastic surfaces (such as a PET bottle with water droplets) are prone to specular reflection under high-intensity LED lighting, forming high-brightness saturated spots in the image. Such reflective areas will seriously block key visual features such as the color and edges of the plastics themselves, resulting in missing image information. Even using a deep learning model, it may misjudge the plastic material or shape due to the interference of features, thus affecting the classification accuracy and sorting effect. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for classifying waste plastics based on visual recognition to solve the deficiencies in the background art.

[0006] To achieve the above purpose, the present invention provides the following technical solutions: A method for classifying waste plastics based on visual recognition, including:

[0007] On a high-speed conveyor belt, an industrial camera equipped with a polarization filter is used to collect images of waste plastic targets, and at the same time, cross-polarized light is emitted by multiple controllable light sources arranged at different angles;

[0008] Perform brightness equalization and adaptive suppression processing on the collected images, extract the boundaries of saturated areas in combination with a reflection area detection algorithm, and use an image restoration model based on edge completion and color restoration to reconstruct the features of the reflection occlusion areas;

[0009] Use an improved deep neural network model to perform semantic segmentation on the restored images, and identify and separate the contours and areas of each waste plastic target;

[0010] Based on the restored complete image, extract the material texture features and color distribution features of the plastic targets, and input them into a trained convolutional neural network model for classification and recognition;

[0011] According to the recognition result, corresponding control signals are generated to drive the sorting actuator to separately classify and output different types of waste plastics.

[0012] Preferably, the brightness equalization processing includes converting the image to the HSV or Lab color space and performing adaptive histogram equalization processing on the brightness channel; the adaptive suppression processing of the highlight area includes setting a dynamic threshold based on the image brightness mean and standard deviation and performing brightness compression transformation on the area exceeding the threshold.

[0013] Preferably, the extraction of the saturation region boundary by combining the reflection region detection algorithm includes:

[0014] Set a saturation threshold in the brightness channel and mark it as a candidate reflection region;

[0015] In the RGB image, if the differences in the R, G, and B channel values all appear less than the set value, it is determined as strong white reflection;

[0016] Use connected component analysis to cluster these regions;

[0017] Adopt Canny edge detection and morphological operations to extract the edge contours of each reflection region.

[0018] Preferably, the image restoration model includes two parts: edge completion and color restoration. Edge completion adopts curvature-driven diffusion or gradient-guided restoration algorithms, and color restoration reconstructs the image content by means of texture synthesis or deep image restoration network.

[0019] Preferably, the restored image is output, and the restoration effect of the reflection occlusion region feature reconstruction is evaluated according to the degree of edge completion and color restoration. Specifically, it includes:

[0020] Calculate the edge gradient direction and intensity differences between the reconstructed region and its adjacent original regions to evaluate the degree of edge completion. Specifically: perform edge detection on the reconstructed region and its adjacent original regions in the image, process the image using the Sobel operator, and extract the gradient intensity and edge direction information of each pixel point; select pixel points at the boundary of the reconstructed region and compare them with the corresponding points at their adjacent positions in the original region to obtain the edge direction difference and edge intensity difference, and comprehensively weight the edge direction difference and intensity difference to obtain the edge completion score.

[0021] Preferably, the colors of the reconstructed region and its surrounding non-occluded regions are compared in the CIELab space to calculate the color difference. Specifically:

[0022] Perform color space conversion on the image, converting the RGB image to the CIELab color space, where L represents the luminance channel, and a and b respectively represent the distribution of colors in the green - red and blue - yellow directions;

[0023] After the conversion is completed, select the target area that has been repaired in the image as the reconstruction area, and select a group of pixel areas without reflection occlusion in the adjacent area around it as the comparison area. For the two areas, calculate the average value of each channel in the Lab space respectively, compare the color means of the two areas, and calculate their difference as the color difference value.

[0024] Preferably, perform normalization processing on the edge completion score and the color restoration color difference value respectively, perform weighted summation on them to obtain the restoration effect score, compare the restoration effect score with a preset score threshold. If the score is higher than the set threshold, it is considered that the feature reconstruction effect of the reflection occlusion area is good, and the image is used for the subsequent processing flow; if the score is lower than the threshold, it indicates that the repair effect is not ideal, and a processing strategy is triggered.

[0025] Preferably, the deep neural network semantic segmentation model is an improved semantic segmentation network introducing an attention mechanism or an edge enhancement module, which is used to extract and separate the pixel - level region information of waste plastics.

[0026] Preferably, the control signal is generated according to the recognition result, combined with the coordinate information of the plastic target in the image, the conveyor belt speed and the system calibration parameters, and is converted into a physical space position signal, which is used to control the sorting execution to complete the classification output.

[0027] The present invention also provides a waste plastic classification system based on visual recognition, including an image acquisition module, a reflection suppression module, a target segmentation module, a classification recognition module, and a sorting execution module;

[0028] Image acquisition module: On the high - speed conveyor belt, use an industrial camera equipped with a polarization filter to collect images of waste plastic targets, and at the same time emit cross - polarized light through multiple controllable light sources arranged at different angles;

[0029] Reflection suppression module: Perform brightness equalization and adaptive suppression processing on the collected image, combine the reflection area detection algorithm to extract the boundary of the saturated area, and adopt an image repair model based on edge completion and color restoration to reconstruct the features of the reflection occlusion area;

[0030] Target segmentation module: Use an improved deep neural network model to perform semantic segmentation on the repaired image, and identify and separate the contours and regions of each waste plastic target;

[0031] Classification and recognition module: Based on the restored complete image, extract the material texture features and color distribution features of plastic targets, and input them into the trained convolutional neural network model for classification and recognition;

[0032] Sorting execution module: According to the recognition result, generate corresponding control signals to drive the sorting execution mechanism to separately classify and output different types of waste plastics.

[0033] In the above technical solution, the technical effects and advantages provided by the present invention are as follows:

[0034] 1. By introducing a polarization vision acquisition system and an image restoration mechanism, the present invention solves the problems of missing image information and incorrect recognition caused by specular reflection on the plastic surface in the prior art. Through an industrial camera equipped with a polarization filter and a multi-angle cross-polarized light source, the interference of the high-brightness reflection area is effectively suppressed; combined with methods of brightness equalization, reflection area detection, and image restoration based on edge completion and color restoration, the structure and color features of the occluded area can be accurately restored, providing a high-quality input image for subsequent deep learning recognition.

[0035] 2. The present invention combines a semantic segmentation network and a convolutional neural network model to achieve precise segmentation and classification recognition of multiple types of waste plastics, and generates control signals according to the recognition results to drive the sorting equipment to complete high-speed and automated multi-category plastic sorting. The overall system still has high robustness and recognition accuracy in an environment with complex lighting and diverse target materials, improving the intelligent level of waste plastic recycling. Description of the Drawings

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0037] Figure 1 It is the method flow chart of the present invention.

[0038] Figure 2 It is the system module diagram of the present invention. Detailed Embodiments

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0040] Example 1. Refer to Figure 1 As shown, a waste plastic classification method based on visual recognition in this example includes:

[0041] On a high-speed conveyor belt, an industrial camera equipped with a polarization filter is used to collect images of waste plastic targets, and at the same time, cross-polarized light is emitted by multiple controllable light sources arranged at different angles.

[0042] Perform brightness equalization and adaptive suppression of highlight regions on the collected images, extract the boundary of the saturated region in combination with the reflection region detection algorithm, and use an image restoration model based on edge completion and color restoration to reconstruct the features of the reflection occlusion region.

[0043] Use an improved deep neural network model to perform semantic segmentation on the restored images, and identify and separate the contours and regions of each waste plastic target.

[0044] Based on the restored complete image, extract the material texture features and color distribution features of the plastic target, and input them into a trained convolutional neural network model for classification and recognition.

[0045] According to the recognition result, generate corresponding control signals to drive the sorting actuator to separately classify and output different types of waste plastics.

[0046] Select a high-speed and high-resolution industrial camera, install a linear polarization filter in front of the lens, and the polarization direction is adjustable, usually set to vertical polarization (such as 90°) to suppress the reflected light waves from a specific direction.

[0047] The system is configured with several linear polarization LED light sources, distributed around the camera shooting area, such as the front left, front right, above, and obliquely behind, and the arrangement angles are usually 30° - 60° to form a multi-directional illumination field. A polarizer is installed in front of each group of light sources, and the polarization direction is adjustable, usually set perpendicular to the polarization direction of the camera filter (such as 0°).

[0048] The image acquisition system is synchronized with the running state of the conveyor belt, and the shooting time and lighting angle switching are precisely controlled through PLC or image trigger signals to ensure the consistency of the image time sequence and the polarization state of the light source.

[0049] Specular reflection usually retains the polarization state of the incident light, while diffuse reflection has weak polarization or a random polarization direction. If the polarization direction of the observation direction is consistent with the reflected light, the specular reflection is enhanced, otherwise it can be significantly suppressed.

[0050] By cross - arranging the polarization directions, that is, the polarization direction of the light source is perpendicular to the camera filter, and using the "polarization selectivity" principle, this system effectively blocks the specular reflection from returning to the camera, thus only capturing the diffuse reflection information of the target object (whose polarization is weakened and can pass through the filter).

[0051] The reflected light on smooth or wet surfaces (such as water droplets on the surface of a PET bottle) is filtered out due to consistent polarization, and the bright spots in the final image are significantly weakened, and the color, texture, and edge information are clearer.

[0052] Polarization direction debugging: The camera polarizer is usually set to 90°; the light source polarizer is adjusted to 0°; optimize the adjustment by testing whether the high - light area in the image is effectively suppressed.

[0053] Light source intensity control: Use an adjustable constant - current power supply to control the light source intensity to ensure uniform overall exposure of the image and avoid being too dark or locally over - exposed.

[0054] Ambient light - shielding design: To avoid interference from external unpolarized light, it is recommended to set up a light - shielding cover or a black light - absorbing background in the shooting area to enhance the robustness of the system.

[0055] Scene example: On a high - speed running (≥1.5 m / s) waste plastic sorting line, for PET bottles whose images are collected by the above - mentioned system, even if there are water stains or oil films on the surface, the bright - spot areas originally caused by specular reflection disappear significantly, and the edge contours and color information in the image are clearly distinguishable.

[0056] Advantages: Effectively reduce the area of the saturated region in the image by ≥70%; improve the target recognition accuracy by 10% - 25%; have stronger discrimination for fine classification of plastic types (such as transparent PET vs semi - transparent PE); reduce the sorting error rate caused by mis - recognition.

[0057] After image acquisition, to solve the problem of missing image features caused by specular reflection, the image is processed in the following multi - stage manner:

[0058] Step 1: Image brightness equalization processing: Used to unify the overall brightness level of the image, weaken locally over - exposed or over - dark areas, and improve the consistency of the image for subsequent processing.

[0059] Convert the original RGB image to the HSV or Lab color space; separate the brightness channel (such as the V channel or L channel); perform adaptive histogram equalization (CLAHE) on the brightness channel to enhance the contrast and limit the amplification of noise; fuse the equalized brightness with the original image color channels to restore the image.

[0060] Step 2: Adaptive suppression processing of high - light areas: Used to initially weaken the intensity of highly reflective areas and prevent them from forming false features in subsequent detections.

[0061] In the image after brightness enhancement, set a dynamic threshold for each pixel brightness value (which can be adjusted according to the image average brightness mean + standard deviation);

[0062] Perform brightness compression transformation on the pixels with brightness higher than this threshold: I′(x,y) = I(x,y) γ , γ ∈ [0.3, 0.6]; where γ is the compression index, usually obtained through experience or small-sample training; I′(x,y) is the pixel after transformation, and I(x,y) is the pixel before transformation; reduce the saturation while maintaining the original hue to avoid color bleaching.

[0063] Step 3: Reflection area detection and boundary extraction: Used to accurately identify the image saturation area caused by specular reflection for subsequent repair preparation.

[0064] Set a saturation threshold in the brightness channel (such as ≥240 in a 255-level image) and mark it as a candidate reflection area;

[0065] In the RGB image, if the values of the R, G, and B channels are all close to the upper limit (the difference is less than the set value, such as 10), it is determined as strong white reflection;

[0066] Use connected component analysis to cluster these areas and filter out small-area noise;

[0067] Adopt Canny edge detection and morphological operations (dilation + erosion) to extract the edge contour of each reflection area. Specifically:

[0068] Before performing edge detection, first determine the highlighted areas in the image that may have specular reflection: From the brightness channel (such as the V channel of HSV or the L channel of Lab), set a brightness threshold (for example, greater than 240) to extract the highlighted pixel points to form an initial candidate reflection area mask; The RGB channel judgment criteria can be fused: If the values of the R, G, and B channels are close to 255 and the difference between the three channels is less than the set threshold (such as 10), it can be regarded as a saturated reflection area; Use this mask to intercept the corresponding area from the original image as the area to be processed.

[0069] Grayscale the area to be processed to convert the color image into a single-channel image to simplify the edge detection calculation; Use Gaussian Blur or median filter to remove the fine noise in the image and prevent the noise from interfering with the edge detection result.

[0070] Set the high and low thresholds of the Canny algorithm (for example, the low threshold is 50 and the high threshold is 150, which can be adaptively adjusted according to the image contrast); Apply the Canny algorithm to extract the significant edge information in the image to obtain a binary edge image, where white pixels represent the edge lines.

[0071] To enhance the connectivity and integrity of the edge contours, morphological processing is performed on the Canny edge map: Dilation operation: used to connect broken edge segments and fill in the discontinuities caused by specular interference; Erosion operation: used to remove the pseudo-edges brought by dilation and maintain the fine structure of the boundary; It can be combined into an opening operation (erosion followed by dilation) or a closing operation (dilation followed by erosion), and the selection method is adjusted according to the edge continuity; A 3×3 or 5×5 structuring element (such as an ellipse, a cross) can be used to control the scope and intensity of the morphological operation.

[0072] Use a contour extraction algorithm (such as the findContours function in OpenCV) to find closed or semi-closed edge regions in the processed image; Encode the set of edge contour coordinates extracted to form the boundary representation of each reflection region for use in subsequent image inpainting or structure completion steps; Attributes such as the area, bounding box, and directionality of each contour can be calculated for further filtering of abnormal noise or non-reflection regions.

[0073] Step 4: Edge Completion and Color Restoration: Used to restore the image content in the region occluded by reflection and enhance the integrity of the overall recognition features.

[0074] Edge Completion: Extract context structure information (such as the surrounding edge direction, angle, etc.) around the boundary of the extracted reflection region; Use a structured image inpainting algorithm, such as curvature-driven diffusion or gradient-guided inpainting, to fill in the occluded contour along the edge direction to make the object structure continuous.

[0075] Color Restoration: Extract the average color vector (or texture patch) of the pixels adjacent to the boundary of the reflection region; Use texture synthesis (such as PatchMatch) or a deep learning image inpainting network (such as U-Net, DeepFillv2) to naturally extend the texture and color from the surrounding area to the occluded area. Ensure that there are no obvious stitching marks in the transition region and the overall color transitions naturally.

[0076] Step 5: Output the inpainted image and evaluate the restoration effect of the feature reconstruction in the reflection-occluded region according to the degree of edge completion and color restoration, specifically including:

[0077] The evaluation methods for the degree of edge completion can include:

[0078] Structural Similarity Evaluation: Conduct a structural comparison between the reference region in the original image and the boundary of the reconstructed region to evaluate the consistency of local texture and edge direction. The closer the SSIM value is to 1, the closer the structural restoration is to the real image.

[0079] Edge coherence detection: Perform edge detection on the original image and the restored image using the Canny or Sobel algorithm respectively; for the boundary points in the reflection occlusion area, analyze whether they are continuously extended or closed in the restored image; evaluate the integrity of the reconstructed structure through the edge connectivity rate index (such as the intersection ratio of the edge of the reconstructed area and the original edge).

[0080] Boundary gradient change analysis: Calculate the difference in edge gradient direction and intensity between the reconstructed area and its adjacent original area; the smaller the difference, the more natural the transition of edge features and the higher the reconstruction quality.

[0081] For example, the present invention can evaluate the edge completion degree by calculating the difference in edge gradient direction and intensity between the reconstructed area and its adjacent original area, specifically as follows:

[0082] First, perform edge detection on the reconstructed area and its adjacent original area in the image respectively. Preferably, use the Sobel operator to process the image to extract the gradient intensity and edge direction information of each pixel point. The gradient intensity is used to measure the obviousness of the edge in the image, while the edge direction represents the trend of the edge.

[0083] Next, select pixel points at the boundary of the reconstructed area and compare these points with the corresponding points at their adjacent positions in the original area. The two core indicators for comparison are the edge direction difference and the edge intensity difference. The edge direction difference represents the degree of consistency in the edge trend between the restored area and the original area, and the calculation expression is: Δθ(x,y) = min(|θ c (x,y) - θ n (x,y)|, 360° - |θ c (x,y) - θ n (x,y)|); Δθ(x,y) is the edge direction difference, and the smaller this value, the more continuous the edge structure; θ is the gradient direction (unit: degree or radian), (x,y) represents the pixel position on the boundary of the reconstructed area, θ c (x,y) represents the edge direction angle at the corresponding position of the image in the reconstructed area, usually in degrees, and the range is [0°, 360°). This angle is obtained by the edge detection algorithm (such as Sobel) through the gradient direction calculation and reflects the trend of the edge line at this point. θ n (x,y) represents the edge direction angle at the corresponding position of the image in the adjacent original area, also in degrees, representing the edge trend at this point.

[0084] The edge intensity difference indicates the degree of proximity between the repaired edge and the original edge in terms of sharpness. The smaller the difference, the more realistic the texture restoration. The calculation formula is: ΔG(x,y) = |Gc(x,y) - Gn(x,y)|; G is the gradient intensity (which can be obtained by the Sobel operator); ΔG(x,y) is the edge intensity difference, and Gc(x,y) represents the edge gradient intensity at the pixel position in the reconstructed area of the image, usually calculated by operators such as Sobel and Prewitt, and is used to measure the "obviousness" or "sharpness" of the edge. Gn(x,y) represents the edge gradient intensity at the pixel position in the original adjacent area of the image.

[0085] To eliminate the influence of image brightness or size changes, the above difference value is normalized, and the results of all comparison points are averaged to obtain a unified evaluation index. The closer this index is to zero, the more consistent the reconstructed area is with the original area in terms of structure, and the more natural the filling effect is.

[0086] Furthermore, the edge direction difference and intensity difference are comprehensively weighted to obtain an edge completion score. Finally, when the score reaches or exceeds the set threshold (such as above 0.85), it is considered that the edge completion effect is good, and the subsequent target segmentation and recognition process can be entered; if the score is insufficient, it can be marked as a reconstruction failure area, triggering a local repair or ignoring processing strategy.

[0087] The evaluation methods for color restoration degree can include:

[0088] Color difference (ΔE): The colors of the reconstructed area and its surrounding non-occluded areas are compared in the CIELab space, and the ΔE color difference is calculated; ΔE < 2 is usually considered an imperceptible difference to the human eye, and ΔE < 5 belongs to the natural transition range; the smaller the color difference, the more realistic the color restoration.

[0089] Consistency of color distribution: The RGB or HSV color histograms of the reconstructed area and its surrounding areas are statistically analyzed and cross-compared; indicators such as Bhattacharyya distance or KL divergence are used to measure the distribution similarity; it can quantify whether the color restoration conforms to the overall color tone trend of the image.

[0090] Degree of texture-color fusion: The reconstructed area is analyzed by a sliding window for texture + color, and whether there are abnormalities such as color block faults and texture mutations in the local area is observed; the spatial continuity is judged using deep features (such as intermediate feature maps of convolutional layers).

[0091] For example, the present invention can compare the colors of the reconstructed area and its surrounding non-occluded areas in the CIELab space, calculate the color difference ΔE, and evaluate the color restoration degree, specifically as follows:

[0092] First, perform color space conversion on the image, converting the RGB image to the CIELab color space. The CIELab space is widely used in color science. Its advantage lies in being able to better approximate the perception law of color differences by the human eye and is suitable for color consistency analysis in image restoration. Among them, L represents the luminance channel, and a and b respectively represent the distribution of color in the green - red and blue - yellow directions.

[0093] After the conversion, select the completed repaired target area in the image as the reconstruction area, and select a group of pixel areas without reflection occlusion in the adjacent area around it as the comparison area. For these two areas, calculate the average value of each channel (L, a, b) in the Lab space respectively, which is used to represent the overall color characteristics of the area.

[0094] Subsequently, use an advanced ΔE color difference model (such as ΔE2000) to compare the color means of the two areas, and calculate their difference as the color difference value.

[0095] The smaller the color difference value ΔE, the closer the color of the reconstruction area is to its adjacent area, and the better the color restoration effect. Generally, when the ΔE value is less than 2, it means that the human eye can hardly perceive the color difference, which belongs to the ideal restoration effect; when the ΔE value is between 2 and 5, the color transition is natural and still within the acceptable range; if the ΔE value exceeds 5, there may be obvious color deviation, and it is necessary to further optimize the restoration model or trigger local re - repair processing.

[0096] Normalize the edge completion score and the color restoration color difference value respectively. The edge completion score itself is a positive index between 0 and 1 and does not require further transformation; while the ΔE color difference value of color restoration is a negative index and needs to be inversely normalized. Preferably, use the following method to map the ΔE value to a normalized score between 0 and 1:

[0097] When ΔE is less than or equal to the ideal color difference value (such as 2), set its color score to 1; when ΔE exceeds the acceptable upper limit (such as 5), the score is set to 0; within this interval, use the linear interpolation method to calculate the corresponding color score, so that the smaller the ΔE value, the higher the normalized score, indicating that the color is closer to the real.

[0098] After completing the normalization processing of the two scores, perform weighted summation on them to obtain the restoration effect score. Compare the restoration effect score with the preset score threshold. If the score is higher than the set threshold (for example, 0.85), it is considered that the feature reconstruction effect of the reflection occlusion area is good, and the image can be used for subsequent target segmentation, classification and recognition and other processing procedures; if the score is lower than the threshold, it indicates that the restoration effect is not ideal, and re - acquisition, local re - repair or non - participation in recognition processing strategies can be triggered, so as to improve the recognition accuracy and stability of the entire system.

[0099] The specific steps for semantic segmentation of the deep neural network model include:

[0100] Scale the repaired image to the input size acceptable by the model (such as 512×512 or 640×640); perform pixel value normalization on the image (such as mapping RGB values to the [0,1] interval); if the model uses specific pre-trained weights (such as ImageNet), perform normalization standardization according to its mean and variance.

[0101] Adopt a lightweight and effective semantic segmentation model, and combine attention mechanism or multi-scale feature fusion technology, such as: DeepLabv3+: introduce dilated convolution and multi-scale context information; HRNet+OCR module: maintain high-resolution features and finely segment the contour; UNet++ or SegFormer: used for segmentation of edge-sensitive materials (such as plastic films); add an edge enhancement branch and a Material-Aware Attention module to strengthen the response to plastic texture and boundary changes.

[0102] Key points of the network structure: Encoder part: extract multi-scale semantic features of the image; Decoder part: gradually upsample the low-resolution semantic features and fuse them with high-resolution features; Output part: generate a pixel-level classification map of the same size as the input image, and each pixel is labeled as the corresponding plastic category or background.

[0103] Use annotation tools (such as LabelMe) to perform manual semantic segmentation annotation on the training images to distinguish plastic types (such as PET, PE, PVC) and background areas;

[0104] Adopt the combination of Cross Entropy Loss+Dice Loss to improve the segmentation ability of small targets; introduce Boundary Loss to further optimize the edge recognition accuracy;

[0105] Use the Adam or SGD optimizer to train the model; set an early stopping strategy to avoid overfitting; use mIoU (mean Intersection over Union) and Pixel Accuracy to evaluate the model performance after multiple rounds of training.

[0106] Input the repaired image into the trained model to output a semantic segmentation image, where each pixel corresponds to a class label; use color coding or transparent layer method to overlay and display the segmentation result.

[0107] Perform contour detection (such as the findContours function of OpenCV) on each classification area (such as the PET class area); extract the contour boundary coordinates for subsequent target positioning and mechanical execution.

[0108] Perform a minimum bounding rectangle or mask extraction operation on the detected contour to obtain separate target regions; geometric features (such as area, aspect ratio, orientation, etc.) of each region can be extracted for further identification or filtering of overlapping targets.

[0109] Output the position, category, and boundary information of each segmented and classified plastic target as structured data (such as JSON or control signal format); provide it to the execution module at the back end (such as a robotic arm or a wind nozzle system) to perform corresponding classification actions.

[0110] The technical steps for plastic target feature extraction and classification recognition based on the repaired image include:

[0111] Separate each identified waste plastic target from the image that has completed semantic segmentation to prepare for subsequent feature extraction and classification. Use the pixel-level mask (mask) map output by semantic segmentation to separate individual plastic target regions; generate a minimum bounding rectangle or an affine cropping window for each target, and extract the image sub-block of this region; retain the original color and texture information of this region (not scaled, not compressed).

[0112] Material texture feature extraction: Extract surface texture structure information that can distinguish different plastic types (such as PET, PE, PVC, etc.).

[0113] Local Binary Pattern (LBP) describes the material surface texture by analyzing the gray-scale change pattern in the local neighborhood around the pixel; generates an LBP histogram as a quantitative feature input. Gray-Level Co-Occurrence Matrix (GLCM) features calculate statistical features such as texture directionality, contrast, energy, and uniformity; it can reflect the surface roughness difference between soft / hard plastics.

[0114] Color distribution feature extraction: Capture the common color differences of different plastics to assist the classification model in making judgments. Color histograms extract three-channel histograms (16 or 32 bins can be set) in the RGB and HSV spaces respectively; describe the overall color distribution density and hue distribution of this region. Calculate the mean and standard deviation of each channel to reflect indicators such as the overall brightness, saturation, and contrast of the color. Calculate the color inhomogeneity index to reflect whether there are obvious stripes, prints, or gradients in the color region, and assist in distinguishing plastics such as packaging films.

[0115] Convolutional neural network classification recognition: Combine material texture and color features to accurately classify plastic types.

[0116] Use lightweight convolutional neural networks (such as MobileNetV3, ResNet18, EfficientNet-B0), taking into account both accuracy and real-time performance;

[0117] The input image size is uniformly scaled to the model input requirements (such as 224×224);

[0118] Optionally, the original image region can be directly used as the input, or the extracted manual features can be nested with the image and input into a multi-branch network structure (image + texture channel);

[0119] If the manual feature channel is used, a 1D fully connected branch is added to process the texture and color vectors and fuse them with the main image path.

[0120] The Softmax classifier is used in the last layer of the network to output the probability distribution of each plastic type (such as PET: 85%, PE: 10%, PVC: 5%); the category corresponding to the maximum probability is used as the final recognition result. The recognition result is bound to the target contour in the image to generate structured output information.

[0121] After the waste plastic image is recognized and classified, the system will bind the recognition result to the spatial position information of each target to ensure that the subsequent control signals correspond one-to-one with the physical objects. This binding process includes recording the category of the target (such as PET, PE, PVC, etc.), the recognition confidence, the two-dimensional coordinate information in the image, and the timestamp of this image frame. Through the calibration parameters pre-completed by the camera, the system can convert the image coordinates into the actual position coordinates in the conveyor belt coordinate system, so as to achieve precise spatial correspondence.

[0122] Since the conveyor belt is moving continuously, the system also needs to dynamically predict the physical position that the target will move to at a certain future moment based on the time point when the target appears, the running speed of the conveyor belt, and the current position of the target in the picture. This can ensure that the sorting mechanism aligns with the target for operation at the correct time. If the target is continuously detected in multiple image frames, target tracking algorithms such as Kalman filtering or optical flow-based methods can also be used to further improve the accuracy and stability of position estimation.

[0123] After the recognition module completes the target classification, the system will judge which channel or position the target should be sorted to according to the preset control logic, and determine the type of action to be performed. Each plastic category corresponds to a clear sorting strategy. For example, PET materials may be blown out laterally through a pneumatic nozzle, PE materials are grabbed and transported by a robotic arm, and PVC materials can be guided to the designated exit by an electric slide rail. These control logics are pre-configured in the system and can be dynamically called according to the recognition result.

[0124] After the control logic is determined, the system will convert the recognition information into standard control signals and trigger them precisely according to the predicted time points. The control signals are sent to the actuating device through industrial communication protocols such as PLC digital output, serial communication, or industrial bus interfaces. After receiving the signals, the actuating device will complete the sorting action at the accurate position and time according to the instructions. The pneumatic nozzle system will release a short-term high-pressure air stream when the target passes by; the slide baffle system will switch the angle of the guide plate to guide the plastic target into the corresponding channel; if a robotic arm is used, path planning and control will be performed according to the position information to complete the grasping and placing of the target.

[0125] During the entire control and execution process, the system can also combine sensor or vision feedback mechanisms to perform secondary verification on the sorting results to ensure the successful classification of the targets. In the case of failed recognition or large errors, a default recycling or re-sorting processing strategy will be adopted. Through the above control process, high-precision, automated, and multi-type intelligent classification output of waste plastics based on vision recognition can be achieved.

[0126] Example 2, please refer to Figure 2 As shown, the waste plastic classification system based on vision recognition in this embodiment includes an image acquisition module, a reflection suppression module, a target segmentation module, a classification recognition module, and a sorting execution module;

[0127] Image acquisition module: On the high-speed conveyor belt, an industrial camera equipped with a polarization filter is used to acquire images of waste plastic targets, and at the same time, cross-polarized light is emitted by multiple controllable light sources arranged at different angles;

[0128] Reflection suppression module: Perform brightness equalization and adaptive suppression processing on the acquired images, extract the boundaries of saturated regions in combination with the reflection region detection algorithm, and use an image restoration model based on edge completion and color restoration to reconstruct the features of the reflection occlusion regions;

[0129] Target segmentation module: Use an improved deep neural network model to perform semantic segmentation on the repaired images to identify and separate the contours and regions of each waste plastic target;

[0130] Classification recognition module: Based on the complete repaired image, extract the material texture features and color distribution features of the plastic targets and input them into the trained convolutional neural network model for classification recognition;

[0131] Sorting execution module: According to the recognition results, generate corresponding control signals to drive the sorting execution mechanism to classify and output different types of waste plastics separately.

[0132] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0133] It should be understood that the term "and / or" in this article is merely an association relationship describing associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship. The specific meaning can be understood by referring to the context before and after.

[0134] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0135] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, and all should be covered within the protection scope of this application.

Claims

1. A method for classifying waste plastics based on visual recognition, characterized in that: Including: On a high-speed conveyor belt, an industrial camera equipped with a polarization filter is used to collect images of waste plastic targets, and at the same time, cross-polarized light is emitted by controllable light sources arranged at multiple different angles; Perform brightness equalization and adaptive suppression of high-brightness regions on the collected images, extract the boundaries of saturated regions in combination with a reflection region detection algorithm, and use an image restoration model based on edge completion and color restoration to reconstruct the features of the reflection occlusion regions; Use an improved deep neural network model to perform semantic segmentation on the restored images, and identify and separate the contours and regions of each waste plastic target; Based on the restored complete image, extract the material texture features and color distribution features of the plastic targets, and input them into a trained convolutional neural network model for classification and recognition; According to the recognition results, generate corresponding control signals to drive the sorting actuator to separately classify and output different types of waste plastics.

2. The method for classifying waste plastics based on visual recognition according to claim 1, wherein: The brightness equalization process includes converting the image to the HSV or Lab color space and performing adaptive histogram equalization on the brightness channel; the adaptive suppression process of the high-brightness region includes setting a dynamic threshold based on the mean and standard deviation of the image brightness and performing brightness compression transformation on the regions exceeding the threshold.

3. The waste plastic classification method based on visual recognition according to claim 1, characterized in that: The extraction of the saturated region boundary in combination with the reflection region detection algorithm includes: Set a saturation threshold in the brightness channel and mark it as a candidate reflection region; In the RGB image, if the differences in the R, G, and B channels all appear less than the set value, it is determined as strong white reflection; Use connected component analysis to cluster these regions; Adopt Canny edge detection and morphological operations to extract the edge contours of each reflection region.

4. A waste plastic classification method based on visual recognition according to claim 3, characterized in that: The image restoration model includes two parts: edge completion and color restoration. Edge completion uses curvature-driven diffusion or gradient-guided restoration algorithms, and color restoration reconstructs the image content based on texture synthesis or deep image restoration networks.

5. A method for classifying waste plastics based on visual recognition according to claim 4, characterized in that: Output the restored image, and evaluate the restoration effect of the feature reconstruction of the reflection occlusion region according to the degree of edge completion and color restoration. Specifically, it includes: Calculate the differences in edge gradient direction and intensity between the reconstructed region and its adjacent original region to evaluate the degree of edge completion. Specifically: perform edge detection on the reconstructed region and its adjacent original region in the image, use the Sobel operator to process the image, and extract the gradient intensity and edge direction information of each pixel point; select pixel points at the boundary of the reconstructed region and compare them with the corresponding points at their adjacent positions in the original region to obtain the edge direction difference and edge intensity difference, and comprehensively weight the edge direction difference and intensity difference to obtain the edge completion score.

6. The method for classifying waste plastics based on visual recognition according to claim 5, characterized in that: Compare the colors of the reconstructed region and its surrounding non-occluded regions in the CIELab space and calculate the color difference. Specifically: Perform color space conversion on the image, convert the RGB image to the CIELab color space, where L represents the brightness channel, and a and b respectively represent the distribution of colors in the green-red and blue-yellow directions; After the conversion is completed, select the target area that has been repaired in the image as the reconstruction area, and select a group of pixel areas without reflection occlusion in the adjacent area around it as the comparison area. For the two areas, calculate the average value of each channel in the Lab color space respectively, compare the color means of the two areas, and calculate their difference as the color difference value.

7. A method for classifying waste plastics based on visual recognition according to claim 6, characterized in that: Normalize the edge completion score and the color difference value of color restoration respectively, perform weighted summation on them to obtain the restoration effect score, and compare the restoration effect score with the preset score threshold. If the score is higher than the set threshold, it is considered that the feature reconstruction effect of the reflection occlusion area is good, and the image is used for the subsequent processing flow; if the score is lower than the threshold, it indicates that the restoration effect is not ideal, and a processing strategy is triggered.

8. A method for classifying waste plastics based on visual recognition according to claim 7, characterized in that: The deep neural network semantic segmentation model is an improved semantic segmentation network introduced with an attention mechanism or an edge enhancement module, which is used to extract and separate the pixel-level regional information of waste plastics.

9. A method for classifying waste plastics based on visual recognition according to claim 8, characterized in that: The control signal is generated according to the recognition result, combined with the coordinate information of the plastic target in the image, the conveyor belt speed and the system calibration parameters, and is converted into a physical space position signal, which is used to control the sorting execution to complete the classification output.

10. A waste plastic classification system based on visual recognition, which is used to implement a waste plastic classification method based on visual recognition according to any one of claims 1-9, and is characterized in that: It includes an image acquisition module, a reflection suppression module, a target segmentation module, a classification recognition module and a sorting execution module; Image acquisition module: On the high-speed conveyor belt, use an industrial camera equipped with a polarization filter to collect images of waste plastic targets, and at the same time emit cross-polarized light through controllable light sources arranged at multiple different angles. Reflection suppression module: Perform brightness equalization and adaptive suppression processing on the collected image, combine the reflection area detection algorithm to extract the boundary of the saturated area, and use an image restoration model based on edge completion and color restoration to reconstruct the features of the reflection occlusion area. Target segmentation module: Use an improved deep neural network model to perform semantic segmentation on the repaired image, and identify and separate the contours and regions of each waste plastic target. Classification recognition module: Based on the restored complete image, extract the material texture features and color distribution features of the plastic target, and input them into the trained convolutional neural network model for classification recognition. Sorting execution module: According to the recognition result, generate a corresponding control signal to drive the sorting execution mechanism to separately classify and output different types of waste plastics.

Images