A multi-material recycling container classification identification method and system based on improved MobileNetV2

Abstract

This invention discloses a multi-material recycling container classification and recognition method and system based on an improved MobileNetV2. The method enhances, standardizes, and partitions recycling container images into a three-level dataset. It improves the MobileNetV2 network by adding two levels of Dropout regularization layers at the end of the backbone network, constructing a composite classification head that includes batch normalization and Dropout. A weighted FocalLoss loss function is used with differentiated class weights, and a two-stage hierarchical training mechanism is employed for iterative optimization. GrabCut foreground segmentation, morphological filtering, and IOU deduplication enable independent extraction of ROIs from multiple containers within a single image. Five sets of transform-based Total Aspect Ratio (TTA) fusion inferences are performed on local recognition sub-images, calculating the average confidence score and standardizing the judgment threshold. Material-specific detection is performed for liquid residue, unopened bottle caps, material confusion, and container deformation. This invention improves the accuracy of hard sample recognition, supports multi-target batch recognition, enhances inference stability, and improves compliance verification dimensions.

Description

Technical Field

[0001] This invention belongs to the field of computer vision image classification and intelligent recycling management software technology, and in particular relates to a multi-material recycling container classification and identification method and system based on an improved MobileNetV2. Background Technology

[0002] With the advancement of environmental protection policies and the popularization of waste sorting, intelligent recycling equipment is being rapidly deployed in public places and community sites. Beverage container recycling and identification, as a core component, commonly employs lightweight convolutional neural networks for image classification to reduce edge computing costs and meet real-time requirements. MobileNetV2, due to its inverted residual structure and depthwise separable convolutional characteristics, has become the mainstream backbone network choice for this scenario.

[0003] Existing recycling identification schemes based on basic lightweight networks have the following problems:

[0004] First, the recognition accuracy is low: the basic MobileNetV2 network classification head regularization is insufficient, resulting in low recognition accuracy for difficult samples with similar textures in cardboard box materials. The conventional cross-entropy loss cannot balance the training weights of various types of samples, and the model is prone to bias towards plastic and metal materials with a large number of samples, while the problem of missed detection and misjudgment of cardboard materials with a small number of samples is prominent.

[0005] Second, single-target recognition is limited: existing image inputs only support single-target recognition of the entire image. Multiple recycling containers in a photo cannot be extracted from the ROI area separately, resulting in overlapping objects interfering with each other and small targets being lost.

[0006] Third, the confidence level of inference fluctuates greatly: the recognition inference only uses a single original image prediction, and the recognition jitter caused by light, angle and deformation is serious, resulting in large fluctuations in confidence level.

[0007] Fourth, the compliance verification dimension is too narrow: the recycling compliance judgment only judges the liquid residue, without differentiating the detection logic between transparent and opaque containers. At the same time, it lacks multiple verifications such as bottle cap residue, container deformation, and material confusion. A large number of containers that do not meet the recycling standards are identified, which increases the cost of manual secondary sorting. Summary of the Invention

[0008] Purpose of the invention: This invention provides a multi-material recycling container classification and identification method and system based on an improved MobileNetV2, aiming to solve the problems of low accuracy in identifying difficult samples, limited single-target identification scenarios, large fluctuations in inference confidence, single dimension of compliance verification, lack of a complete set of visualization evaluation tools, and the disconnect between identification function and operation management in existing technologies.

[0009] Technical solution: The present invention provides a multi-material recycling container classification and identification method based on an improved MobileNetV2, comprising:

[0010] S1. Obtain images of recycling containers and divide them into training set, validation set and test set according to a preset ratio; perform image enhancement on the training set to obtain an enhanced training set, and perform standardization processing on the validation set and test set to obtain a standard validation set and a standard test set.

[0011] S2. Construct an improved MobileNetV2 network by adding a regularization layer and building a composite classification head at the end of the original MobileNetV2 backbone network; use a two-stage hierarchical training mechanism to iteratively train the improved MobileNetV2 network on the enhanced training set and the standard validation set to obtain the trained improved MobileNetV2 network.

[0012] S3. Obtain the image to be recognized, and sequentially perform bilateral filtering noise reduction, GrabCut foreground segmentation and morphological operations to extract all effective container regions and corresponding local recognition sub-images of the image to be recognized.

[0013] S4. Perform enhanced fusion inference during testing for each local recognition sub-image to obtain multiple sets of transformed images. Input the transformed images into the trained improved MobileNetV2 network to obtain the probability of each set of transformed images in each category. Calculate the average confidence of the effective container region corresponding to the local recognition sub-image in each category. Based on the average confidence of the effective container region in each category, determine whether the effective container region is identifiable using a preset confidence threshold. For identifiable effective container regions, define the category label of the effective container region as the category corresponding to the maximum average confidence.

[0014] S5. For identifiable valid container areas, perform multi-dimensional recycling compliance judgment according to their category labels to obtain the compliance status of the valid container areas; draw the identifiable valid container areas on the image to be identified, label the category labels, confidence levels and compliance status, and generate labeled images;

[0015] S6. Perform steps S3-S5 on each image in the standard test set as the image to be recognized to obtain the labeled standard test set.

[0016] Furthermore, the improved MobileNetV2 network adds a first Dropout regularization layer, an adaptive average pooling layer, a second Dropout regularization layer, and a composite classification head at the end of the original MobileNetV2 backbone network; the composite classification head includes a first batch normalization layer, a third Dropout regularization layer, a second batch normalization layer, and a fourth Dropout regularization layer.

[0017] Furthermore, the two-stage hierarchical training mechanism includes: in the first stage, freezing the backbone network convolutional layers of the improved MobileNetV2 network and training only the composite classification head; in the second stage, unfreezing the backbone network convolutional layers of the improved MobileNetV2 network and training them together with the composite classification head.

[0018] Furthermore, the iterative training of the improved MobileNetV2 network to obtain the trained improved MobileNetV2 network includes:

[0019] Using weighted FocalLoss as the loss function, the formula is as follows:

[0020] ;

[0021] in, To balance the weights for each category, This represents the predicted probability of the sample corresponding to its class. For focusing parameters;

[0022] Differentiated category weights are configured based on the category sample size, with higher weights assigned to categories whose sample size is less than the preset category ratio;

[0023] During training, a gradient cutoff mechanism is integrated to set an upper limit for the gradient norm, and a learning rate cosine annealing mechanism is integrated to periodically adjust the learning rate. An early stopping control mechanism is also integrated to monitor the loss trend of the standard validation set. If the loss of the standard validation set does not decrease for a preset number of consecutive rounds, training is terminated, and the weights of the improved MobileNetV2 network with the lowest loss on the standard validation set are saved to obtain the trained improved MobileNetV2 network.

[0024] Furthermore, the execution of bilateral filtering for noise reduction, GrabCut foreground segmentation, and morphological operations includes: performing bilateral filtering on the image to be recognized; initializing the GrabCut segmentation region based on the filtered image using a fixed rectangular bounding box, iterating a preset number of times to distinguish foreground pixels from background pixels to obtain an initial foreground mask; performing a closing operation on the initial foreground mask to fill the internal holes of objects, and then performing an opening operation to remove minor noise to obtain a foreground segmentation mask; filtering the image's external contours on the foreground segmentation mask, calculating the bounding rectangle and pixel area of ​​each contour, wherein the area enclosed by the bounding rectangle is the effective container region. Invalid contours are filtered based on both pixel area and aspect ratio. An effective detection pixel area range is set, and valid contours that meet the conditions are retained to obtain a set of effective container regions. For the retained set of effective container regions, the overlap degree between the overlapping areas of each pair of bounding rectangles is calculated using the IOU formula. An overlap judgment threshold is set, and duplicate bounding rectangles that exceed the overlap judgment threshold are merged. The bounding rectangle with the larger area is retained to obtain all the deduplicated effective container regions. For each deduplicated effective container region, its bounding rectangle is expanded outward by a preset number of pixels, and the expanded rectangular area is cropped as a local recognition sub-image.

[0025] Furthermore, the enhanced fusion reasoning during the test includes: sequentially performing horizontal flipping, 15-degree rotation, color perturbation, translational affine transformation, and perspective transformation operations on the local recognition sub-image;

[0026] The formula for calculating the average confidence level of the effective container region corresponding to the local recognition sub-image in each category is as follows:

[0027] ;

[0028] in, The average confidence level of the c-th category. Let c be the probability of the c-th category in the output of the k-th transformed image. This represents the number of groups of images to be transformed.

[0029] Furthermore, the categories of the output of the improved MobileNetV2 network after training are materials, including plastic, metal, glass, and cardboard.

[0030] The multi-dimensional recycling compliance assessment includes: liquid compliance testing, bottle cap compliance testing, material compliance testing, and deformation compliance testing.

[0031] Liquid compliance testing: If the category label is glass or plastic, extract the bottom 30% area of ​​the effective container area and calculate the grayscale variance. If the grayscale variance is greater than a preset variance threshold, the liquid compliance test for that category is deemed unqualified; otherwise, the liquid compliance test for that category is deemed qualified. If the category label is metal or cardboard, calculate the overall saturation mean of the effective container area. If the overall saturation mean is greater than a preset saturation threshold, the liquid compliance test for that category is deemed unqualified; otherwise, the liquid compliance test for that category is deemed qualified.

[0032] Bottle cap compliance inspection: Extract the top 25% of the effective container area, calculate the edge pixel ratio of the effective container area. If the edge pixel ratio is greater than the preset edge ratio threshold, it is determined that the bottle cap compliance inspection of the corresponding category of the effective container area is not up to standard; otherwise, it is determined that the bottle cap compliance inspection of the corresponding category of the effective container area is up to standard.

[0033] Material compliance testing: If the category label is cardboard, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is less than a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is glass, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is greater than or equal to a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is plastic or metal, the material compliance test for that category is deemed qualified.

[0034] Deformation compliance detection: Calculate the left-right asymmetry of the effective container area. If the left-right asymmetry is greater than the preset asymmetry threshold, it is determined that the deformation compliance detection of the corresponding category of the effective container area is not up to standard; otherwise, it is determined that the deformation compliance detection of the corresponding category of the effective container area is up to standard.

[0035] If any of the above verification items fails to meet the standard, the compliance status of the valid container area is determined to be non-compliant; otherwise, the compliance status of the valid container area is determined to be compliant.

[0036] The present invention discloses a multi-material recycling container classification and identification system based on an improved MobileNetV2, comprising:

[0037] The dataset management module is used to acquire images of recycling containers and divide them into training, validation, and test sets according to a preset ratio; perform image enhancement on the training set to obtain an enhanced training set; and perform standardization on the validation and test sets to obtain standard validation and test sets.

[0038] The model training module is used to build an improved MobileNetV2 network. A regularization layer is added to the end of the original MobileNetV2 backbone network and a composite classification head is constructed. The improved MobileNetV2 network is iteratively trained on the enhanced training set and the standard validation set using a two-stage hierarchical training mechanism to obtain the trained improved MobileNetV2 network.

[0039] The multi-object detection module is used to acquire the image to be recognized, and sequentially perform bilateral filtering noise reduction, GrabCut foreground segmentation and morphological operations to extract all effective container regions and corresponding local recognition sub-images of the image to be recognized;

[0040] The image recognition inference module is used to perform enhanced fusion inference during testing on each local recognition sub-image to obtain multiple sets of transformed images. These transformed images are then input into the trained improved MobileNetV2 network to obtain the probability of each set of transformed images in each category. The module calculates the average confidence score of the effective container region corresponding to the local recognition sub-image in each category. Based on the average confidence score of the effective container region in each category, a preset confidence threshold is used to determine whether the effective container region is identifiable. For identifiable effective container regions, the category label of the effective container region is defined as the category corresponding to the maximum average confidence score.

[0041] The recycling compliance verification module is used to perform multi-dimensional recycling compliance judgment on identifiable valid container areas according to their category labels to obtain the compliance status of valid container areas; it draws identifiable valid container areas on the image to be identified, marks category labels, confidence levels and compliance status, and generates labeled images;

[0042] The loop execution module is used to sequentially execute the contents of the multi-object detection module, the image recognition inference module, and the compliance verification module on each image in the standard test set as the image to be recognized, so as to obtain the standard test set with labels.

[0043] Furthermore, in the model training module, the improved MobileNetV2 network adds a first Dropout regularization layer, an adaptive average pooling layer, a second Dropout regularization layer, and a composite classification head at the end of the original MobileNetV2 backbone network; the composite classification head includes a first batch normalization layer, a third Dropout regularization layer, a second batch normalization layer, and a fourth Dropout regularization layer.

[0044] Furthermore, the categories of the output of the improved MobileNetV2 network after training are materials, including plastic, metal, glass, and cardboard.

[0045] In the recycling compliance verification module, the multi-dimensional recycling compliance determination includes: liquid compliance testing, bottle cap compliance testing, material compliance testing, and deformation compliance testing;

[0046] Liquid compliance testing: If the category label is glass or plastic, extract the bottom 30% area of ​​the effective container area and calculate the grayscale variance. If the grayscale variance is greater than a preset variance threshold, the liquid compliance test for that category is deemed unqualified; otherwise, the liquid compliance test for that category is deemed qualified. If the category label is metal or cardboard, calculate the overall saturation mean of the effective container area. If the overall saturation mean is greater than a preset saturation threshold, the liquid compliance test for that category is deemed unqualified; otherwise, the liquid compliance test for that category is deemed qualified.

[0047] Bottle cap compliance inspection: Extract the top 25% of the effective container area, calculate the edge pixel ratio of the effective container area. If the edge pixel ratio is greater than the preset edge ratio threshold, it is determined that the bottle cap compliance inspection of the corresponding category of the effective container area is not up to standard; otherwise, it is determined that the bottle cap compliance inspection of the corresponding category of the effective container area is up to standard.

[0048] Material compliance testing: If the category label is cardboard, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is less than a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is glass, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is greater than or equal to a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is plastic or metal, the material compliance test for that category is deemed qualified.

[0049] Deformation compliance detection: Calculate the left-right asymmetry of the effective container area. If the left-right asymmetry is greater than the preset asymmetry threshold, it is determined that the deformation compliance detection of the corresponding category of the effective container area is not up to standard; otherwise, it is determined that the deformation compliance detection of the corresponding category of the effective container area is up to standard.

[0050] If any of the above verification items fails to meet the standard, the compliance status of the valid container area is determined to be non-compliant; otherwise, the compliance status of the valid container area is determined to be compliant.

[0051] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0052] 1. By overlaying multiple regularized classification heads on the original MobileNetV2 and using weighted FocalLoss, high class weights are configured for difficult cardboard box samples. The two-stage hierarchical training takes into account both the preservation of pre-trained features and scene adaptation. Compared with the basic MobileNet, it improves the accuracy of multi-class cardboard box recognition and reduces false positives and false negatives in small samples.

[0053] 2. Introducing GrabCut multi-target segmentation combined with morphological filtering and IOU deduplication, supporting the independent identification of ROIs from multiple containers in a single image, solving the problems of overlap and interference in the identification of small targets, and adapting to batch mixed image recognition scenarios;

[0054] 3. Five sets of transformed TTA average confidence inference are used to eliminate recognition jitter caused by lighting and shooting angle, and the confidence judgment threshold is standardized to improve the stability of recognition output;

[0055] 4. Differentiated compliance verification logic based on material type, simultaneously detecting four types of non-compliance: liquid residue, unopened bottle caps, mixed materials, and container deformation, reducing the number of non-compliant containers included in the score and lowering the cost of manual secondary sorting; Attached Figure Description

[0056] Figure 1 This is a cumulative reward convergence curve.

[0057] Figure 2 This represents the curve of decreasing system learning costs.

[0058] Figure 3 The confusion matrix is ​​expressed as a percentage.

[0059] Figure 4 For training and validation loss curves.

[0060] Figure 5 To train and validate the accuracy curve. Detailed Implementation

[0061] Example 1

[0062] The present invention provides a multi-material recycling container classification and identification method based on an improved MobileNetV2, comprising:

[0063] S1. Obtain images of recycling containers and divide them into training set, validation set and test set according to a preset ratio; perform image enhancement on the training set to obtain an enhanced training set, and perform standardization processing on the validation set and test set to obtain a standard validation set and a standard test set.

[0064] Specifically, the system automatically creates three-level classification folders for training, validation, and test sets, supporting the storage of images of recycling containers made of materials such as plastic, metal, glass, and cardboard. After acquiring recycling container images and importing them in batches, the system performs file integrity verification. It verifies the correctness of the image format by reading the file header information, checks whether the image size meets the requirements, and excludes damaged, undecoded, or abnormally formatted samples. The five categories of recycling container images that pass the integrity verification are divided into training, validation, and test sets in a 70:15:15 ratio.

[0065] Image enhancement is performed on the training set, including 256-size scaling, 224-pixel random cropping, horizontal flipping, 90-degree random rotation, brightness and contrast adjustment, Gaussian blur, Gaussian noise addition, perspective transformation, random occlusion, hue and saturation shift, etc., to obtain an enhanced version of the training set.

[0066] The validation and test sets are standardized to obtain standard versions of the validation and test sets. The standardization formula is as follows:

[0067] ;

[0068] in, For standardized data, For data before standardization, This embodiment uses the mean vector. , This embodiment uses the standard deviation vector. ;

[0069] S2. Construct an improved MobileNetV2 network by adding a regularization layer and constructing a composite classification head at the end of the backbone network of the original MobileNetV2 network, using a weighted FocalLoss as the loss function, and configuring differentiated class weights according to the class sample size; adopt a two-stage hierarchical training mechanism to iteratively train the improved MobileNetV2 network on the enhanced training set and the standard validation set to obtain the trained improved MobileNetV2 network.

[0070] Specifically, two Dropout regularization layers are added to the end of the original MobileNetV2 backbone network. After adaptive average pooling, a composite classification head containing two batch normalization layers and a Dropout layer is constructed to form an improved MobileNetV2 network.

[0071] Using weighted FocalLoss as the loss function, the formula is as follows:

[0072] ;

[0073] in, To balance the weights for each category, This represents the predicted probability of the sample corresponding to its class. To focus parameters, the loss contribution of easily classified samples is dynamically modulated, reducing the dominant role of simple samples in gradient updates, so that the training focus of the improved MobileNetV2 network is concentrated on difficult-to-classify samples, such as samples of cardboard material.

[0074] A two-stage hierarchical training mechanism is adopted:

[0075] In the first stage, all backbone convolutional layers in the improved MobileNetV2 network are frozen, and only the composite classification head is trained. In this stage, forward and backward propagation are performed on the enhanced training set, the pre-trained feature extraction parameters are fixed, and only the classification head weights are updated to quickly converge the classification decision boundary. At the same time, the standard validation set is used to evaluate the convergence status of the classification head and monitor the loss reduction trend of the standard validation set.

[0076] The second stage involves unfreezing and improving the last four convolutional blocks of the backbone network in the MobileNetV2 network, which are then trained together with the composite classification head. In this stage, the deep feature extraction parameters of the backbone network are fine-tuned on the enhanced training set to adapt to the recycling container recognition scenario. At the same time, the standard validation set is used to evaluate the global fine-tuning effect to prevent overfitting.

[0077] Differentiated class weights are configured based on the sample size of each class. Higher weights are assigned to classes with a sample size less than the preset class ratio (such as cardboard boxes) to alleviate gradient dominance bias caused by class imbalance. At the same time, a gradient truncation mechanism is integrated into the training process to set an upper limit on the gradient norm and prevent gradient explosion during backpropagation. A learning rate cosine annealing mechanism is integrated to periodically adjust the learning rate and avoid local optima stagnation. An early stopping control mechanism is integrated to monitor the trend of the validation set loss. If there is no decrease for several consecutive rounds (such as 15 rounds), the training is terminated, and the weights of the improved MobileNetV2 network when the standard version validation set loss is the lowest are saved to obtain the improved MobileNetV2 network after training.

[0078] S3. Load the trained and improved MobileNetV2 network, open the graphical interactive login page, enter the account information and load the corresponding score data.

[0079] S4. Obtain the image to be recognized uploaded by the user, and sequentially perform bilateral filtering noise reduction, GrabCut foreground segmentation and morphological operations to extract the effective container region of the image to be recognized and the local recognition sub-image corresponding to the effective container region;

[0080] Specifically, the image to be identified uploaded by the user is obtained, and bilateral filtering is performed on the image to be identified to eliminate image noise;

[0081] Based on the filtered image, GrabCut segmentation regions are initialized using fixed rectangular boxes, and foreground and background pixels are distinguished by 5 iterations to obtain the initial foreground mask.

[0082] First, a closing operation is performed on the initial foreground mask to fill the holes inside the object, and then an opening operation is performed to remove small noise points, thus obtaining the foreground segmentation mask;

[0083] The outer contours of the image are filtered on the foreground segmentation mask, and the bounding rectangle and pixel area of ​​each contour are calculated. The area enclosed by the bounding rectangle is the effective container area.

[0084] Invalid contours are filtered based on both pixel area and aspect ratio. The effective detection pixel area range is set to 20,000 to 50,000. The aspect ratio threshold filters out extremely long or flat areas, and retains the valid contours that meet the conditions to obtain a set of valid container regions.

[0085] For the set of retained valid container regions, the overlap degree between the overlapping regions of each pair of bounding rectangles is calculated using the IOU formula. The IOU calculation formula is as follows:

[0086] ;

[0087] in, This represents the area of ​​the intersection region between the two target boxes. The area of ​​the union region of the two target boxes is used; with 0.2 as the overlap judgment threshold, duplicate bounding rectangles exceeding the threshold are merged, and the bounding rectangle with the larger area is retained to eliminate duplicate detection and obtain all valid container regions after deduplication.

[0088] For each deduplicated valid container region, its bounding rectangle is expanded outward by 15 pixels, and the expanded rectangular region is extracted as a local recognition sub-image. This sub-image contains the complete container and its edge context information for subsequent recognition and reasoning.

[0089] S5. Perform enhanced fusion inference during testing for each local recognition sub-image to obtain multiple sets of transformed images. Input the transformed images into the trained improved MobileNetV2 network to obtain the probability of each set of transformed images in each category. Calculate the average confidence of the effective container region corresponding to the local recognition sub-image in each category. Determine whether the effective container region is identifiable using a preset confidence threshold. For identifiable effective container regions, define the category label of the effective container region as the category corresponding to the maximum average confidence.

[0090] Specifically, for each local recognition subgraph, perform Test-Time Enhancement (TTA) fusion inference:

[0091] The local recognition sub-images are sequentially subjected to horizontal flipping, 15-degree rotation, color perturbation, affine translation, and perspective transformation operations, resulting in K=5 sets of transformed images. Each set of transformed images is input into the improved MobileNetV2 network, which outputs the probability of each transformed image in each category. The average confidence score for each category is then calculated using the average confidence score formula:

[0092] ;

[0093] in, Let c be the probability of the c-th category in the output of the k-th transformed image. Let be the average confidence level of the c-th category; set the confidence threshold to 0.45. If the maximum value of the average confidence level of each category in the local recognition sub-image is lower than the confidence threshold, the effective container region is determined to be unrecognizable; otherwise, the effective container region is determined to be recognizable. Define the category label of the effective container region as the category corresponding to the maximum average confidence level, and the confidence level of the effective container region is the maximum average confidence level.

[0094] S6. For identifiable valid container areas, perform multi-dimensional recycling compliance judgment according to their category labels to obtain the compliance status of valid container areas; draw identifiable valid container areas on the image to be identified, label the category labels, confidence level and compliance status, generate labeled images, and accumulate user points for all compliant valid container areas according to the preset point standard, and simultaneously record the recycling quantity of each user category.

[0095] Specifically, the categories of the output of the improved MobileNetV2 network after training are materials, including plastic, metal, glass, and cardboard.

[0096] For identifiable valid container areas, multi-dimensional recycling compliance determination is performed according to the category label of the valid container area, including: liquid compliance testing, cap compliance testing, material compliance testing, and deformation compliance testing;

[0097] Liquid compliance testing: If the category label is glass or plastic, extract the bottom 30% area of ​​the effective container region and calculate the grayscale variance. If the grayscale variance is greater than a preset variance threshold (e.g., 25), the liquid in that category is deemed to have failed the compliance test; otherwise, the liquid in that category has passed the compliance test. If the category label is metal or cardboard, the overall average saturation value of the effective container area is calculated. If the overall average saturation value is greater than the preset saturation threshold (e.g., 60), then the liquid in this category is deemed to be non-compliant; otherwise, the liquid in this category is deemed compliant.

[0098] Bottle cap compliance inspection: Extract the top 25% of the valid container area and calculate the edge pixel ratio of the valid container area. If the proportion of edge pixels is greater than the preset edge proportion threshold (e.g., 0.15), it is determined that the bottle cap of the corresponding category in the effective container area fails the compliance test; otherwise, it is determined that the bottle cap of the corresponding category in the effective container area passes the compliance test.

[0099] Material compliance testing: If the category label is cardboard box, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is less than a preset dispersion threshold (e.g., 30), the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is glass, the Laplacian texture dispersion of the effective container area is calculated. If the Laplacian texture dispersion is greater than or equal to a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is plastic or metal, the material compliance test for that category is deemed qualified.

[0100] Deformation compliance inspection: Calculate the left-right asymmetry of the effective container area. If the left-right asymmetry is greater than the preset asymmetry threshold (e.g., 40), then the deformation compliance detection of the corresponding category of the effective container area is determined to be substandard; otherwise, the deformation compliance detection of the corresponding category of the effective container area is determined to be compliant.

[0101] If any of the above verification items fails to meet the standard, the compliance status of the valid container area is determined to be non-compliant and will not be counted as valid recycling points; if all the above verification items meet the standard, the compliance status of the valid container area is determined to be compliant, and user points are accumulated according to the preset point standard: 2.0 points for a single metal container, 1.0 point for a single glass container, and 0.5 points for a single plastic or cardboard container, and the recycling quantity of each category under the user account is recorded simultaneously.

[0102] S7. Draw a recognizable valid container area on the image to be identified, label the category, confidence level and compliance status, generate a labeled image and save it;

[0103] S8. Perform steps S4-S7 on each image in the standard test set as the image to be identified to obtain the labeled standard test set; output a classification report based on the true and predicted categories of all valid container regions in the image.

[0104] Specifically, steps S4-S7 are performed on each image in the standard test set to obtain an labeled standard test set;

[0105] Existing technologies lack comprehensive visualization and evaluation tools, relying solely on fragmented script outputs for training loss, recognition accuracy, and confusion distribution. This prevents the batch generation of convergence curves and heatmaps, hindering intuitive quantitative data for algorithm iteration and optimization. This invention, however, plots training and validation loss curves, accuracy convergence curves, cumulative rewards, and system cost curves based on the true and predicted categories of all valid container regions in an image. It also generates a normalized confusion heatmap based on inference results from a standard test set. All models, images, and logs are persistently stored, supporting offline loading for recognition. The dataset and training parameters are modularly configurable, allowing for rapid adaptation to different regions and categories of recyclable containers, demonstrating strong versatility.

[0106] S9. Switch to the points ranking display interface. The system displays the points data of all registered accounts from highest to lowest points.

[0107] Specifically, existing technologies lack a graphical login, points statistics, and ranking interaction module, making it impossible to achieve integrated management of automatic accumulation, query, and display of recyclers' points. This invention integrates identification, verification, and points GUI, allowing ordinary recycling users to upload images and automatically calculate points without professional operation. It is compatible with the commercial deployment of offline environmental recycling equipment. The points ranking display interface shows the points data of all registered accounts from high to low.

[0108] Regarding the above technical solution in this embodiment, the following is a detailed explanation. Figures 1 to 5 This will detail the experimental process of applying the technical solution to a specific experiment and the technical effects of the technical solution.

[0109] This embodiment builds a local PC-based environmental recycling identification system with a general-purpose CPU and 8GB of memory. The dataset contains 12,000 images, including 2,800 images of plastic bottles, 2,600 images of aluminum cans, 2,200 images of glass bottles, 2,200 images of soft drink cartons, and 2,200 images of milk cartons. The categories are based on materials, including plastic, metal, glass, and cardboard. The training, validation, and test sets are divided in a 70:15:15 ratio. The images are all 224×224 pixels in size. The training set uses all ten image enhancement transformations, including 256-pixel scaling, 224-pixel random cropping, horizontal flipping, 90-degree random rotation, brightness and contrast adjustment, Gaussian blur, Gaussian noise addition, perspective transformation, random occlusion, and hue and saturation shift, to obtain an enhanced training set. The validation and test sets are standardized to obtain standard validation and test sets.

[0110] During model training, an improved MobileNetV2 network is built, with two levels of Dropout added to the end of the original MobileNetV2 backbone network. The composite classification head is configured with two layers of Batch Normalization (BN) and Dropout layers; FocalLoss parameters are set. =3.0, weight array [1.0,1.0,1.2,3.0,3.5,0.3], to increase the training weights of the two types of cardboard boxes; in the first stage, freeze all backbone convolutions, iterate for 5 rounds, with a learning rate of 1e-3, and only update the classification head; in the second stage, unfreeze the last 4 groups of convolutional blocks, with a total training round limit of 30, the learning rate is reduced to 1e-4, gradient clipping and cosine annealing learning rate are used, and early stopping is triggered after 15 consecutive rounds of validation loss without decrease. After training is completed, save the best_recycle_model.pth weight file, and synchronously record the time series logs of training loss, validation loss, training accuracy, and validation accuracy for each round.

[0111] Recognition and inference process: Read the locally collected image, preprocess it through bilateral filtering, iterate 5 times with GrabCut to distinguish the foreground and background, clean up the mask with closing and opening operations, filter the contours and filter by area and aspect ratio, merge overlapping boxes with IOU=0.2, and expand outward by 15 pixels to crop the sub-image of each container ROI; perform 5 sets of TTA transformations on a single ROI, including horizontal flip, 15-degree rotation, color perturbation, translation, and perspective, calculate the average confidence of each category, and set the judgment threshold to 0.45. If the value is below the threshold, it is marked as unrecognizable.

[0112] Compliance verification is performed based on material: For glass and plastic materials, the bottom 30% grayscale variance is taken as the indicator; if it is greater than 25, liquid is found. For metal and cardboard materials, the average HSV saturation is taken as the indicator; if it is greater than 60, liquid residue is found. If the top 25% edge density of the image exceeds 0.15, the bottle cap is not removed. If the Laplacian texture variance of cardboard materials is less than 30, it is misidentified as glass. If the left-right asymmetry is greater than 40, the container is deformed. If any item fails to meet the requirements, no points will be awarded.

[0113] The visualization module reads training time-series data and automatically plots training and validation loss curves (e.g., ...). Figure 4 As shown), the accuracy convergence curve (as shown) Figure 5 As shown), cumulative rewards (such as...) Figure 1 (as shown), system cost curve (as shown) Figure 2 As shown), a normalized confusion heatmap is generated based on the inference results of the test set (e.g., Figure 3 As shown), the data is saved in batches to the visualization_results directory; the GUI interaction unit has three preset test accounts. After compliance identification is completed, the points are automatically accumulated according to the following: 2.0 points for a single account made of metal, 1.0 points for a single account made of glass, and 0.5 points for a single account made of plastic or cardboard. The points ranking window displays the account data in descending order of points.

[0114] Example 2

[0115] The present invention discloses a multi-material recycling container classification and recognition system based on an improved MobileNetV2, comprising: a dataset management module, a model training module, a loading and login module, a multi-target detection module, an image recognition and inference module, a recycling compliance verification module, a labeling module, a loop execution and visualization evaluation module, and an points display module;

[0116] The dataset management module is used to acquire images of recycling containers and divide them into training, validation, and test sets according to a preset ratio; perform image enhancement on the training set to obtain an enhanced training set; and perform standardization on the validation and test sets to obtain standard validation and test sets.

[0117] Specifically, the system automatically creates three-level classification folders for training, validation, and test sets, supporting the storage of images of recycling containers made of materials such as plastic, metal, glass, and cardboard. After acquiring recycling container images and importing them in batches, the system performs file integrity verification. It verifies the correctness of the image format by reading the file header information, checks whether the image size meets the requirements, and excludes damaged, undecoded, or abnormally formatted samples. The five categories of recycling container images that pass the integrity verification are divided into training, validation, and test sets in a 70:15:15 ratio.

[0118] Image enhancement is performed on the training set, including 256-size scaling, 224-pixel random cropping, horizontal flipping, 90-degree random rotation, brightness and contrast adjustment, Gaussian blur, Gaussian noise addition, perspective transformation, random occlusion, hue and saturation shift, etc., to obtain an enhanced version of the training set.

[0119] The validation and test sets are standardized to obtain standard versions of the validation and test sets. The standardization formula is as follows:

[0120] ;

[0121] in, For standardized data, For data before standardization, This embodiment uses the mean vector. , This embodiment uses the standard deviation vector. ;

[0122] The model training module is used to build an improved MobileNetV2 network. A regularization layer is added to the end of the backbone network of the original MobileNetV2 and a composite classification head is built. A weighted FocalLoss loss function is used, and differentiated class weights are configured according to the sample size of each class. A two-stage hierarchical training mechanism is used to iteratively train the improved MobileNetV2 network on the enhanced training set and the standard validation set to obtain the trained improved MobileNetV2 network.

[0123] Specifically, two Dropout regularization layers are added to the end of the original MobileNetV2 backbone network. After adaptive average pooling, a composite classification head containing two batch normalization layers and a Dropout layer is constructed to form an improved MobileNetV2 network.

[0124] Using weighted FocalLoss as the loss function, the formula is as follows:

[0125] ;

[0126] in, To balance the weights for each category, This represents the predicted probability of the sample corresponding to its class. To focus parameters, the loss contribution of easily classified samples is dynamically modulated, reducing the dominant role of simple samples in gradient updates, so that the training focus of the improved MobileNetV2 network is concentrated on difficult-to-classify samples, such as samples of cardboard material.

[0127] A two-stage hierarchical training mechanism is adopted:

[0128] In the first stage, all backbone convolutional layers in the improved MobileNetV2 network are frozen, and only the composite classification head is trained. In this stage, forward and backward propagation are performed on the enhanced training set, the pre-trained feature extraction parameters are fixed, and only the classification head weights are updated to quickly converge the classification decision boundary. At the same time, the standard validation set is used to evaluate the convergence status of the classification head and monitor the loss reduction trend of the standard validation set.

[0129] The second stage involves unfreezing and improving the last four convolutional blocks of the backbone network in the MobileNetV2 network, which are then trained together with the composite classification head. In this stage, the deep feature extraction parameters of the backbone network are fine-tuned on the enhanced training set to adapt to the recycling container recognition scenario. At the same time, the standard validation set is used to evaluate the global fine-tuning effect to prevent overfitting.

[0130] Differentiated class weights are configured based on the sample size of each class. Higher weights are assigned to classes with a sample size less than the preset class ratio (such as cardboard boxes) to alleviate gradient dominance bias caused by class imbalance. At the same time, a gradient truncation mechanism is integrated into the training process to set an upper limit on the gradient norm and prevent gradient explosion during backpropagation. A learning rate cosine annealing mechanism is integrated to periodically adjust the learning rate and avoid local optima stagnation. An early stopping control mechanism is integrated to monitor the trend of the validation set loss. If there is no decrease for several consecutive rounds (such as 15 rounds), the training is terminated, and the weights of the improved MobileNetV2 network when the standard version validation set loss is the lowest are saved to obtain the improved MobileNetV2 network after training.

[0131] Load the login module, which is used to load the improved MobileNetV2 network after training, open the graphical interactive login page, and load the corresponding score data by entering account information.

[0132] The multi-target detection module is used to acquire the image to be identified uploaded by the user, and sequentially perform bilateral filtering noise reduction, GrabCut foreground segmentation and morphological operations to extract the effective container region of the image to be identified and the local recognition sub-image corresponding to the effective container region.

[0133] Specifically, the image to be identified uploaded by the user is obtained, and bilateral filtering is performed on the image to be identified to eliminate image noise;

[0134] Based on the filtered image, GrabCut segmentation regions are initialized using fixed rectangular boxes, and foreground and background pixels are distinguished by 5 iterations to obtain the initial foreground mask.

[0135] First, a closing operation is performed on the initial foreground mask to fill the holes inside the object, and then an opening operation is performed to remove small noise points, thus obtaining the foreground segmentation mask;

[0136] The outer contours of the image are filtered on the foreground segmentation mask, and the bounding rectangle and pixel area of ​​each contour are calculated. The area enclosed by the bounding rectangle is the effective container area.

[0137] Invalid contours are filtered based on both pixel area and aspect ratio. The effective detection pixel area range is set to 20,000 to 50,000. The aspect ratio threshold filters out extremely long or flat areas, and retains the valid contours that meet the conditions to obtain a set of valid container regions.

[0138] For the set of retained valid container regions, the overlap degree between the overlapping regions of each pair of bounding rectangles is calculated using the IOU formula. The IOU calculation formula is as follows:

[0139] ;

[0140] in, This represents the area of ​​the intersection region between the two target boxes. The area of ​​the union region of the two target boxes is used; with 0.2 as the overlap judgment threshold, duplicate bounding rectangles exceeding the threshold are merged, and the bounding rectangle with the larger area is retained to eliminate duplicate detection and obtain all valid container regions after deduplication.

[0141] For each deduplicated valid container region, its bounding rectangle is expanded outward by 15 pixels, and the expanded rectangular region is extracted as a local recognition sub-image. This sub-image contains the complete container and its edge context information for subsequent recognition and reasoning.

[0142] The image recognition inference module is used to perform enhanced fusion inference during testing on each local recognition sub-image to obtain multiple sets of transformed images. The transformed images are then input into the trained improved MobileNetV2 network to obtain the probability of each set of transformed images in each category. The average confidence of the effective container region corresponding to the local recognition sub-image in each category is calculated. A preset confidence threshold is used to determine whether the effective container region is identifiable. For identifiable effective container regions, the category label of the effective container region is defined as the category corresponding to the maximum average confidence.

[0143] Specifically, for each local recognition subgraph, perform Test-Time Enhancement (TTA) fusion inference:

[0144] The local recognition sub-images are sequentially subjected to horizontal flipping, 15-degree rotation, color perturbation, affine translation, and perspective transformation operations, resulting in K=5 sets of transformed images. Each set of transformed images is input into the improved MobileNetV2 network, which outputs the probability of each transformed image in each category. The average confidence score for each category is then calculated using the average confidence score formula:

[0145] ;

[0146] in, Let c be the probability of the c-th category in the output of the k-th transformed image. Let be the average confidence level of the c-th category; set the confidence threshold to 0.45. If the maximum value of the average confidence level of each category in the local recognition sub-image is lower than the confidence threshold, the effective container region is determined to be unrecognizable; otherwise, the effective container region is determined to be recognizable. Define the category label of the effective container region as the category corresponding to the maximum average confidence level, and the confidence level of the effective container region is the maximum average confidence level.

[0147] The recycling compliance verification module is used to perform multi-dimensional recycling compliance judgment on identifiable valid container areas according to their category labels to obtain the compliance status of valid container areas; draw identifiable valid container areas on the image to be identified, label the category labels, confidence level and compliance status, generate labeled images, and accumulate user points for all compliant valid container areas according to the preset scoring standard, and simultaneously record the recycling quantity of each category of user.

[0148] Specifically, the categories of the output of the improved MobileNetV2 network after training are materials, including plastic, metal, glass, and cardboard.

[0149] For identifiable valid container areas, multi-dimensional recycling compliance determination is performed according to the category label of the valid container area, including: liquid compliance testing, cap compliance testing, material compliance testing, and deformation compliance testing;

[0150] Liquid compliance testing: If the category label is glass or plastic, extract the bottom 30% area of ​​the effective container region and calculate the grayscale variance. If the grayscale variance is greater than a preset variance threshold (e.g., 25), the liquid in that category is deemed to have failed the compliance test; otherwise, the liquid in that category has passed the compliance test. If the category label is metal or cardboard, the overall average saturation value of the effective container area is calculated. If the overall average saturation value is greater than the preset saturation threshold (e.g., 60), then the liquid in this category is deemed to be non-compliant; otherwise, the liquid in this category is deemed compliant.

[0151] Bottle cap compliance inspection: Extract the top 25% of the valid container area and calculate the edge pixel ratio of the valid container area. If the proportion of edge pixels is greater than the preset edge proportion threshold (e.g., 0.15), it is determined that the bottle cap of the corresponding category in the effective container area fails the compliance test; otherwise, it is determined that the bottle cap of the corresponding category in the effective container area passes the compliance test.

[0152] Material compliance testing: If the category label is cardboard box, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is less than a preset dispersion threshold (e.g., 30), the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is glass, the Laplacian texture dispersion of the effective container area is calculated. If the Laplacian texture dispersion is greater than or equal to a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is plastic or metal, the material compliance test for that category is deemed qualified.

[0153] Deformation compliance inspection: Calculate the left-right asymmetry of the effective container area. If the left-right asymmetry is greater than the preset asymmetry threshold (e.g., 40), then the deformation compliance detection of the corresponding category of the effective container area is determined to be substandard; otherwise, the deformation compliance detection of the corresponding category of the effective container area is determined to be compliant.

[0154] If any of the above verification items fails to meet the standard, the compliance status of the valid container area is determined to be non-compliant and will not be counted as valid recycling points; if all the above verification items meet the standard, the compliance status of the valid container area is determined to be compliant, and user points are accumulated according to the preset point standard: 2.0 points for a single metal container, 1.0 point for a single glass container, and 0.5 points for a single plastic or cardboard container, and the recycling quantity of each category under the user account is recorded simultaneously.

[0155] The annotation module is used to draw identifiable and valid container regions on the image to be identified, annotate the category labels, confidence levels and compliance status, generate labeled an image and save it;

[0156] The loop execution and visualization evaluation module is used to sequentially execute the multi-object detection module, image recognition inference module, compliance verification module, and annotation module on each image in the standard test set as the image to be identified, to obtain a labeled standard test set; and output a classification report based on the true and predicted categories of all valid container regions in the image.

[0157] Specifically, for each image in the standard test set, the multi-object detection module, image recognition inference module, compliance verification module, and annotation module are executed sequentially to obtain the labeled standard test set;

[0158] Existing technologies lack comprehensive visualization and evaluation tools, relying solely on fragmented script outputs for training loss, recognition accuracy, and confusion distribution. This prevents the batch generation of convergence curves and heatmaps, hindering intuitive quantitative data for algorithm iteration and optimization. This invention, however, plots training and validation loss curves, accuracy convergence curves, cumulative rewards, and system cost curves based on the true and predicted categories of all valid container regions in an image. It also generates a normalized confusion heatmap based on inference results from a standard test set. All models, images, and logs are persistently stored, supporting offline loading for recognition. The dataset and training parameters are modularly configurable, allowing for rapid adaptation to different regions and categories of recyclable containers, demonstrating strong versatility.

[0159] The points display module is used to switch to the points ranking display interface. The system displays the points data of all registered accounts in descending order of points.

[0160] Specifically, existing technologies lack a graphical login, points statistics, and ranking interaction module, making it impossible to achieve integrated management of automatic accumulation, query, and display of recyclers' points. This invention integrates identification, verification, and points GUI, allowing ordinary recycling users to upload images and automatically calculate points without professional operation. It is compatible with the commercial deployment of offline environmental recycling equipment. The points ranking display interface shows the points data of all registered accounts from high to low.

[0161] Since Example 2 is a modular product representation of the monitoring method in Example 1, the experiments corresponding to Example 2 are the same as those in Example 1, and will not be repeated here.

Claims

1. A multi-material recycling container classification and identification method based on an improved MobileNetV2, characterized in that, include: S1. Obtain images of recycling containers and divide them into training, validation and test sets according to a preset ratio; Image augmentation is performed on the training set to obtain an enhanced training set, and normalization is performed on the validation and test sets to obtain standard validation and test sets. S2. Build an improved MobileNetV2 network by adding a regularization layer and constructing a composite classification head at the end of the original MobileNetV2 backbone network. The improved MobileNetV2 network is iteratively trained on the enhanced training set and the standard validation set using a two-stage hierarchical training mechanism to obtain the trained improved MobileNetV2 network. S3. Obtain the image to be recognized, and sequentially perform bilateral filtering noise reduction, GrabCut foreground segmentation and morphological operations to extract all effective container regions and corresponding local recognition sub-images of the image to be recognized. S4. Perform enhanced fusion inference during testing for each local recognition sub-image to obtain multiple sets of transformed images. Input the transformed images into the trained improved MobileNetV2 network to obtain the probability of each set of transformed images in each category. Calculate the average confidence of the effective container region corresponding to the local recognition sub-image in each category. Based on the average confidence of the effective container region in each category, determine whether the effective container region is identifiable using a preset confidence threshold. For identifiable valid container regions, the category label of the valid container region is defined as the category corresponding to the maximum average confidence score; S5. For identifiable valid container areas, perform multi-dimensional recycling compliance judgment according to their category labels to obtain the compliance status of the valid container areas; draw the identifiable valid container areas on the image to be identified, label the category labels, confidence levels and compliance status, and generate labeled images; S6. Perform steps S3-S5 on each image in the standard test set as the image to be recognized to obtain the labeled standard test set.

2. The multi-material recycling container classification and identification method based on improved MobileNetV2 according to claim 1, characterized in that, The improved MobileNetV2 network adds a first Dropout regularization layer, an adaptive average pooling layer, a second Dropout regularization layer, and a composite classification head at the end of the original MobileNetV2 backbone network; the composite classification head includes a first batch normalization layer, a third Dropout regularization layer, a second batch normalization layer, and a fourth Dropout regularization layer.

3. The multi-material recycling container classification and identification method based on the improved MobileNetV2 according to claim 1, characterized in that, The two-stage hierarchical training mechanism includes: in the first stage, freezing the backbone network convolutional layers of the improved MobileNetV2 network and training only the composite classification head; in the second stage, unfreezing the backbone network convolutional layers of the improved MobileNetV2 network and training them together with the composite classification head.

4. The multi-material recycling container classification and identification method based on improved MobileNetV2 according to claim 3, characterized in that, The iterative training of the improved MobileNetV2 network to obtain the trained improved MobileNetV2 network includes: Using weighted FocalLoss as the loss function, the formula is as follows: ； in, To balance the weights for each category, This represents the predicted probability of the sample corresponding to its class. For focusing parameters; Differentiated category weights are configured based on the category sample size, with higher weights assigned to categories whose sample size is less than the preset category ratio; During training, a gradient cutoff mechanism is integrated to set an upper limit for the gradient norm, and a learning rate cosine annealing mechanism is integrated to periodically adjust the learning rate. An early stopping control mechanism is also integrated to monitor the loss trend of the standard validation set. If the loss of the standard validation set does not decrease for a preset number of consecutive rounds, training is terminated, and the weights of the improved MobileNetV2 network with the lowest loss on the standard validation set are saved to obtain the trained improved MobileNetV2 network.

5. The multi-material recycling container classification and identification method based on improved MobileNetV2 according to claim 1, characterized in that, The process of performing bilateral filtering for noise reduction, GrabCut foreground segmentation, and morphological operations includes: performing bilateral filtering on the image to be recognized; initializing the GrabCut segmentation region based on the filtered image using a fixed rectangular bounding box, iterating a preset number of times to distinguish foreground pixels from background pixels, and obtaining an initial foreground mask; performing a closing operation on the initial foreground mask to fill the internal holes of objects, and then performing an opening operation to remove minor noise, obtaining a foreground segmentation mask; filtering the image's outer contours on the foreground segmentation mask, calculating the bounding rectangle and pixel area of ​​each contour, with the area enclosed by the bounding rectangle being the effective container region; and so on. Invalid contours are filtered based on both pixel area and aspect ratio. An effective detection pixel area range is set, and valid contours that meet the conditions are retained to obtain a set of effective container regions. For the retained set of effective container regions, the overlap degree between the overlapping areas of each pair of bounding rectangles is calculated using the IOU formula. An overlap judgment threshold is set, and duplicate bounding rectangles that exceed the overlap judgment threshold are merged. The bounding rectangle with the larger area is retained to obtain all the deduplicated effective container regions. For each deduplicated effective container region, its bounding rectangle is expanded outward by a preset number of pixels, and the expanded rectangular area is cropped as a local recognition sub-image.

6. The multi-material recycling container classification and identification method based on improved MobileNetV2 according to claim 1, characterized in that, The enhanced fusion reasoning during the test includes: sequentially performing horizontal flipping, 15-degree rotation, color perturbation, translational affine transformation, and perspective transformation operations on the local recognition sub-image; The formula for calculating the average confidence level of the effective container region corresponding to the local recognition sub-image in each category is as follows: ； in, The average confidence level of the c-th category. Let c be the probability of the c-th category in the output of the k-th transformed image. This represents the number of groups of images to be transformed.

7. The multi-material recycling container classification and identification method based on improved MobileNetV2 according to claim 1, characterized in that, The categories of the improved MobileNetV2 network output after training are materials, including plastic, metal, glass, and cardboard. The multi-dimensional recycling compliance assessment includes: liquid compliance testing, bottle cap compliance testing, material compliance testing, and deformation compliance testing. Liquid compliance testing: If the category label is glass or plastic, extract the bottom 30% area of ​​the effective container area and calculate the grayscale variance. If the grayscale variance is greater than a preset variance threshold, the liquid compliance test for that category is deemed unqualified; otherwise, the liquid compliance test for that category is deemed qualified. If the category label is metal or cardboard, calculate the overall saturation mean of the effective container area. If the overall saturation mean is greater than a preset saturation threshold, the liquid compliance test for that category is deemed unqualified; otherwise, the liquid compliance test for that category is deemed qualified. Bottle cap compliance inspection: Extract the top 25% of the effective container area, calculate the edge pixel ratio of the effective container area. If the edge pixel ratio is greater than the preset edge ratio threshold, it is determined that the bottle cap compliance inspection of the corresponding category of the effective container area is not up to standard; otherwise, it is determined that the bottle cap compliance inspection of the corresponding category of the effective container area is up to standard. Material compliance testing: If the category label is cardboard, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is less than a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is glass, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is greater than or equal to a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is plastic or metal, the material compliance test for that category is deemed qualified. Deformation compliance detection: Calculate the left-right asymmetry of the effective container area. If the left-right asymmetry is greater than the preset asymmetry threshold, it is determined that the deformation compliance detection of the corresponding category of the effective container area is not up to standard; otherwise, it is determined that the deformation compliance detection of the corresponding category of the effective container area is up to standard. If any of the above verification items fails to meet the standard, the compliance status of the valid container area is determined to be non-compliant; otherwise, the compliance status of the valid container area is determined to be compliant.

8. A multi-material recycling container classification and identification system based on an improved MobileNetV2, characterized in that, include: The dataset management module is used to acquire images of recycling containers and divide them into training, validation, and test sets according to a preset ratio. Image augmentation is performed on the training set to obtain an enhanced training set, and normalization is performed on the validation and test sets to obtain standard validation and test sets. The model training module is used to build an improved MobileNetV2 network by adding a regularization layer and constructing a composite classification head at the end of the original MobileNetV2 backbone network. The improved MobileNetV2 network is iteratively trained on the enhanced training set and the standard validation set using a two-stage hierarchical training mechanism to obtain the trained improved MobileNetV2 network. The multi-object detection module is used to acquire the image to be recognized, and sequentially perform bilateral filtering noise reduction, GrabCut foreground segmentation and morphological operations to extract all effective container regions and corresponding local recognition sub-images of the image to be recognized; The image recognition inference module is used to perform enhanced fusion inference during testing on each local recognition sub-image to obtain multiple sets of transformed images. The transformed images are then input into the trained improved MobileNetV2 network to obtain the probability of each set of transformed images in each category. The average confidence of the effective container region corresponding to the local recognition sub-image in each category is calculated. Based on the average confidence of the effective container region in each category, a preset confidence threshold is used to determine whether the effective container region is identifiable. For identifiable valid container regions, the category label of the valid container region is defined as the category corresponding to the maximum average confidence score; The recycling compliance verification module is used to perform multi-dimensional recycling compliance judgment on identifiable valid container areas according to their category labels to obtain the compliance status of valid container areas; it draws identifiable valid container areas on the image to be identified, marks category labels, confidence levels and compliance status, and generates labeled images; The loop execution module is used to sequentially execute the contents of the multi-object detection module, the image recognition inference module, and the compliance verification module on each image in the standard test set as the image to be recognized, so as to obtain the standard test set with labels.

9. The multi-material recycling container classification and identification system based on the improved MobileNetV2 according to claim 8, characterized in that, In the model training module, the improved MobileNetV2 network adds a first Dropout regularization layer, an adaptive average pooling layer, a second Dropout regularization layer, and a composite classification head at the end of the original MobileNetV2 backbone network; the composite classification head includes a first batch normalization layer, a third Dropout regularization layer, a second batch normalization layer, and a fourth Dropout regularization layer.

10. The multi-material recycling container classification and identification system based on the improved MobileNetV2 according to claim 8, characterized in that, The categories of the improved MobileNetV2 network output after training are materials, including plastic, metal, glass, and cardboard. In the recycling compliance verification module, the multi-dimensional recycling compliance determination includes: liquid compliance testing, bottle cap compliance testing, material compliance testing, and deformation compliance testing; Liquid compliance testing: If the category label is glass or plastic, extract the bottom 30% area of ​​the effective container area and calculate the grayscale variance. If the grayscale variance is greater than a preset variance threshold, the liquid compliance test for that category is deemed unqualified; otherwise, the liquid compliance test for that category is deemed qualified. If the category label is metal or cardboard, calculate the overall saturation mean of the effective container area. If the overall saturation mean is greater than a preset saturation threshold, the liquid compliance test for that category is deemed unqualified; otherwise, the liquid compliance test for that category is deemed qualified. Bottle cap compliance inspection: Extract the top 25% of the effective container area, calculate the edge pixel ratio of the effective container area. If the edge pixel ratio is greater than the preset edge ratio threshold, it is determined that the bottle cap compliance inspection of the corresponding category of the effective container area is not up to standard; otherwise, it is determined that the bottle cap compliance inspection of the corresponding category of the effective container area is up to standard. Material compliance testing: If the category label is cardboard, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is less than a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is glass, calculate the Laplacian texture dispersion of the effective container area. If the Laplacian texture dispersion is greater than or equal to a preset dispersion threshold, the material compliance test for that category is deemed unqualified; otherwise, the material compliance test for that category is deemed qualified. If the category label is plastic or metal, the material compliance test for that category is deemed qualified. Deformation compliance detection: Calculate the left-right asymmetry of the effective container area. If the left-right asymmetry is greater than the preset asymmetry threshold, it is determined that the deformation compliance detection of the corresponding category of the effective container area is not up to standard; otherwise, it is determined that the deformation compliance detection of the corresponding category of the effective container area is up to standard. If any of the above verification items fails to meet the standard, the compliance status of the valid container area is determined to be non-compliant; otherwise, the compliance status of the valid container area is determined to be compliant.

Images