Exposed garbage and overflowed garbage detection method based on deep learning
By using the YOLOX framework and Darknet53+FPN feature extraction network in garbage detection, combined with post-processing rules and data augmentation strategies, the problem of insufficient real-time and accuracy in garbage detection is solved, and efficient and accurate garbage detection effect is achieved.
Patent Information
- Application Number
- CN202311763451.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-20
- Publication Date
- 2024-12-20
AI Technical Summary
The existing technology has problems of insufficient real-time and accuracy in garbage detection, especially in complex scenarios. The traditional method has high time complexity and poor artificial feature extraction effect, while the multi-stage object detection algorithm based on deep learning is high in accuracy. However, the speed is slow and requires a large amount of manual annotation training data, which is relatively expensive.
Using YOLOX as the basic framework, combining Darknet53+FPN feature extraction network, design post-processing rules to filter bounding boxes that are prone to error detection, use the pre-training parameters of COCO data set and the manual annotation data set for fine adjustment, and adopt the data augmentation strategy. Improve the accuracy and real-time performance of the model for detecting exposed and overflowing waste.
It realizes high-precision garbage detection in complex scenarios, meets the real-time requirements of industrial applications, reduces the demand for training data, and improves detection efficiency and accuracy.
Smart Images
Figure CN119169336A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision target detection, and in particular to a method for detecting exposed garbage and overflowing garbage based on deep learning. Background Art
[0002] Garbage detection and cleaning are the main tasks in urban sanitation management, especially for key places where a large amount of garbage is gathered, such as garbage houses, outdoor garbage bins and garbage transfer stations. Simply relying on the inspection and cleaning of sanitation workers is inefficient and cannot ensure that the garbage is completely removed. For this reason, the present invention proposes a method for detecting exposed garbage and overflowing garbage based on deep learning. The real-time images of key places are obtained by monitoring cameras to detect whether there are exposed garbage and overflowing garbage. If there are, a warning is issued so that relevant managers can purposefully dispatch sanitation workers to clean up the garbage.
[0003] At present, target detection methods can be divided into traditional target detection methods and deep learning-based target detection methods. Traditional target detection methods: Based on the sliding window strategy, different window sizes are set to traverse the image, and the manually designed feature extractor is used to extract features from the image in the window. Finally, the extracted features are classified using the pattern recognition method to obtain the target detection result. The candidate region selection strategy based on the sliding window of the traditional target detection method is not targeted, there are a large number of redundant windows, the time complexity is high, and it cannot meet the industrial real-time requirements. At the same time, the manually designed features are difficult to achieve good results in complex and changeable actual application scenarios. The methods based on deep learning can be divided into single-stage target detection and two-stage target detection algorithms. Deep learning single-stage target detection algorithm: A single-stage target detection algorithm based on deep learning is characterized by only performing feature extraction once, and directly using the extracted features to perform bounding box regression and classification tasks simultaneously. Compared with the multi-stage target detection algorithm, the single-stage target detection algorithm is faster, but the accuracy is slightly lower. The most typical single-stage target detection algorithm is the YOLO series [1]. Deep learning multi-stage object detection algorithm: The multi-stage object detection algorithm based on deep learning, taking the most typical RCNN[2] as an example, its steps are divided into three stages: candidate region selection, CNN feature extraction, classification and bounding box regression. RCNN first uses the selective search algorithm to extract about 2000 candidate regions, adjusts each candidate region to a fixed size and extracts CNN features, and finally completes the classification and bounding box regression tasks based on the features of each region. Compared with the single-stage object detection algorithm, the multi-stage object detection algorithm has higher accuracy, but is slightly slower, and it is difficult to meet the real-time requirements of exposed garbage and overflowing garbage detection tasks. At the same time, the object detection algorithms based on deep learning all require a large amount of training data. For the exposed garbage and overflowing garbage detection tasks, all training data need to be manually labeled, and the cost of obtaining a large amount of training data is high. Summary of the invention
[0004] The purpose of the present invention is to solve the technical problems raised in the above-mentioned background technology. The present invention proposes an exposed garbage and overflowing garbage detection algorithm based on deep learning. The technical idea of the present invention is to use the existing single-stage target detection algorithm YOLOX[3] as the basic framework to implement a preliminary detection network model for exposed garbage and overflowing garbage, thereby ensuring the real-time requirements of garbage detection. At the same time, based on the bounding box predicted by the detection model, corresponding post-processing rules are designed to screen the bounding boxes that are prone to misdetection, thereby improving the accuracy of the detection model. Furthermore, in order to solve the problem of less training data, the present invention first loads the pre-training parameters obtained by training based on the COCO dataset[4] for the YOLOX framework, and then fine-tunes them on the manually labeled garbage detection dataset. In addition, for scenes with fewer training samples and more difficult detection, the present invention adopts a data augmentation strategy to improve the detection accuracy of the model for the above two types of scenes.
[0005] The present invention adopts the following technical solution: a method for detecting exposed garbage and overflowing garbage based on deep learning, comprising the following steps:
[0006] S1: Collect images of exposed garbage and overflowing garbage in different real-world scenarios;
[0007] S2: Manually label the acquired garbage detection images.
[0008] S3: Build a spam detection network. The target detection module is mainly implemented based on the YOLOX-L[3] network framework, including:
[0009] S301: Extract features from the image based on the Darknet53+FPN[5] feature extraction network to obtain a feature map with a dimension of H×W×{128,256,512};
[0010] S302: Execute two parallel convolution branches to complete the anchor box classification and regression tasks respectively;
[0011] S303: The classification task branch of the model outputs the anchor box category probability distribution with a dimension of 8400×cls.
[0012] S4: Training network model:
[0013] The IoU loss used to train the anchor box regression branch and the binary cross entropy loss function used to train the anchor box classification branch and the confidence prediction branch are:
[0014]
[0015] The binary cross entropy loss function used to train the confidence prediction branch is:
[0016]
[0017] The binary cross entropy loss function used to train the bounding box classification branch is:
[0018]
[0019] The loss function of the final model is:
[0020] Loss = IoU weight L iou +L obj +L cls (4)
[0021] S5: Based on the trained network model inference, the probability distribution of anchor box categories, anchor box regression parameters and the intersection-over-union confidence of the anchor box and the true bounding box are obtained, and all anchor boxes with an intersection-over-union confidence greater than a certain threshold are returned as the final predicted bounding boxes;
[0022] S6: post-processing, obtaining the exposed garbage and overflowing garbage bounding boxes predicted by the object detection network;
[0023] S7: Evaluation model;
[0024] S8: The post-processed bounding box predictions are visualized on the image.
[0025] Preferably, in S1, exposed garbage is defined as all garbage scattered outside the garbage bin; overflowing garbage is defined as all garbage in the garbage bin that exceeds 1 / 2 of the capacity of the garbage bin; and non-overflowing garbage is defined as all garbage in the garbage bin that does not exceed 1 / 2 of the capacity of the garbage bin. Here, it is helpful to collect garbage using surveillance cameras.
[0026] Preferably, in S1, images of exposed garbage and overflowing garbage in different scenes are collected based on surveillance cameras. Here, it is convenient to collect garbage images in different scenes.
[0027] Preferably, in S2, during training, the training set and the validation set are randomly divided in a ratio of 9:1; during testing, images in the non-training set and the validation set are used for reasoning. Here, it is helpful to use rectangular boxes to mark exposed garbage, overflowing garbage, and garbage cans in the image.
[0028] Preferably, in S3, H is the number of anchor boxes in the height direction, W is the number of anchor boxes in the width direction, and in the present invention, H and W have three scales of 80, 40 and 20, so a total of 6400+1600+400=8400 anchor boxes can be obtained, and cls is the number of categories. Here, it is helpful to use the network to load the pre-trained parameters obtained by training based on the COCO dataset [4], and use the garbage detection dataset to fine-tune the parameters.
[0029] Preferably, in S4, num pred is the number of bounding boxes predicted by the final model based on label assignment [3], B pred is the bounding box predicted by the model, B gt is the real bounding box corresponding to the predicted bounding box, IoU() is the intersection over union ratio, obj is used to indicate whether the target box is the foreground or the background, obj = 1 means the target box is the foreground and contains the detected object; obj = 0 means the target box is the background. pred and obj target They represent the foreground background of the 8400 anchor boxes predicted by the model and the real foreground background of the 8400 anchor boxes, cls pred is the category of the bounding box predicted by the model, cls target To predict the true category of the bounding box, all are represented by one-hot vectors, BCE_Loss is the binary cross entropy loss function, and IoU weight is the weight coefficient of IoU loss. Here, it is convenient for people to calculate the loss function and conduct training.
[0030] Preferably, in S6,
[0031] For each predicted bounding box that exposes garbage:
[0032] If the prediction confidence is greater than 0.85, the predicted bounding box is returned directly;
[0033] If the prediction confidence is greater than 0.7, it is determined whether the predicted bounding box is contained in a trash can prediction box. If it is contained in a trash can prediction box, the prediction box is discarded, otherwise the prediction bounding box is returned.
[0034] For each predicted box that is full of garbage:
[0035] If the prediction confidence is greater than 0.75, the predicted bounding box is returned directly;
[0036] If the prediction confidence is greater than 0.5, it is determined whether the predicted bounding box intersects with a trash can prediction box. If it does not intersect with a trash can prediction box, the prediction box is discarded, otherwise the prediction bounding box is returned.
[0037] Here, we can filter out samples that are prone to false detection, and after filtering out bounding boxes that are prone to false detection, we can get the final prediction result of the model.
[0038] Preferably, in S7, when the IoU between the bounding box predicted by the model and the real bounding box is greater than 0.5, the detection is considered correct:
[0039] For images with garbage, if the model successfully detects all garbage in the image, even if the model returns redundant predicted bounding boxes, it is considered a detection success;
[0040] For images without garbage, the model does not predict any bounding boxes, which means the detection is successful.
[0041] Here, the overfitting of the model is avoided to a certain extent. At the same time, the selected model does not affect the actual application and can fully meet the business logic requirements of garbage detection.
[0042] Preferably, in S8, the locations of exposed garbage and overflowing garbage can be well predicted in garbage houses, transfer stations and outdoor scenes, which effectively helps city managers to quickly dispatch sanitation workers to designated locations for garbage cleaning.
[0043] Compared with the prior art, the advantages and positive effects of the present invention are:
[0044] 1. In view of the complex actual application scenarios and high real-time requirements of garbage detection models, the present invention proposes a garbage detection model that combines a single-stage target detection algorithm based on deep learning and post-processing rules. While achieving higher garbage detection accuracy, it meets the real-time requirements of industrial applications.
[0045] 2. In this invention, data augmentation is performed for scenes with fewer samples and difficult to detect. At the same time, two post-processing rules are proposed for exposed garbage and overflowing garbage, which effectively improves the detection accuracy.
[0046] 3. In the present invention, a performance indicator based on the business logic of the garbage detection task is proposed, which is convenient for selecting the model that is most beneficial to the garbage detection task. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 A flowchart of a method for detecting exposed garbage and overflowing garbage based on deep learning is proposed for the present invention;
[0048] Figure 2 The present invention proposes a post-processing flow chart of an exposed garbage prediction bounding box of an exposed garbage and overflowing garbage detection method based on deep learning;
[0049] Figure 3 The present invention proposes a post-processing flow chart of overflow garbage prediction bounding box of an exposed garbage and overflow garbage detection method based on deep learning. DETAILED DESCRIPTION
[0050] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments of the following disclosure.
[0052] Embodiment 1
[0053] See also Figure 1-2 The present invention provides a technical solution: a method for detecting exposed garbage and overflowing garbage based on deep learning, comprising the following steps:
[0054] S1: Collect images of exposed garbage and overflowing garbage in different real-world scenarios;
[0055] S2: Manually label the acquired garbage detection images.
[0056] S3: Build a spam detection network. The target detection module is mainly implemented based on the YOLOX-L[3] network framework, including:
[0057] S301: Extract features from the image based on the Darknet53+FPN[5] feature extraction network to obtain a feature map with a dimension of H×W×{128,256,512};
[0058] S302: Execute two parallel convolution branches to complete the anchor box classification and regression tasks respectively;
[0059] S303: The classification task branch of the model outputs the anchor box category probability distribution with a dimension of 8400×cls.
[0060] S4: Training network model:
[0061] The IoU loss used to train the anchor box regression branch and the binary cross entropy loss function used to train the anchor box classification branch and the confidence prediction branch are:
[0062]
[0063] The binary cross entropy loss function used to train the confidence prediction branch is:
[0064]
[0065] The binary cross entropy loss function used to train the bounding box classification branch is:
[0066]
[0067] The loss function of the final model is:
[0068] Loss = IoU weight L iou +L obj +L cls (4)
[0069] S5: Based on the trained network model inference, the probability distribution of anchor box categories, anchor box regression parameters and the intersection-over-union confidence of the anchor box and the true bounding box are obtained, and all anchor boxes with an intersection-over-union confidence greater than a certain threshold are returned as the final predicted bounding boxes;
[0070] S6: post-processing, obtaining the exposed garbage and overflowing garbage bounding boxes predicted by the object detection network;
[0071] S7: Evaluation model;
[0072] S8: The post-processed bounding box predictions are visualized on the image.
[0073] Exposed garbage is defined as all garbage scattered outside the trash can; overflowing garbage is defined as all garbage in the trash can that exceeds 1 / 2 of the trash can capacity; non-overflowing garbage is defined as all garbage in the trash can that does not exceed 1 / 2 of the trash can capacity, which helps to collect garbage using surveillance cameras.
[0074] Based on surveillance cameras, images of exposed garbage and overflowing garbage in different scenes are collected to facilitate the collection of garbage images in different scenes.
[0075] During training, the training set and validation set are randomly divided in a ratio of 9:1; during testing, images in non-training and validation sets are used for inference, which helps to use rectangular boxes to mark exposed garbage, overflowing garbage, and trash cans in the images.
[0076] H is the number of anchor boxes in the height direction, and W is the number of anchor boxes in the width direction. In the present invention, H and W have three scales of 80, 40 and 20, and a total of 6400+1600+400=8400 anchor boxes can be obtained. cls is the number of categories, which helps to use the network to load the pre-trained parameters obtained by training based on the COCO dataset [4] and use the garbage detection dataset to fine-tune the parameters.
[0077] num pred is the number of bounding boxes predicted by the final model based on label assignment [3], B pred is the bounding box predicted by the model, B gtis the real bounding box corresponding to the predicted bounding box, IoU() is the intersection over union ratio, obj is used to indicate whether the target box is the foreground or the background, obj = 1 means the target box is the foreground and contains the detected object; obj = 0 means the target box is the background. pred and obj target They represent the foreground background of the 8400 anchor boxes predicted by the model and the real foreground background of the 8400 anchor boxes, cls pred is the category of the bounding box predicted by the model, cls target To predict the true category of the bounding box, all are represented by one-hot vectors, BCE_Loss is the binary cross entropy loss function, and IoU weight It is the weight coefficient of IoU loss, which can facilitate personnel to calculate the loss function and conduct training.
[0078] For each predicted bounding box that exposes garbage:
[0079] If the prediction confidence is greater than 0.85, the predicted bounding box is returned directly;
[0080] If the prediction confidence is greater than 0.7, it is determined whether the predicted bounding box is contained in a trash can prediction box. If it is contained in a trash can prediction box, the prediction box is discarded, otherwise the prediction bounding box is returned.
[0081] For each predicted box that is full of garbage:
[0082] If the prediction confidence is greater than 0.75, the predicted bounding box is returned directly;
[0083] If the prediction confidence is greater than 0.5, it is determined whether the predicted bounding box intersects with a trash can prediction box. If it does not intersect with a trash can prediction box, the prediction box is discarded, otherwise the prediction bounding box is returned.
[0084] The final prediction result of the model can be obtained by filtering out samples that are prone to false detection and filtering out bounding boxes that are prone to false detection.
[0085] When the IoU between the bounding box predicted by the model and the real bounding box is greater than 0.5, the detection is considered correct:
[0086] For images with garbage, if the model successfully detects all garbage in the image, even if the model returns redundant predicted bounding boxes, it is considered a detection success;
[0087] For images without garbage, the model does not predict any bounding boxes, which means the detection is successful.
[0088] To a certain extent, it can avoid overfitting of the model. At the same time, the selected model does not affect the actual application and can fully meet the business logic requirements of garbage detection.
[0089] The location of exposed and overflowing garbage can be well predicted in garbage houses, transfer stations, and outdoor scenarios, effectively helping city managers to quickly dispatch sanitation workers to designated locations for garbage cleanup.
[0090] Working principle: A batch of images are randomly selected from the training set and data augmentation operations are performed on them, including random image color jitter, random horizontal image flipping, and mosaic and confusion data augmentation algorithms.
[0091] The size of the input image is unified as 640×640×3. The model first extracts features of three different scales based on the feature extraction network of Yolox-Darknet53[3], namely 20×20×512, 40×40×256 and 80×80×128. The features of the three different scales are input into three parallel decoupling head networks respectively, and the anchor boxes predicted based on the features of different scales are obtained respectively. Specifically, the structure of each decoupling head branch is as follows: Figure 3 As shown. The decoupling head first unifies the number of channels of each scale feature to 256 through a 1×1 convolution module, and then completes the anchor box classification task, anchor box parameter regression task and anchor box foreground background prediction task respectively through two parallel convolution branches. The upper branch completes the anchor box classification task, which includes 2 3×3 convolution modules, 1 classification convolution module and sigmoid activation function. Finally, the branch outputs the anchor box category prediction result with a dimension of S×S×3, where S=20, 40, 80 is the characteristic scale of the current decoupling head branch. In the present invention, the model predicts a total of four categories: "exposed garbage", "non-overflowing garbage", "overflowing garbage" and "trash can", so the category of each anchor box is represented by a 4-dimensional unique hot encoding. The lower branch completes the anchor box parameter regression task and the anchor box foreground background prediction task. The anchor box parameter regression task branch contains two 3×3 convolution modules and one regression convolution module. Finally, the branch outputs the anchor box parameter regression result with a dimension of S×S×4. The four parameters are x, y and h, w, which represent the center coordinates of the anchor box and the ratio of the height and width of the anchor box to the height and width of the image, respectively. The anchor box foreground and background prediction task branch contains two 3×3 convolution modules, one classification convolution module and a sigmoid activation function. Finally, the branch outputs the anchor box foreground and background prediction result with a dimension of S×S×1, where 1 indicates that the anchor box is the foreground and contains the predicted object; 0 indicates that the anchor box is the background and does not contain the predicted object. The decoupling head network concatenates the information predicted by each branch and finally obtains the prediction output with a dimension of S×S×9. Finally, the network concatenates the prediction outputs of all decoupling heads and performs a transposition operation to obtain all anchor box prediction information with a dimension of 9×8400. Training and prediction will be performed based on the anchor box prediction information in the future.
[0092] Based on the label assignment strategy in YOLOX[3], the model selects bounding prediction boxes from 8400 anchor prediction boxes, which are dynamically determined by the number of true bounding boxes, the predicted category of the anchor box, and the foreground and background of the anchor box. For the selected bounding prediction boxes, the model calculates the IoU loss to train the anchor box regression branch, as shown in formula (1). The binary cross entropy loss is calculated to train the anchor box classification branch and the anchor box foreground and background prediction branch, as shown in formula (2) and formula (3). The final loss function of the model is shown in formula (4).
[0093] In the inference phase, the model uses the same method as the training process to obtain the probability distribution of anchor box categories, anchor box regression parameters, and the intersection-over-union confidence of the anchor box and the true bounding box. Following the post-processing mechanism in YOLOX[3], all anchor boxes whose intersection-over-union confidence is greater than a certain threshold are returned as the final predicted bounding boxes.
[0094] For each prediction bounding box of exposed garbage, if its prediction confidence is greater than 0.85, the prediction bounding box is directly returned; otherwise, if its prediction confidence is greater than 0.7, it is determined whether the prediction bounding box is contained in a trash can prediction box. If it is contained in a trash can prediction box, the prediction box is discarded, otherwise the prediction bounding box is returned. The post-processing rules of exposed garbage are mainly used to screen out the situation where the model misjudges the dirt and light spots on the trash can as exposed garbage. For each prediction box of overflowing garbage, if its prediction confidence is greater than 0.75, the prediction bounding box is directly returned; otherwise, if its prediction confidence is greater than 0.5, it is determined whether the prediction bounding box has an intersection with a trash can prediction box. If it does not have an intersection with a trash can prediction box, the prediction box is discarded, otherwise the prediction bounding box is returned. The post-processing rules of overflowing garbage are mainly used to screen out the situation where the model identifies non-recognized targets, such as garbage in carts, non-target trash cans, garbage trucks, etc. as overflowing garbage.
[0095] When the IoU between the bounding box predicted by the model and the true bounding box is greater than 0.5, the detection is considered correct. For images with garbage, if the model successfully detects all the garbage in the image, even if the model returns redundant predicted bounding boxes, the detection is considered successful. The above model evaluation indicators are mainly based on two considerations. First, the data set for training the garbage detection model is small, and the use of the above evaluation indicators can avoid overfitting of the model to a certain extent. Second, the model selected based on the above evaluation indicators does not affect the actual application and can fully meet the business logic requirements of garbage detection.
[0096] The post-processed boundary prediction box is visualized on the image. It can be seen that the present invention can well predict the location of exposed garbage and overflowing garbage in garbage houses, transfer stations and outdoor scenes, which can effectively help urban managers quickly dispatch sanitation workers to designated locations for garbage cleaning.
[0097] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.
Claims
1. A method for detecting exposed garbage and overflowing garbage based on deep learning, comprising the following steps: S1: Collect images of exposed garbage and overflowing garbage in different real-world scenarios; S2: Manually label the acquired garbage detection images. S3: Build a spam detection network. The target detection module is mainly implemented based on the YOLOX-L[3] network framework, including: S301: Extract features from the image based on the Darknet53+FPN[5] feature extraction network to obtain a feature map with a dimension of H×W×{128,256,512}; S302: Execute two parallel convolution branches to complete the anchor box classification and regression tasks respectively; S303: The classification task branch of the model outputs the anchor box category probability distribution with a dimension of 8400×cls. S4: Training network model: The IoU loss used to train the anchor box regression branch and the binary cross entropy loss function used to train the anchor box classification branch and the confidence prediction branch are: The binary cross entropy loss function used to train the confidence prediction branch is: The binary cross entropy loss function used to train the bounding box classification branch is: The loss function of the final model is: Loss=IoU weight L iou +L obj +L cls (4) S5: Based on the trained network model inference, the probability distribution of anchor box categories, anchor box regression parameters and the intersection-over-union confidence of the anchor box and the true bounding box are obtained, and all anchor boxes with an intersection-over-union confidence greater than a certain threshold are returned as the final predicted bounding boxes; S6: post-processing, obtaining the exposed garbage and overflowing garbage bounding boxes predicted by the object detection network; S7: Evaluation model; S8: The post-processed bounding box predictions are visualized on the image.
2. The method for detecting exposed garbage and overflowing garbage based on deep learning according to claim 1, characterized in that: In S1, exposed garbage is defined as all garbage scattered outside the trash can; overflowing garbage is defined as all garbage existing in the trash can and exceeding 1 / 2 of the trash can capacity; non-overflowing garbage is defined as all garbage existing in the trash can and not exceeding 1 / 2 of the trash can capacity.
3. The method for detecting exposed garbage and overflowing garbage based on deep learning according to claim 1, characterized in that: In S1, images of exposed garbage and overflowing garbage in different scenes are collected based on surveillance cameras.
4. The method for detecting exposed garbage and overflowing garbage based on deep learning according to claim 1, characterized in that: In S2, exposed garbage, overflowing garbage, and garbage cans in the image are marked with rectangular boxes. During training, the training set and the validation set are randomly divided into a ratio of 9:1; during testing, images in the non-training set and the validation set are used for reasoning.
5. The method for detecting exposed garbage and overflowing garbage based on deep learning according to claim 1, characterized in that: In S3, H is the number of anchor boxes in the height direction, and W is the number of anchor boxes in the width direction. In the present invention, H and W have three scales of 80, 40 and 20, and a total of 6400+1600+400=8400 anchor boxes can be obtained, and cls is the number of categories.
6. The method for detecting exposed garbage and overflowing garbage based on deep learning according to claim 1, characterized in that: In S4, num pred is the number of bounding boxes predicted by the final model based on label assignment [3], B pred is the bounding box predicted by the model, B gt is the real bounding box corresponding to the predicted bounding box, IoU() is the intersection over union ratio, obj is used to indicate whether the target box is the foreground or the background, obj = 1 means the target box is the foreground and contains the detected object; obj = 0 means the target box is the background. pred and obj target They represent the foreground background of the 8400 anchor boxes predicted by the model and the real foreground background of the 8400 anchor boxes, cls pred is the category of the bounding box predicted by the model, cls target To predict the true category of the bounding box, all are represented by one-hot vectors, BCE_Loss is the binary cross entropy loss function, and IoU weight is the weight coefficient of IoU loss.
7. The method for detecting exposed garbage and overflowing garbage based on deep learning according to claim 1, characterized in that: In S6, For each predicted bounding box that exposes garbage: If the prediction confidence is greater than 0.85, the predicted bounding box is returned directly; If the prediction confidence is greater than 0.7, it is determined whether the predicted bounding box is contained in a trash can prediction box. If it is contained in a trash can prediction box, the prediction box is discarded, otherwise the prediction bounding box is returned. For each predicted box that is full of garbage: If the prediction confidence is greater than 0.75, the predicted bounding box is returned directly; If the prediction confidence is greater than 0.5, it is determined whether the predicted bounding box intersects with a trash can prediction box. If it does not intersect with a trash can prediction box, the prediction box is discarded, otherwise the prediction bounding box is returned.
8. The method for detecting exposed garbage and overflowing garbage based on deep learning according to claim 1, characterized in that: In S7, when the IoU between the bounding box predicted by the model and the true bounding box is greater than 0.5, the detection is considered correct: For images with garbage, if the model successfully detects all garbage in the image, even if the model returns redundant predicted bounding boxes, it is considered a detection success; For images without garbage, the model does not predict any bounding boxes, which means the detection is successful.
9. The method for detecting exposed garbage and overflowing garbage based on deep learning according to claim 1, characterized in that: In the S8, the locations of exposed garbage and overflowing garbage can be well predicted in garbage houses, transfer stations and outdoor scenarios.