Garbage bin overflow detection method, device, storage medium and electronic device

Abstract

The present invention discloses a method, device, storage medium and electronic device for detecting the overflow state of a trash can, wherein the method comprises: obtaining an image to be detected; performing multi-target detection on the image to be detected to obtain at least one target detection frame; if at least one target detection frame includes a trash can detection frame, then performing target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame; if the overlap between the trash can tracking frame in at least one target tracking frame and other target tracking frames is greater than a preset threshold, then obtaining the current state of the trash can according to the historical state of the trash can. Thus, at least one target tracking frame is obtained through multi-target tracking, and when the overlap between the trash can tracking frame in at least one target tracking frame and other target tracking frames is greater than a preset threshold, the current state of the trash can is determined according to the historical state of the trash can, thereby reducing false alarms caused by occlusion of the trash can.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device, storage medium and electronic device for detecting overflow of a trash can. Background Art

[0002] Currently, most trash can management is still at the stage where cleaning staff regularly clean the trash cans. To reduce the number of times trash cans cannot be cleaned in time, the cleaning interval is usually shortened, but this increases the workload of cleaning staff. To address this issue, related technologies have proposed using computer vision algorithms to alarm when trash cans overflow. Specifically, the video image to be detected is input into a detection model for target detection to obtain a detection result, and the detection result is compared with a detection threshold. When the detection result is greater than the detection threshold, a classification result is obtained based on a classifier. If the classification result is garbage overflow, a prompt message is sent. Although this method can achieve remote monitoring of trash cans, when there are objects such as pedestrians and motor vehicles blocking the trash cans, although the trash cans can be detected, the probability of misclassification and false alarms is greatly increased when the classifier is used for classification, which in turn increases the workload of cleaning staff. Summary of the Invention

[0003] The present invention aims to at least partially address one of the technical problems in the related art. To this end, a first object of the present invention is to provide a method for detecting a trash can overflow. The method uses multi-target tracking to obtain at least one target tracking frame. When the overlap between the trash can tracking frame and other target tracking frames within the at least one target tracking frame exceeds a preset threshold, the method determines the current state of the trash can based on its historical state. This method can reduce false alarms caused by occlusion of the trash can, thereby reducing the workload of cleaning staff.

[0004] A second object of the present invention is to provide a device for detecting an overflow state of a trash can.

[0005] A third object of the present invention is to provide a computer-readable storage medium.

[0006] A fourth object of the present invention is to provide an electronic device.

[0007] To achieve the above-mentioned objectives, an embodiment of the first aspect of the present invention proposes a method for detecting a trash can overflow state, comprising: acquiring an image to be detected; performing multi-target detection on the image to be detected to obtain at least one target detection frame; if the at least one target detection frame includes a trash can detection frame, then performing target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame; if the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold, then obtaining the current state of the trash can based on the historical state of the trash can.

[0008] According to an embodiment of the present invention, a method for detecting an overflowing trash can is employed to obtain an image to be detected and perform multi-target detection on the image to obtain at least one target detection frame. If the at least one target detection frame includes a trash can detection frame, target tracking is performed on each target detection frame within the at least one target detection frame to obtain at least one target tracking frame. If the overlap between the trash can tracking frame within the at least one target tracking frame and other target tracking frames exceeds a preset threshold, the current state of the trash can is determined based on the trash can's historical state. Thus, by obtaining at least one target tracking frame through multi-target tracking, and determining the current state of the trash can based on its historical state when the overlap between the trash can tracking frame within the at least one target tracking frame and other target tracking frames exceeds a preset threshold, false alarms caused by occlusion of the trash can are reduced, thereby reducing the workload of cleaning staff.

[0009] According to one embodiment of the present invention, obtaining the current state of the trash can according to the historical state of the trash can includes: obtaining the historical state of the trash can; if the trash can has a historical state, updating the current state of the trash can to the historical state of the previous frame, and not issuing a trash can overflow alarm reminder; if the trash can has no historical state, updating the current state of the trash can to a non-overflow state, and not issuing a trash can overflow alarm reminder.

[0010] According to one embodiment of the present invention, the method for detecting a trash can overflow state further includes: if the degree of overlap between a trash can tracking frame in at least one target tracking frame and other target tracking frames is less than or equal to a preset threshold, classifying the trash can detection frame to obtain the current state of the trash can.

[0011] According to one embodiment of the present invention, a trash can detection frame is classified to obtain the current state of the trash can, including: performing edge expansion processing on the trash can detection frame; obtaining a trash can image in the trash can detection frame after the edge expansion processing; and performing state classification on the trash can image to obtain the current state of the trash can.

[0012] According to one embodiment of the present invention, after classifying the trash can detection frame and obtaining the current state of the trash can, the method further includes: if the current state of the trash can is an overflow state, obtaining the previous frame historical state of the trash can; if the previous frame historical state of the trash can is a non-overflow state or the trash can has no historical state, performing a trash can overflow alarm reminder; if the previous frame historical state of the trash can is an overflow state, no trash can overflow alarm reminder is performed.

[0013] According to one embodiment of the present invention, multi-target detection is performed on an image to be detected by a target detection model to obtain at least one target detection frame, wherein TensorRT is used to perform forward reasoning on the target detection model.

[0014] According to one embodiment of the present invention, a trash can detection frame is classified by a trash can state classification model to obtain the current state of the trash can, wherein TensorRT is used to perform forward reasoning on the trash can state classification model.

[0015] According to one embodiment of the present invention, the at least one target detection frame includes at least one of a trash can detection frame, a pedestrian detection frame, a motor vehicle detection frame, and a non-motor vehicle detection frame.

[0016] To achieve the above-mentioned objectives, an embodiment of the second aspect of the present invention proposes a trash can overflow status detection device, comprising: an image acquisition module, used to acquire an image to be detected; a target detection module, used to perform multi-target detection on the image to be detected to obtain at least one target detection frame; a target tracking module, used to, if at least one target detection frame includes a trash can detection frame, perform target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame; a state acquisition module, used to, if the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold, obtain the current state of the trash can according to the historical state of the trash can.

[0017] According to an embodiment of the present invention, a device for detecting a trash can overflowing state acquires an image to be detected through an image acquisition module, and performs multi-target detection on the image to be detected through a target detection module to obtain at least one target detection frame. If the at least one target detection frame includes a trash can detection frame, the target tracking module performs target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame. If the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames exceeds a preset threshold, the state acquisition module determines the current state of the trash can based on the trash can's historical state. Thus, by obtaining at least one target tracking frame through multi-target tracking, and when the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames exceeds a preset threshold, the current state of the trash can is determined based on the trash can's historical state. This reduces false alarms caused by occlusion of the trash can, thereby reducing the workload of cleaning staff.

[0018] To achieve the above-mentioned purpose, the third embodiment of the present invention proposes a computer-readable storage medium on which a trash can overflow status detection program is stored. When the trash can overflow status detection program is executed by a processor, it implements the trash can overflow status detection method as in the first embodiment.

[0019] According to the computer-readable storage medium of an embodiment of the present invention, through the above-mentioned trash can overflow state detection method, at least one target tracking frame is obtained through multi-target tracking, and when the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold, the current state of the trash can is determined based on the historical state of the trash can, thereby reducing false alarms caused by occlusion of the trash can and further reducing the workload of cleaning staff.

[0020] To achieve the above-mentioned purpose, the fourth embodiment of the present invention proposes an electronic device, comprising: a memory, a processor, and a trash can overflow status detection program stored in the memory and runnable on the processor. When the processor executes the program, it implements the trash can overflow status detection method as in the first embodiment.

[0021] According to an electronic device according to an embodiment of the present invention, the above-mentioned method for detecting a trash can overflowing state is used to obtain at least one target tracking frame through multi-target tracking. When the degree of overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold, the current state of the trash can is determined based on the historical state of the trash can. This can reduce false alarms caused by occlusion of the trash can, thereby reducing the workload of cleaning staff.

[0022] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 2. FIG. 1 is an application environment diagram of a method for detecting a trash can overflow state according to an embodiment of the present invention;

[0024] Figure 2 is a flowchart of a method for detecting a trash can overflow state according to an embodiment of the present invention;

[0025] Figure 3 is a flow chart of a method for detecting a trash can overflow state according to another embodiment of the present invention;

[0026] Figure 4 Schematic diagram of a device for detecting overflow status of a trash can according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present invention, but are not to be construed as limiting the present invention.

[0028] The garbage bin overflow detection method provided in this application can be applied to Figure 1 In the application environment shown. The camera communicates with the processing device through the network. The camera collects the image of the trash can, obtains the image to be detected, and sends it to the processing device. The processing device obtains the image to be detected and performs multi-target detection on the image to be detected to obtain at least one target detection frame. When at least one target detection frame includes a trash can detection frame, target tracking is performed on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame. When the overlap between the trash can tracking frame and other target tracking frames in the at least one target tracking frame is greater than a preset threshold, the current state of the trash can is obtained according to the historical state of the trash can. It should be noted that the camera can be a high-definition / full-HD starlight camera set around the trash can, etc., which can be shared with cameras of other services, such as smart security / campus cameras, to save installation and maintenance costs, and at the same time realize image sharing; the processing device can be a server or an edge computing box, etc.

[0029] In some embodiments, as Figure 2 As shown, a method for detecting a trash can overflowing state is provided, which is applied to Figure 1 Taking the processing device in the example as an example, the following steps may be included:

[0030] Step S101: Acquire an image to be detected.

[0031] Specifically, when detecting a trash can overflow, the system first determines the camera identification information, threshold information, and other information corresponding to the trash can to be detected. Based on this information, a configuration file, such as a YAML file, is generated and then imported into the processing device. After obtaining the configuration file, the processing device retrieves the image to be detected from the corresponding camera's video stream based on the camera identification information in the configuration file and performs trash can overflow detection.

[0032] Step S102: perform multi-target detection on the image to be detected to obtain at least one target detection frame.

[0033] Specifically, after obtaining the image to be detected, the processing device performs multi-target detection on the image to be detected to obtain at least one target detection frame, wherein the multiple targets may include at least one of a trash can, a pedestrian, a motor vehicle, and a non-motor vehicle, and the corresponding at least one target detection frame may include at least one of a trash can detection frame, a pedestrian detection frame, a motor vehicle detection frame, and a non-motor vehicle detection frame. For example, the image to be detected may be subjected to trash can detection, pedestrian detection, motor vehicle detection, and non-motor vehicle detection. Since the trash can, pedestrian, motor vehicle, and non-motor vehicle may only be partially present in the same image to be detected, the obtained target detection frame may only include a portion of the trash can detection frame, the pedestrian detection frame, the motor vehicle detection frame, and the non-motor vehicle detection frame, as determined by the detection results.

[0034] Optionally, a target detection model can be used to perform multi-target detection on the image to be detected to obtain at least one target detection frame, wherein TensorRT is used to perform forward reasoning on the target detection model.

[0035] Specifically, an initial object detection model can be established and trained using images containing trash cans, pedestrians, motor vehicles, and non-motor vehicles to obtain a trained object detection model. The state-of-the-art open-source detection model YOLOV5 can be used as the initial object detection model for customized training to ensure detection accuracy and throughput performance for trash cans, pedestrians, motor vehicles, and non-motor vehicles. TensorRT can also be used to perform forward inference on the object detection model, optimizing model inference throughput through operator fusion, kernel function optimization, and weight quantization, ensuring real-time multi-object detection and optimal performance within a limited budget. In practical applications, the image to be detected is input into the trained object detection model. The trained object detection model can then identify multiple objects present in the image and generate corresponding object detection frames. The number and types of object detection frames are determined based on the actual image to be detected. This means that the number and types of object detection frames can include multiple, and each object detection frame can also include multiple. For example, if two pedestrians appear in the image to be detected, two pedestrian detection frames can be generated for each person.

[0036] Step S103 : If the at least one target detection frame includes a trash can detection frame, target tracking is performed on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame.

[0037] Specifically, when the processing device detects that there is a trash can in the image to be detected, that is, at least one target detection frame includes a trash can detection frame, the Simple Online And Realtime Tracking (SORT) algorithm can be used to track each target detection frame to obtain a target tracking frame corresponding to each target detection frame, for example, the trash can detection frame corresponds to the trash can tracking frame, the pedestrian detection frame corresponds to the pedestrian tracking frame, the motor vehicle detection frame corresponds to the motor vehicle tracking frame, and the non-motor vehicle detection frame corresponds to the non-motor vehicle tracking frame.

[0038] It should be noted that the multi-target tracking algorithm is an algorithm that uses IoU (Intersection over Union), Kalman filter, Hungarian algorithm, etc. to perform online real-time tracking of detection targets. Each target detection frame obtained is tracked by the multi-target tracking algorithm, so that at least one of the following can be formed: a trash can tracking frame, a pedestrian tracking frame, a motor vehicle tracking frame, and a non-motor vehicle tracking frame. Each target tracking frame may have more than one target tracking frame. For example, two pedestrian tracking frames and three non-motor vehicle tracking frames can be formed according to the image to be detected.

[0039] Step S104 : if the degree of overlap between the trash can tracking frame in at least one target tracking frame and other target tracking frames is greater than a preset threshold, the current state of the trash can is obtained according to the historical state of the trash can.

[0040] Specifically, after obtaining at least one target tracking frame, the processing device determines whether the trash can tracking frame overlaps with other target tracking frames and whether the overlap is greater than a preset threshold. If so, the current state of the trash can is determined based on the trash can's historical state. The overlap is the percentage of the area of ​​the trash can tracking frame that overlaps with all other target tracking frames. The preset threshold can be set based on actual conditions, for example, 10%.

[0041] Furthermore, in some embodiments, the current state of the trash can is obtained based on the historical state of the trash can, including: obtaining the historical state of the trash can; if the trash can has a historical state, updating the current state of the trash can to the historical state of the previous frame, and not issuing a trash can overflow alarm reminder; if the trash can has no historical state, updating the current state of the trash can to a non-overflow state, and not issuing a trash can overflow alarm reminder.

[0042] Specifically, when the degree of overlap between the trash can tracking frame and other target tracking frames exceeds a preset threshold, the processing device obtains the trash can's historical state. If the trash can has a historical state, the current trash can state is updated to the historical state of the previous frame, and no trash can overflow alarm is issued. If the trash can has no historical state, that is, there is no historical data for the trash can in the previous frame, the default is that the trash can is currently in a non-overflow state, and no trash can overflow alarm is issued. In other words, when the degree of overlap between the trash can tracking frame and other target tracking frames exceeds a preset threshold, only the current state of the trash can is updated, and no trash can overflow alarm is issued, thereby preventing false alarms caused by excessive obstruction of the trash can by pedestrians, motor vehicles, and non-motor vehicles.

[0043] In the above embodiment, multi-target detection can be performed on the image to be detected first, and when a trash can is detected in the image to be detected, multi-target tracking can be performed. When there is an area overlap between the trash can tracking frame and other target tracking frames in the multi-target tracking and the overlap rate is higher than a preset threshold, the current state of the trash can is determined based on the historical data of the trash can, that is, the obscured trash can is screened out, and the current state of the trash can is determined based on the historical data of the trash can without classifying the trash can, thereby effectively reducing the probability of misclassification and false alarm caused by occlusion, and improving the accuracy of trash can overflow state detection; at the same time, by real-time tracking of the trash can, it can be ensured that the trash can is always the same trash can, which can improve the reliability of trash can overflow state detection, and by real-time tracking of pedestrians, motor vehicles, and non-motor vehicles, reusable detection results can be provided for other businesses of smart parks or smart cities, such as intrusion into restricted areas, illegal parking of motor vehicles, illegal parking of non-motor vehicles, etc., thereby increasing the scalability of the trash can overflow state detection business scenario.

[0044] In some embodiments, the above-mentioned trash can overflow state detection method further includes: if the overlap between the trash can tracking frame in at least one target tracking frame and other target tracking frames is less than or equal to a preset threshold, the trash can detection frame is classified to obtain the current state of the trash can.

[0045] Specifically, the processing device may first acquire an image to be detected, and perform multi-target detection on the image to be detected to obtain at least one target detection frame. When the at least one target detection frame includes a trash can detection frame, target tracking is performed on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame. Then, the processing device determines whether there is an area overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames, and whether the overlap is greater than a preset threshold. If not, the trash can detection frame is classified to obtain the current state of the trash can. It should be noted that if there is only a trash can tracking frame in the at least one target tracking frame, the trash can detection frame is also classified to obtain the current state of the trash can. Therefore, when the trash can is not blocked or the blocked area is very small, the current state of the trash can is obtained by using a classification processing method to ensure that the actual state of the trash can can be followed in real time.

[0046] Furthermore, in some embodiments, the trash can detection frame is classified to obtain the current state of the trash can, including: performing edge expansion processing on the trash can detection frame; obtaining the trash can image in the trash can detection frame after the edge expansion processing; and performing state classification on the trash can image to obtain the current state of the trash can.

[0047] Specifically, when the processing device classifies the trash can detection frame, it can first expand the side length of the trash can detection frame by a certain multiple, such as one times, and extract the trash can image from the expanded trash can detection frame. This can ensure the integrity of the trash can image as much as possible, and then perform state classification on the trash can image to obtain the current state of the trash can, which can improve the accuracy of the classification processing, and then improve the accuracy of the trash can detection, and reduce the probability of misjudgment and false alarm.

[0048] Optionally, the trash can detection frame can be classified by a trash can state classification model to obtain the current state of the trash can, wherein TensorRT is used to perform forward inference on the trash can state classification model.

[0049] Specifically, an initial trash can state classification model can be established first, and then trained using images of trash cans in overflow and non-overflow states to obtain a trained trash can state classification model. ResNet18, with its relatively balanced performance, can be used as the initial trash can state classification model for training to ensure the accuracy of overflow and non-overflow state judgments and throughput performance. At the same time, TensorRT can be used to perform forward inference on the trash can state classification model, performing operations such as operator fusion, kernel function optimization, and weight quantization to optimize the model's inference throughput, to ensure real-time trash can state classification and optimal performance within a limited budget. In actual use, the trash can image is input into the trained trash can state classification model, and the state of the trash can is obtained through the trained trash can state classification model.

[0050] Furthermore, in some embodiments, after classifying the trash can detection frame and obtaining the current state of the trash can, the method also includes: if the current state of the trash can is an overflow state, obtaining the previous frame historical state of the trash can; if the previous frame historical state of the trash can is a non-overflow state or the trash can has no historical state, performing a trash can overflow alarm reminder; if the previous frame historical state of the trash can is an overflow state, no trash can overflow alarm reminder is performed.

[0051] Specifically, after the processing equipment classifies the trash can detection frame and obtains the current state of the trash can, it also judges the current state of the trash can. If the current state of the trash can is an overflow state, the previous frame historical state of the trash can is first obtained. If the previous frame historical state of the trash can is a non-overflow state or the trash can does not have a historical state, it means that the trash can has changed from a non-overflow state to an overflow state. At this time, a trash can overflow alarm reminder is issued. If the previous frame historical state of the trash can is an overflow state, it means that a trash can overflow alarm reminder has been issued. At this time, the trash can overflow alarm reminder is no longer issued, thereby avoiding continuous alarm of the trash can overflow state and reducing the situation where continuous alarm does not play a role in assisting the monitoring personnel but increases the burden on the monitoring personnel.

[0052] As a specific example, see Figure 3 As shown, the method for detecting a trash can overflowing state may include the following steps:

[0053] Step S201: Acquire an image to be detected.

[0054] Specifically, the image to be detected can be obtained from the video stream of the camera corresponding to the trash can to be detected.

[0055] Step S202: perform multi-target detection on the image to be detected to obtain at least one target detection frame.

[0056] Specifically, after obtaining the image to be detected, the pre-trained target detection model can be used to perform target detection on the image to be detected, such as trash cans, pedestrians, motor vehicles and non-motor vehicles, to obtain at least one target detection frame, such as one or more of the trash can detection frame, pedestrian detection frame, motor vehicle detection frame and non-motor vehicle detection frame.

[0057] Step S203: Determine whether there is a trash can detection frame. If yes, go to step S204; otherwise, go to step S217.

[0058] Step S204: perform target tracking on each target detection frame to obtain a corresponding target tracking frame.

[0059] Specifically, a multi-target tracking algorithm can be used to track each target detection frame, thereby forming a one-to-one corresponding target tracking frame, such as a trash can tracking frame, a pedestrian tracking frame, and the like.

[0060] Step S205: Determine whether the other target tracking frames overlap with the trash can tracking frame and whether the degree of overlap is greater than a preset threshold. If so, proceed to step S206; otherwise, proceed to step S210.

[0061] Step S206: Determine whether the trash can has a history state. If yes, proceed to step S207; otherwise, proceed to step S208.

[0062] Step S207: Update the current state of the trash can to the previous frame's historical state. That is, if the trash can has a historical state, then update the current trash can state to the previous frame's historical state.

[0063] Step S208: Update the current state of the trash can to a non-overflow state. That is, if the trash can has no historical state, the current state of the trash can is updated to a non-overflow state.

[0064] Step S209: No trash can overflow alarm is issued. That is, when the overlap between the trash can tracking frame and other target tracking frames exceeds a preset threshold, only the current state of the trash can is updated without issuing a trash can overflow alarm, thereby preventing false alarms caused by excessive obstruction of the trash can by pedestrians, motor vehicles, and non-motor vehicles.

[0065] Step S210 , performing edge expansion processing on the trash can detection frame to obtain a trash can image in the trash can detection frame after the edge expansion processing.

[0066] Specifically, when the other target tracking frames have no area overlap with the trash can tracking frame or the area overlap is less than or equal to a preset threshold, the edge of the trash can detection frame is expanded, and the trash can image is extracted from the expanded trash can detection frame.

[0067] Step S211 , classifying the status of the trash can image to obtain the current status of the trash can.

[0068] Specifically, the trash can image can be input into a pre-trained trash can state classification model to obtain the current state of the trash can, such as overflow state or non-overflow state.

[0069] Step S212: Determine whether the trash can is overflowing. If so, proceed to step S213; otherwise, return to step S208.

[0070] Step S213: Update the current state of the trash can to an overflow state.

[0071] Step S214: Determine whether there is a previous frame history state and whether it is an overflow state. If yes, return to step S209; otherwise, execute step S215.

[0072] Step S215: issuing a trash bin overflow alarm.

[0073] Specifically, if the trash can is currently in an overflow state, and the trash can's previous historical state is a non-overflow state or the trash can does not have a historical state, it means that the trash can has changed from an initial non-overflow state to an overflow state, and thus a trash can overflow alarm is issued.

[0074] Step S216: Update the trash can status of the current frame to the history status, that is, save the current trash can status and use it as the history status of the previous frame for next use.

[0075] Step S217, waiting for the next frame of decoded data.

[0076] Therefore, when the degree of overlap between the trash can tracking frame and other target tracking frames is greater than a preset threshold, only the current state of the trash can is updated, and no trash can overflow alarm is issued, thereby preventing false alarms caused by excessive obstruction of the trash can by pedestrians, motor vehicles and non-motor vehicles; at the same time, when the degree of overlap between the trash can tracking frame and other target tracking frames is less than or equal to a preset threshold, a trash can overflow alarm will only be issued when the trash can changes from a non-overflow state to an overflow state. If the previous frame historical state of the trash can is an overflow state, indicating that the trash can has been in an overflow state for a period of time, no trash can overflow alarm will be issued, thereby solving the problem of continuous alarm of the trash can overflow state.

[0077] In summary, according to an embodiment of the present invention, a method for detecting an overflowing trash can is employed to obtain an image to be detected and perform multi-target detection on the image to obtain at least one target detection frame. If the at least one target detection frame includes a trash can detection frame, target tracking is performed on each target detection frame within the at least one target detection frame to obtain at least one target tracking frame. If the overlap between the trash can tracking frame within the at least one target tracking frame and other target tracking frames exceeds a preset threshold, the current state of the trash can is determined based on the trash can's historical state. Thus, by obtaining at least one target tracking frame through multi-target tracking, and when the overlap between the trash can tracking frame within the at least one target tracking frame and other target tracking frames exceeds a preset threshold, the current state of the trash can is determined based on the trash can's historical state. This reduces false alarms caused by occlusion of the trash can, thereby reducing the workload of cleaning staff.

[0078] Figure 4 FIG. 1 is a schematic diagram of a structure of a device for detecting overflow status of a trash can according to an embodiment of the present invention. Figure 4 As shown, the trash can overflow state detection device 100 includes: an image acquisition module 110 , a target detection module 120 , a target tracking module 130 and a state acquisition module 140 .

[0079] Among them, the image acquisition module 110 is used to acquire the image to be detected; the target detection module 120 is used to perform multi-target detection on the image to be detected to obtain at least one target detection frame; the target tracking module 130 is used to, if at least one target detection frame includes a trash can detection frame, perform target tracking on each target detection frame in at least one target detection frame to obtain at least one target tracking frame; the state acquisition module 140 is used to, if the overlap between the trash can tracking frame in at least one target tracking frame and other target tracking frames is greater than a preset threshold, obtain the current state of the trash can according to the historical state of the trash can.

[0080] In some embodiments, the status acquisition module 140 is specifically used to: obtain the historical status of the trash can; if the trash can has a historical status, then update the current status of the trash can to the historical status of the previous frame, and do not issue a trash can overflow alarm reminder; if the trash can does not have a historical status, then update the current status of the trash can to a non-overflow status, and do not issue a trash can overflow alarm reminder.

[0081] In some embodiments, the state acquisition module 140 is further configured to: if the degree of overlap between the trash can tracking frame in at least one target tracking frame and other target tracking frames is less than or equal to a preset threshold, classify the trash can detection frame to obtain the current state of the trash can.

[0082] In some embodiments, the state acquisition module 140 is further used to: perform edge expansion processing on the trash can detection frame; obtain the trash can image in the trash can detection frame after edge expansion processing; and perform state classification on the trash can image to obtain the current state of the trash can.

[0083] In some embodiments, the status acquisition module 140 is also used for: after classifying the trash can detection frame and obtaining the current status of the trash can, if the current status of the trash can is an overflow status, obtaining the previous frame historical status of the trash can; if the previous frame historical status of the trash can is a non-overflow status or the trash can has no historical status, a trash can overflow alarm reminder is issued; if the previous frame historical status of the trash can is an overflow status, no trash can overflow alarm reminder is issued.

[0084] In some embodiments, the target detection module 120 is specifically used to: perform multi-target detection on the image to be detected through a target detection model to obtain at least one target detection frame, wherein TensorRT is used to perform forward reasoning on the target detection model.

[0085] In some embodiments, the state acquisition module 140 is specifically used to: classify the trash can detection frame through the trash can state classification model to obtain the current state of the trash can, wherein TensorRT is used to perform forward reasoning on the trash can state classification model.

[0086] In some embodiments, the at least one target detection frame includes at least one of a trash can detection frame, a pedestrian detection frame, a motor vehicle detection frame, and a non-motor vehicle detection frame.

[0087] It should be noted that for the description of the trash can overflow status detection device in this application, please refer to the description of the trash can overflow status detection method in this application, and the details will not be repeated here.

[0088] According to an embodiment of the present invention, a device for detecting a trash can overflowing state acquires an image to be detected through an image acquisition module, and performs multi-target detection on the image to be detected through a target detection module to obtain at least one target detection frame. If the at least one target detection frame includes a trash can detection frame, the target tracking module performs target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame. If the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames exceeds a preset threshold, the state acquisition module determines the current state of the trash can based on the trash can's historical state. Thus, by obtaining at least one target tracking frame through multi-target tracking, and when the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames exceeds a preset threshold, the current state of the trash can is determined based on the trash can's historical state. This reduces false alarms caused by occlusion of the trash can, thereby reducing the workload of cleaning staff.

[0089] An embodiment of the present invention further provides a computer-readable storage medium storing a trash can overflow status detection program. When the trash can overflow status detection program is executed by a processor, the trash can overflow status detection method described above is implemented.

[0090] According to the computer-readable storage medium of an embodiment of the present invention, through the above-mentioned trash can overflow state detection method, at least one target tracking frame is obtained through multi-target tracking, and when the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold, the current state of the trash can is determined based on the historical state of the trash can, thereby reducing false alarms caused by occlusion of the trash can and further reducing the workload of cleaning staff.

[0091] An embodiment of the present invention further provides an electronic device comprising: a memory, a processor, and a trash can overflow status detection program stored in the memory and executable on the processor. When the processor executes the program, the trash can overflow status detection method described above is implemented.

[0092] According to an electronic device according to an embodiment of the present invention, through the above-mentioned trash can overflow state detection method, multi-target tracking obtains at least one target tracking frame, and when the overlap between the trash can tracking frame and other target tracking frames in at least one target tracking frame is greater than a preset threshold, the current state of the trash can is determined based on the historical state of the trash can, thereby reducing false alarms caused by occlusion of the trash can and further reducing the workload of cleaning staff.

[0093] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0094] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0095] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0096] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0097] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0098] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for detecting a trash can overflow state, characterized in that: The method comprises: Acquire the image to be detected; Performing multi-target detection on the image to be detected to obtain at least one target detection frame; If the at least one target detection frame includes a trash can detection frame, performing target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame; If the degree of overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold, the current state of the trash can is acquired according to the historical state of the trash can.

2. The method for detecting overflow status of a trash can according to claim 1, characterized in that: The obtaining the current state of the trash can according to the historical state of the trash can includes: Get the historical status of the trash can; If the trash can has a historical state, the current state of the trash can is updated to the historical state of the previous frame, and no trash can overflow alarm reminder is issued; If the trash can has no historical status, the current status of the trash can is updated to a non-overflowing status, and no trash can overflow alarm reminder is performed.

3. The method for detecting overflow status of a trash can according to claim 1, characterized in that: The method further comprises: If the degree of overlap between the trash can tracking frame in the at least one target tracking frame and the other target tracking frames is less than or equal to the preset threshold, the trash can detection frame is classified to obtain the current state of the trash can.

4. The method for detecting overflow status of a trash can according to claim 3, characterized in that: The classifying process of the trash can detection frame to obtain the current state of the trash can includes: Performing edge enlargement processing on the trash can detection frame; Obtain the trash can image in the trash can detection frame after edge enlargement processing; The state of the trash can image is classified to obtain the current state of the trash can.

5. The method for detecting overflow status of a trash can according to claim 3, characterized in that: After classifying the trash can detection frame to obtain the current state of the trash can, the method further includes: If the current state of the trash can is an overflow state, obtaining the previous frame history state of the trash can; If the previous frame history state of the trash can is a non-overflow state or the trash can has no history state, a trash can overflow alarm is issued; If the previous frame history state of the trash can is an overflow state, no trash can overflow alarm reminder is performed.

6. The method for detecting overflow status of a trash can according to claim 1, characterized in that: Multi-target detection is performed on the image to be detected through a target detection model to obtain at least one target detection frame, wherein TensorRT is used to perform forward reasoning on the target detection model.

7. The method for detecting overflow status of a trash can according to claim 3, characterized in that: The trash can detection frame is classified and processed by a trash can state classification model to obtain the current state of the trash can, wherein TensorRT is used to perform forward reasoning on the trash can state classification model.

8. The method for detecting overflow status of a trash can according to any one of claims 1 to 7, characterized in that: The at least one target detection frame includes at least one of a trash can detection frame, a pedestrian detection frame, a motor vehicle detection frame, and a non-motor vehicle detection frame.

9. A garbage bin overflow detection device, characterized in that: The device comprises: An image acquisition module, used for acquiring an image to be detected; An object detection module is used to perform multi-object detection on the image to be detected to obtain at least one object detection frame; a target tracking module, configured to, if the at least one target detection frame includes a trash can detection frame, perform target tracking on each target detection frame in the at least one target detection frame to obtain at least one target tracking frame; The state acquisition module is used to acquire the current state of the trash can according to the historical state of the trash can if the overlap between the trash can tracking frame in the at least one target tracking frame and other target tracking frames is greater than a preset threshold.

10. A computer-readable storage medium, characterized in that: A trash can overflow state detection program is stored thereon, and when the trash can overflow state detection program is executed by a processor, a trash can overflow state detection method according to any one of claims 1-8 is implemented.

11. An electronic device, characterized in that: include: A memory, a processor, and a trash can overflow status detection program stored in the memory and executable on the processor, wherein when the processor executes the program, a trash can overflow status detection method according to any one of claims 1 to 8 is implemented.

Images