Intelligent networked garbage bin system and garbage placement monitoring method

Abstract

The present invention discloses an intelligent networked garbage bin system and a method for monitoring garbage placement, including: a garbage bin device, a cloud service device and a front-end display device. The garbage bin device includes: a controller, an ultrasonic device for detecting the full bin rate of garbage, a single-line radar for detecting pedestrians, a GPS device for locating the position of the garbage bin, a garbage image acquisition device for acquiring images of the situation inside the garbage bin, a pedestrian image acquisition device for acquiring images of pedestrians around the garbage bin, a bin cover driving device for realizing the opening and closing of the garbage bin, and a storage device for realizing data storage. The garbage bin system disclosed in the present invention has a networking function, and uses cloud service devices to realize data accumulation. Collecting garbage bin data can monitor the working status of the garbage bin on the one hand, and provide data to the algorithm on the other hand to help the algorithm to better train and improve recognition accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent garbage bins, and in particular relates to an intelligent networked garbage bin system and a garbage placement monitoring method. Background Art

[0002] With the development of technology, smart trash cans have gradually entered thousands of communities, but existing smart trash cans have the following problems:

[0003] First, the existing smart trash bin system does not have an Internet connection function and cannot accurately analyze the trash bin data.

[0004] Second, the existing intelligent trash bin system uses very simple sensors and controllers to realize automatic opening and closing of the lid, but the detection accuracy is affected by many factors and the detection effect is unstable. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an intelligent networked garbage bin system and a garbage disposal monitoring method.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] In a first aspect, the present invention discloses an intelligent networked trash bin system, comprising:

[0008] The garbage bin device includes: a controller, an ultrasonic device for detecting the full bin rate of garbage, a single-line radar for detecting pedestrians, a GPS device for locating the position of the garbage bin, a garbage image acquisition device for acquiring images of the situation inside the garbage bin, a pedestrian image acquisition device for acquiring images of pedestrians around the garbage bin, a bin cover driving device for realizing the opening and closing of the garbage bin, and a storage device for realizing data storage. The ultrasonic device, the single-line radar, the GPS device, the garbage image acquisition device, the pedestrian image acquisition device, the bin cover driving device, and the storage device are respectively connected to the controller for communication. The garbage bin device is used to collect data, analyze data, and upload the data to a cloud service device;

[0009] A cloud service device is connected to the garbage bin device network and is used to analyze the uploaded garbage bin fullness rate, GPS information, images inside the garbage bin, and pedestrian images;

[0010] The front-end display device is connected to the cloud service device network to realize data display.

[0011] In the second aspect, the present invention discloses an intelligent network trash bin system, which has networking function and uses cloud service equipment to accumulate data. The collection of trash bin data can monitor the working status of the trash bin on the one hand, and provide data to the algorithm on the other hand to help the algorithm better train and improve recognition accuracy.

[0012] In summary, the present invention discloses a highly efficient, low-cost, universal intelligent network garbage bin system.

[0013] Based on the above technical solution, the following improvements can be made:

[0014] As a preferred solution, the cloud service device includes: a main server and a server and a cloud server respectively connected to the main server for communication;

[0015] The server is used to implement garbage recognition, and the cloud server is used to implement face detection.

[0016] By adopting the above preferred solution, the server and the cloud server each perform their respective functions, reducing the processing pressure on the server.

[0017] As a preferred solution, the server uses the Yolov5 model for spam identification;

[0018] The Yolov5 model compares and identifies the two pictures on the timeline, removes duplicate parts, and obtains the type and location of the current new garbage.

[0019] The adoption of the above preferred solution effectively reduces the waste of network resources and server computing resources and increases the processing speed.

[0020] As a preferred solution, the garbage bin device further includes: a lighting lamp for illuminating the garbage bin, and / or a fan for cooling the controller;

[0021] The lighting lamp and / or the fan are respectively connected to the controller for communication.

[0022] By adopting the above preferred solution, the lighting lamp can provide supplementary light to the environment around the garbage bin according to the time threshold set by the controller. The fan can dissipate heat and cool the controller, extend the service life of the controller, and avoid malfunction of the controller.

[0023] In addition, on the other hand, the present invention also discloses a method for monitoring garbage placement, which uses any of the above-mentioned intelligent networked garbage bin systems to monitor garbage placement, and specifically includes the following steps:

[0024] The ultrasonic device detects the fullness rate of the garbage bin, the single-line radar detects pedestrians, the GPS device locates the position of the garbage bin, the garbage image collection device collects images of the situation inside the garbage bin, and the pedestrian image collection device collects images of pedestrians around the garbage bin;

[0025] The controller analyzes the above data and sends the fullness rate of the trash bin, GPS information, images inside the trash bin, and pedestrian images to the cloud service device;

[0026] The cloud service device analyzes the uploaded data;

[0027] The front-end display device realizes data display.

[0028] The present invention discloses an intelligent network garbage bin method, which utilizes cloud service equipment to realize data accumulation. The garbage bin data collected can monitor the working status of the garbage bin on the one hand, and provide data to the algorithm on the other hand to help the algorithm to better train and improve recognition accuracy.

[0029] As a preferred solution, the controller controls the lid driving device to operate through the following steps to realize the opening and closing of the trash bin:

[0030] The controller cyclically obtains the UDP message of the single-line radar;

[0031] Parse UDP packets into point cloud data;

[0032] Screen and cluster each circle of point cloud data;

[0033] After clustering, calculate the centroid of each cluster;

[0034] Kalman filtering is used to track multiple targets, and the state of each target is analyzed to determine whether it meets the conditions for opening the cover.

[0035] If it meets the requirements, the lid driving device will be activated and the trash can will be opened;

[0036] Otherwise, no action is taken.

[0037] The above preferred solution has a good tracking effect and has low processing requirements on the controller.

[0038] As a preferred solution, Kalman filtering is used for multi-target tracking, and the state of each target is analyzed to determine whether it meets the conditions for opening the cover.

[0039] If it meets the requirements, the lid driving device will be activated and the trash can will be opened;

[0040] Otherwise, no action;

[0041] Specifically include the following:

[0042] Use Kalman filtering to track multiple targets and perform state analysis on each tracked target;

[0043] Determine whether the target is stationary or moving.

[0044] If it is in a stationary state, it is further determined whether the distance between it and the trash can is less than the distance threshold D1.

[0045] If it is less than, the lid driving device will be activated and the trash can will be opened;

[0046] Otherwise, no action;

[0047] If it is in motion, the speed, direction and distance from the trash bin are further determined.

[0048] If the moving speed is greater than the speed threshold, the moving direction is toward the trash bin, and the distance between the trash bin and the trash bin is less than the distance threshold D2, D2>D1, then the bin cover driving device is activated and the trash bin is opened;

[0049] Otherwise, no action is taken.

[0050] By adopting the above preferred solution, the reaction speed of opening the lid of the garbage bin can be adjusted by adjusting the detection distance of the moving target.

[0051] As a preferred solution, after each time the garbage image acquisition device acquires images in the garbage bin, the controller needs to perform image deduplication processing to remove duplicate images in the garbage bin, which specifically includes the following steps:

[0052] Acquire a new image in the trash bin collected by a trash image collection device;

[0053] Obtain a new full box rate of the trash box collected by the ultrasonic device;

[0054] Obtaining the last image in the trash bin and the last full bin rate of the trash bin from the storage device;

[0055] Compare the changes in the images in the trash can and / or the changes in the full box rate to determine whether new trash has been put into the trash can and remove duplicate images.

[0056] The above preferred solution is adopted to avoid the opening and closing of the box cover due to the misjudgment of the controller, which leads to the waste of computing resources for processing the uploaded data. Furthermore, for a trash bin containing two or more trash bins, not all trash bins may have trash in them every time the cover is opened and closed. Image deduplication can remove data without trash.

[0057] As a preferred solution, the changes in the images in the trash bin and / or the changes in the full bin rate are compared to determine whether new trash is put into the trash bin and remove duplicate images; specifically, the following are included:

[0058] Compare whether the change in the full box rate collected by the ultrasonic device is greater than the threshold,

[0059] If it is greater, new garbage is put into the trash can;

[0060] Otherwise, no new garbage is put into the trash bin and the image is deduplicated.

[0061] By adopting the above preferred solution, the method is simple and has low processing requirements on the controller.

[0062] As a preferred solution, the difference hash algorithm is used to compare images in the trash bin and / or the change of the full bin rate to determine whether there is new trash in the trash bin and remove duplicate images, which specifically includes the following steps:

[0063] Preprocess the two bin images;

[0064] The two images in the trash bins are segmented separately according to the set image block size;

[0065] For each image in the trash bin, all image blocks are sorted, and the image blocks close to the center are placed at the front of the queue to form two image block queues;

[0066] Each time, an image block is extracted from each of the two image block queues, and the corresponding dHash value is calculated;

[0067] The Hamming distance is used to compare the difference between the dHash values ​​of two image blocks to determine whether the Hamming distance is greater than the threshold.

[0068] If it is greater than, the difference between the two images in the trash can is large, stop comparing, and new trash is put into the trash can;

[0069] Otherwise, continue to compare the dHash values ​​of the image blocks in the queue until the queue is empty. If the comparison differences of all image blocks are small, no new garbage is put into the trash can and the image is deduplicated.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] 1. Existing smart trash cans are usually designed with very simple sensors and controllers to realize automatic opening and closing of the lid, but the detection accuracy is affected by many factors and the detection effect is unstable. This application uses an ultrasonic device, a single-line radar, a GPS device, a garbage image acquisition device, a pedestrian image acquisition device, a lid drive device, and a storage device to greatly improve the detection accuracy, reduce the probability of false detection, and enhance the user experience;

[0072] 2. The image deduplication algorithm is used to effectively reduce the waste of network resources and server computing resources. Although the image deduplication algorithm is relatively simple to implement, it effectively utilizes the existing system characteristics and can achieve a good balance between processing speed and accuracy.

[0073] 3. The intelligent trash bin system disclosed in the present invention has a networking function. Data accumulation plays an irreplaceable role in the current technological development. Collecting trash bin data can monitor the working status of the trash bin on the one hand, and provide data to the algorithm on the other hand to help the algorithm better train and improve recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0075] Figure 1 This is a functional block diagram of the intelligent networked trash bin system provided in an embodiment of the present invention.

[0076] Figure 2 This is a program initialization diagram of the smart networked trash bin system provided in an embodiment of the present invention.

[0077] Figure 3 This is a program logic diagram of the intelligent networked trash bin system provided in an embodiment of the present invention.

[0078] Figure 4 A schematic diagram of a cloud service device provided in an embodiment of the present invention.

[0079] Figure 5 A schematic diagram of the Kalman filter multi-target tracking principle provided in an embodiment of the present invention.

[0080] Figure 6 A schematic diagram of the motion state analysis principle provided by an embodiment of the present invention.

[0081] Figure 7 This is a schematic diagram of the image deduplication algorithm provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0083] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0084] Using ordinal numbers “first,” “second,” “third,” etc. to describe common objects merely indicates that different instances of similar objects are involved and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.

[0085] In addition, the expression of “comprising” an element is an “open” expression, which merely means that corresponding components or steps exist, and should not be interpreted as excluding additional components or steps.

[0086] In order to achieve the purpose of the present invention, some embodiments of an intelligent networked garbage bin system and a garbage placement monitoring method include: Figure 1 As shown, the intelligent networked trash bin system includes:

[0087] The garbage bin device includes: a controller, an ultrasonic device for detecting the full bin rate of garbage, a single-line radar for detecting pedestrians, a GPS device for locating the position of the garbage bin, a garbage image acquisition device for acquiring images of the situation inside the garbage bin, a pedestrian image acquisition device for acquiring images of pedestrians around the garbage bin, a bin cover driving device for realizing the opening and closing of the garbage bin, and a storage device for realizing data storage. The ultrasonic device, the single-line radar, the GPS device, the garbage image acquisition device, the pedestrian image acquisition device, the bin cover driving device, and the storage device are respectively connected to the controller for communication. The garbage bin device is used to collect data, analyze data, and upload the data to a cloud service device;

[0088] A cloud service device is connected to the garbage bin device network and is used to analyze the uploaded garbage bin fullness rate, GPS information, images inside the garbage bin, and pedestrian images;

[0089] The front-end display device is connected to the cloud service device network to realize data display.

[0090] In some specific embodiments, the garbage image acquisition device is a USB camera, the pedestrian image acquisition device is an RTSP (Real Time Streaming Protocol) camera, and the box cover driving device is a push rod.

[0091] The present invention discloses an intelligent network garbage bin system, which has a networking function and utilizes cloud service equipment to realize data accumulation. The garbage bin data collected can monitor the working status of the garbage bin on the one hand, and provide data to the algorithm on the other hand to help the algorithm to better train and improve recognition accuracy.

[0092] In summary, the present invention discloses a highly efficient, low-cost, universal intelligent network garbage bin system.

[0093] In order to further optimize the implementation effect of the present invention, in some other embodiments, the remaining characteristic technologies are the same, except that, Figure 4 As shown, the cloud service device includes: a main server and servers and cloud servers respectively connected to the main server for communication;

[0094] The server is used to implement garbage recognition, and the cloud server is used to implement face detection.

[0095] The above servers and cloud servers each perform their respective functions, reducing the processing pressure on the servers.

[0096] The cloud service device mainly analyzes the uploaded data, which includes two parts: numbers and pictures.

[0097] The digital part mainly includes the full box rate and GPS information. Through web front-end technology, you can intuitively view the current working status of the trash can: offline / online, location on the map, garbage loading rate. Further, you can monitor the data intelligently and notify the relevant responsible persons to conduct on-site inspections if abnormal data is found.

[0098] The image part is mainly taken by the garbage image collection device and the pedestrian image collection device. The processing of the image needs to consider real-time and accuracy. Face recognition is mainly used to detect faces in the picture and screen out the person who is most likely to throw garbage according to the position of the face detection frame. The face database is used to record and search for face feature vectors. For face-related work, the existing cloud services can meet business needs. The garbage image collection device takes pictures of garbage in the trash can. Directly performing deep learning detection algorithms on it cannot solve the needs of garbage classification detection. The reason is that each detection result may contain garbage that was delivered incorrectly in the past. This is a system feature of the trash can lid opening and closing solution. The detection here requires more detailed processing:

[0099] The first is the problem of detection categories. The garbage classification standards are relatively detailed, but small target detection is not very meaningful, so in the actual labeling process, small targets (<16*16pixel) are not labeled.

[0100] Creating a dataset requires data cleaning: deleting images that were not delivered correctly at all, or images that were delivered correctly. Many images that were not delivered correctly are garbage with indistinguishable boundaries, which makes labeling more difficult.

[0101] Furthermore, the server uses the Yolov5 model for spam identification;

[0102] The Yolov5 model compares and identifies the two pictures on the timeline, removes duplicate parts, and obtains the type and location of the current new garbage.

[0103] Use the Yolov5 model for training. The results of model detection need to be post-processed. Query the latest detection records of the corresponding trash can, and then loop through the class and IOU of the current detection box and the historical detection box. If the class type is the same and IOU>Threshold, the current detection object is judged as historical data, otherwise it is judged as newly detected illegal garbage. All newly detected illegal garbage is converted into json data and returned to the server for display. The detection results of this Yolov5 model are updated to the database for the next detection.

[0104] The use of the Yolov5 model effectively reduces the waste of network resources and server computing resources, and has a fast processing speed.

[0105] In order to further optimize the implementation effect of the present invention, in some other embodiments, the remaining characteristic technologies are the same, the difference is that the garbage bin equipment also includes: a lighting lamp for illuminating the garbage bin and a fan for cooling the controller; the lighting lamp and the fan are respectively communicated with the controller.

[0106] The lighting can supplement the light around the trash bin according to the time threshold set by the controller (or according to the set photosensor). The fan can dissipate heat and cool the controller, extend the service life of the controller, and avoid controller failure. Furthermore, a temperature sensor can be added. The controller detects the temperature around it according to the temperature sensor and controls the fan to dissipate heat according to the real-time temperature. For areas with high temperatures in summer, heat dissipation is an important measure to ensure the normal operation of the equipment.

[0107] The use of lighting can make it easier for users to dispose of garbage at night, further supplement the lighting, and improve the clarity of pictures taken by pedestrian image collection devices.

[0108] The embodiment of the present invention further discloses a method for monitoring garbage placement, which uses the intelligent networked garbage bin system disclosed in any of the above embodiments to monitor garbage placement, and specifically includes the following steps:

[0109] The ultrasonic device detects the fullness rate of the garbage bin, the single-line radar detects pedestrians, the GPS device locates the position of the garbage bin, the garbage image collection device collects images of the situation inside the garbage bin, and the pedestrian image collection device collects images of pedestrians around the garbage bin;

[0110] The controller analyzes the above data and sends the fullness rate of the trash bin, GPS information, images inside the trash bin, and pedestrian images to the cloud service device;

[0111] The cloud service device analyzes the uploaded data;

[0112] The front-end display device realizes data display.

[0113] like Figure 2-3 As shown, further, before monitoring, the intelligent networked trash bin system needs to be initialized.

[0114] Initialize the trash bin device and detect the startup status of each device. For devices that are operating abnormally, log them and upload them to the cloud service device through the network.

[0115] After the device is initialized, multithreading technology is used to open a sub-thread for devices that need to run independently for a long time, and the logic of the device operation is implemented in the sub-thread. The main thread is used to maintain the status of the trash can lid and realize automatic opening and closing of the lid. Devices that need to run sub-threads independently include lighting, fans, single-line radars, and pedestrian image acquisition devices.

[0116] The pedestrian image acquisition device takes pictures of pedestrians delivering garbage, starts a thread to obtain the video stream and stores it in a storage device (such as memory or hard disk), and stores a certain number of video streams according to time. When the shooting signal arrives, another thread extracts frames from the appropriate video stream file and saves it as an image. In the process of delivering garbage, since the camera is in a fixed position, only the initial pedestrian approaching the garbage bin may obtain a better face picture, and the subsequent pictures are taken from above and from the back. By recording the video stream file queue, the picture before the detection signal can be obtained, which improves the accuracy of face detection.

[0117] The present invention discloses an intelligent network garbage bin method, which utilizes cloud service equipment to realize data accumulation. The garbage bin data collected can monitor the working status of the garbage bin on the one hand, and provide data to the algorithm on the other hand to help the algorithm to better train and improve recognition accuracy.

[0118] In order to further optimize the implementation effect of the present invention, in some other embodiments, the remaining characteristic technologies are the same, except that the controller controls the action of the bin cover driving device through the following steps to realize the opening and closing of the trash bin:

[0119] The controller cyclically obtains the UDP message of the single-line radar;

[0120] Parse UDP packets into point cloud data;

[0121] Screen and cluster each circle of point cloud data;

[0122] After clustering, calculate the centroid of each cluster;

[0123] Kalman filtering is used to track multiple targets, and the state of each target is analyzed to determine whether it meets the conditions for opening the cover.

[0124] If it meets the requirements, the lid driving device will be activated and the trash can will be opened;

[0125] Otherwise, no action is taken.

[0126] The above preferred solution has a good tracking effect and has low processing requirements on the controller.

[0127] Furthermore, “multi-target tracking is performed using Kalman filtering, and the state of each target is analyzed to determine whether it meets the conditions for opening the cover.

[0128] If it meets the requirements, the lid driving device will be activated and the trash can will be opened;

[0129] Otherwise, no action is taken;" specifically includes the following:

[0130] Use Kalman filtering to track multiple targets and perform state analysis on each tracked target;

[0131] Determine whether the target is stationary or moving.

[0132] If it is in a stationary state, it is further determined whether the distance between it and the trash can is less than the distance threshold D1.

[0133] If it is less than, the lid driving device will be activated and the trash can will be opened;

[0134] Otherwise, no action;

[0135] If it is in motion, the speed, direction and distance from the trash bin are further determined.

[0136] If the moving speed is greater than the speed threshold, the moving direction is toward the trash bin, and the distance between the trash bin and the trash bin is less than the distance threshold D2, D2>D1, then the bin cover driving device is activated and the trash bin is opened;

[0137] Otherwise, no action is taken.

[0138] By adopting the above preferred solution, the reaction speed of opening the lid of the garbage bin can be adjusted by adjusting the detection distance of the moving target.

[0139] The single-line radar is the core device for realizing the intelligent opening and closing function of the trash can. In the radar sub-thread, UDP messages are continuously obtained, UDP messages are parsed into point cloud data, and each circle of point cloud data is screened and clustered. After clustering, the center of gravity p of each category is calculated, and Kalman filtering is used for multi-target tracking. The state of each target is analyzed. If it meets the conditions for opening the lid, the state variable is updated for access by the main thread. The main algorithm flow of clustering and Kalman filtering is as follows:

[0140] Algorithm 1Proposed ClusteringAlgorithm.

[0141] Procedure Cluster(unlabelDataList)

[0142] while length(unlabelDataList)>0do

[0143] unlabelPoint<-unlabelDataList.front()

[0144] unlabelDataList.pop_front()

[0145] curPoint=pointList.back()

[0146] distance=EuclideanDistance(curPoint,unlabelPoint)

[0147] ifdistance>clusterThresh do

[0148] labelDataLists.push_back(pointList)

[0149] pointList.clear()

[0150] pointList.push_back(unlabelPoint)

[0151] iflength(pointList)do

[0152] labelDataLists.push_back(pointList)

[0153] return labelDataLists

[0154] end procedure

[0155] The above algorithm has a time complexity of O(n) and a space complexity of O(n). Based on "Connected Components for aFast and Robust 2D LIDAR Data Segmentation", the algorithm simplifies the search for connected domains and obtains clustering results in one traversal. In addition, since there is no 4(8) neighborhood search, the memory access efficiency is higher, and a higher processing frame rate can be obtained on embedded devices with limited computing power, thereby achieving better tracking effects.

[0156] For Kalman filter multi-target tracking:

[0157] The SORT (Simple Online and Realtime Tracking) tracking algorithm has extremely high efficiency. The algorithm consists of the Kalman filter algorithm and the Hungarian algorithm. Figure 5 The detection in this article is the clustering process. The centroid coordinates (x, y) of the clustered point cloud are used as the measurement vector, and (x_, y_, xv_, yv_) is the state vector, where x_ / y_ is the position state quantity and xv_ / yv_ is the velocity state quantity.

[0158] Perform status analysis on Matched Tracks. The main analysis logic is as follows: Figure 6 shown.

[0159] Perform status analysis on each tracked target. If the target is stationary, determine whether it meets the detection distance threshold D1. If it does, set Status to True. The main thread determines whether to open or close the lid by accessing the Status value. For moving targets, it is necessary to calculate their movement speed and direction to determine whether the target is moving towards the trash can. On the one hand, determine whether the target's speed is greater than the V2 speed threshold, and on the other hand, determine whether the target's movement angle is toward the trash can. If it enters the detection range D2 (D2>D1), update Status to True. By adjusting the detection distance of the moving target, the reaction speed of opening the trash can lid can be adjusted.

[0160] Furthermore, it is worth noting that when establishing a state machine for the trash can lid, the trash can has four states: close, closestill, open, and openstill, which are described in detail below.

[0161] Close state: the moment the trash can lid is closed, it enters the Close state. At this time, the program needs to call the ultrasonic device to measure the distance and convert it into the full box rate; then call the garbage image acquisition device to take pictures. At this time, the garbage has been thrown into the trash can, so the picture in the bucket can know the changes of the garbage in the trash can; then after the shooting is completed, the camera picture needs to be deduplicated. The following is a detailed description of deduplication.

[0162] After completing the above work, the trash can lid enters the next state: Close still.

[0163] Close still state, the trash can enters the state of continuous closing, at this time the trash can program continuously detects the button status and radar status. If the button is pressed or the radar detection Status is True, the trash can enters the Open state.

[0164] Open state: in this state, the garbage image acquisition device of the trash can starts shooting, and at the same time, the lid driving device starts working to open the trash can lid, and then the trash can state changes to Open still.

[0165] In the Open still state, the trash can enters the continuously open state. The program continuously polls the button status and radar detection status. If no person is detected within a certain period of time, the trash can lid is closed, otherwise the trash can updates the opening time and restarts the timing.

[0166] The present invention utilizes the state machine design mode to implement the garbage bin operation through programming, so as to achieve clear and reasonable logic, realize related functions according to different states, and effectively utilize the controller system resources.

[0167] Furthermore, each time the garbage image acquisition device acquires images in the garbage bin, the controller needs to perform image deduplication processing to remove duplicate images in the garbage bin, which specifically includes the following steps:

[0168] Acquire a new image in the trash bin collected by a trash image collection device;

[0169] Obtain a new full box rate of the trash box collected by the ultrasonic device;

[0170] Obtaining the last image in the trash bin and the last full bin rate of the trash bin from the storage device;

[0171] Compare the changes in the images in the trash can and / or the changes in the full box rate to determine whether new trash has been put into the trash can and remove duplicate images.

[0172] The above preferred solution is adopted to avoid the opening and closing of the box cover due to the misjudgment of the controller, which leads to the waste of computing resources for processing the uploaded data. Furthermore, for a trash bin containing two or more trash bins, not all trash bins may have trash in them every time the cover is opened and closed. Image deduplication can remove data without trash.

[0173] The following are two deduplication methods.

[0174] The first type, "Compare the changes in the images in the trash bin and / or the changes in the full bin rate, determine whether there is new trash in the trash bin, and remove duplicate images" specifically includes the following:

[0175] Compare whether the change in the full box rate collected by the ultrasonic device is greater than the threshold,

[0176] If it is greater, new garbage is put into the trash can;

[0177] Otherwise, no new garbage is put into the trash bin and the image is deduplicated.

[0178] By adopting the above preferred solution, the method is simple and has low processing requirements on the controller.

[0179] The second type, "Compare the changes in the images in the trash bin and / or the changes in the full bin rate, determine whether there is new trash in the trash bin, and remove duplicate images", uses a differential hashing algorithm to compare images, such as Figure 7 As shown, the specific steps include:

[0180] Preprocess the two bin images;

[0181] The two images in the trash bins are segmented separately according to the set image block size;

[0182] For each image in the trash bin, all image blocks are sorted, and the image blocks close to the center are placed at the front of the queue to form two image block queues;

[0183] Each time, an image block is extracted from each of the two image block queues, and the corresponding dHash value is calculated;

[0184] The Hamming distance is used to compare the difference between the dHash values ​​of two image blocks to determine whether the Hamming distance is greater than the threshold.

[0185] If it is greater than, the difference between the two images in the trash can is large, stop comparing, and new trash is put into the trash can;

[0186] Otherwise, continue to compare the dHash values ​​of the image blocks in the queue until the queue is empty. If the comparison differences of all image blocks are small, no new garbage is put into the trash can and the image is deduplicated.

[0187] By adopting the above preferred solution, the accuracy of image deduplication can be improved.

[0188] It is worth noting that the above preprocessing processes the size of the original image and sets the block size. An image block of blocksize*block size corresponds to one pixel of the dhash resized image. If the block size is too large, the difference detection sensitivity is low. If it is too small, the calculation amount is large and the noise resistance is poor.

[0189] The size of each image patch is block size*9x block size*8. The image is divided according to the size of the patch, and then all the patches are sorted, with the patches close to the center at the front of the queue. Each time a patch is obtained from the patch queue, the dHash value is calculated, and then the difference in the dHash values ​​of the two image blocks before and after is compared, and the Hamming distance is used to evaluate the difference between the two dHash values. If the Hamming distance is greater than the threshold, the two images are very different and the comparison is stopped. Otherwise, the comparison of patches in the queue continues until the queue is empty.

[0190] The above multiple implementations can be implemented in parallel.

[0191] In summary, the present invention discloses an intelligent networked garbage bin and a garbage placement monitoring method, which have the following beneficial effects:

[0192] First, in the past, smart trash cans were usually designed with very simple sensors and controllers to achieve automatic opening and closing of the lid, but the detection accuracy was affected by many factors and the detection effect was unstable. This application uses ultrasonic devices, single-line radars, GPS devices, garbage image collection devices, pedestrian image collection devices, lid drive devices, and storage devices to greatly improve detection accuracy, reduce the probability of false detection, and enhance user experience.

[0193] Second, the use of image deduplication algorithms effectively reduces the waste of network resources and server computing resources. Although the image deduplication algorithm is relatively simple to implement, it effectively utilizes the characteristics of the existing system and can achieve a good balance between processing speed and accuracy.

[0194] Third, the intelligent trash can system disclosed in the present invention has an Internet connection function, and data accumulation plays an irreplaceable role in the current technological development. Collecting trash can data can monitor the working status of the trash can on the one hand, and provide data to the algorithm on the other hand to help the algorithm better train and improve recognition accuracy.

[0195] Fourthly, the software solution of the present invention has excellent scalability. The entire software development is carried out around the function of the switch cover. As long as it is a trash can solution for the switch cover, this solution can be applied for expansion to meet the requirements of different sensors and different functions.

[0196] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "screw-on" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0197] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for illustrating the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A garbage placement monitoring method based on an intelligent networked garbage bin system, characterized in that: The intelligent networked trash bin system includes: trash bin equipment, cloud service equipment and front-end display equipment; The garbage bin device comprises: a controller, an ultrasonic device for detecting the full bin rate of garbage, a single-line radar for detecting pedestrians, a GPS device for locating the position of the garbage bin, a garbage image acquisition device for acquiring images of the situation inside the garbage bin, a pedestrian image acquisition device for acquiring images of pedestrians around the garbage bin, a bin cover driving device for realizing the opening and closing of the garbage bin, and a storage device for realizing data storage. The ultrasonic device, the single-line radar, the GPS device, the garbage image acquisition device, the pedestrian image acquisition device, the bin cover driving device, and the storage device are respectively connected to the controller for communication, and the garbage bin device is used to collect data, analyze data, and upload the data to the cloud service device; The cloud service device is connected to the garbage bin device network and is used to analyze the uploaded garbage bin fullness rate, GPS information, images inside the garbage bin, and pedestrian images; The front-end display device is connected to the cloud service device network for realizing data display; The garbage placement monitoring method specifically comprises the following steps: The ultrasonic device detects the fullness rate of the garbage bin, the single-line radar detects pedestrians, the GPS device locates the position of the garbage bin, the garbage image acquisition device acquires images of the situation inside the garbage bin, and the pedestrian image acquisition device acquires images of pedestrians around the garbage bin; The controller analyzes the data obtained in the above steps, and sends the fullness rate of the trash bin, GPS information, images inside the trash bin, and pedestrian images to the cloud service device; The cloud service device analyzes the uploaded data; The front-end display device realizes data display; The controller controls the lid driving device to operate through the following steps to realize the opening and closing of the trash bin: The controller cyclically obtains the UDP message of the single-line radar; Parse UDP packets into point cloud data; Screen and cluster each circle of point cloud data; After clustering, calculate the centroid of each cluster; Kalman filtering is used to track multiple targets, and the state of each target is analyzed to determine whether it meets the conditions for opening the cover. If it meets the requirements, the lid driving device will be activated and the trash can will be opened; Otherwise, no action; Use Kalman filtering to track multiple targets, analyze the state of each target, and determine whether it meets the conditions for opening the cover. If it meets the requirements, the lid driving device will be activated and the trash can will be opened; Otherwise, no action; Use Kalman filtering to track multiple targets and perform state analysis on each tracked target; Determine whether the target is stationary or moving. If it is in a stationary state, it is further determined whether the distance between it and the trash bin is less than the distance threshold D1. If it is less than, the lid driving device will be activated and the trash can will be opened; Otherwise, no action; If it is in motion, the speed, direction and distance from the trash bin are further determined. If the moving speed is greater than the speed threshold, the moving direction is toward the trash bin, and the distance to the trash bin is less than the distance threshold D2, D2>D1, then the bin cover driving device is activated and the trash bin is opened; Otherwise, no action; After the garbage image acquisition device acquires the image in the garbage bin each time, the controller needs to perform image deduplication processing to remove duplicate images in the garbage bin, which specifically includes the following steps: Acquire a new image in the trash bin collected by a trash image collection device; Obtain a new full box rate of the trash box collected by the ultrasonic device; Obtaining the last image in the trash bin and the last full bin rate of the trash bin from the storage device; Compare the changes in the images in the trash bin and / or the changes in the full bin rate to determine whether new trash has been put into the trash bin and remove duplicate images; Compare the changes in the images in the trash bin and / or the changes in the full bin rate to determine whether there is new trash in the trash bin, and use the difference hash algorithm to compare the images in the duplicate images, which specifically includes the following steps: Preprocess the two bin images; The two images in the trash bins are segmented separately according to the set image block size; For each image in the trash bin, all image blocks are sorted, and the image blocks close to the center are placed at the front of the queue to form two image block queues; Each time, an image block is extracted from each of the two image block queues, and the corresponding dHash value is calculated; The Hamming distance is used to compare the difference between the dHash values ​​of two image blocks to determine whether the Hamming distance is greater than the threshold. If it is greater, the difference between the two images in the trash can is large, stop comparing, and new garbage is put into the trash can; otherwise, continue to compare the dHash values ​​of the image blocks in the queue until the queue is empty. If the difference between all image blocks is small, no new garbage is put into the trash can and the image is deduplicated.

2. The method for monitoring garbage placement according to claim 1, characterized in that: The cloud service device includes: a main server and a server and a cloud server respectively connected to the main server for communication; The server is used to realize garbage identification, and the cloud server is used to realize face detection.

3. The method for monitoring garbage placement according to claim 2, characterized in that: The server uses the Yolov5 model to identify garbage; The Yolov5 model compares and identifies the two pictures before and after the timeline, removes duplicate parts, and obtains the type and location of the current new garbage.

4. The method for monitoring garbage placement according to any one of claims 1 to 3, characterized in that: The garbage bin device further includes: a lighting lamp for illuminating the garbage bin, and / or a fan for cooling the controller; The lighting lamp and / or the fan are respectively connected to the controller for communication.

Images