An intelligent garbage recycling station monitoring method and system based on target tracking

Abstract

The present invention discloses an intelligent garbage recycling station monitoring method and system based on target tracking, specifically related to the field of garbage recycling monitoring, including installing a shooting unit in a target area and collecting image data; the target area includes fixed points and moving points; the fixed points include garbage stations; the moving points include garbage collection vehicles; based on the collected image data, detecting garbage and personnel through YOLOv5, and combining ByteTrack target tracking to maintain continuous monitoring of multiple garbage targets; during continuous monitoring, if the garbage status is determined to be abnormal, an alarm is issued and the moving point is prompted to perform recycling. A hierarchical scheduling mechanism is formed through first-level scheduling + second-level scheduling, and scheduling is triggered when the filling state of the trash can exceeds the threshold, without frequently recalculating the global path due to slight weight changes, thereby reducing the consumption of computing resources.

Description

Technical Field

[0001] The present invention relates to the technical field of garbage collection monitoring. More specifically, the present invention relates to an intelligent garbage recycling station monitoring method and system based on target tracking. Background Art

[0002] In the process of detecting and dispatching garbage sites in the prior art, manual monitoring and manual dispatching are usually relied on, which not only increases the labor cost, but is also easily affected by problems such as human error, response delay, and unbalanced dispatching, resulting in low garbage collection efficiency and being unfavorable to the garbage management requirements;

[0003] In the prior art, the publication number is CN114372746A, which discloses a classified garbage intelligent transportation management system and method, which relies on the dynamic monitoring of the weight of trash cans to optimize transportation dispatching, and the reference variables are relatively limited, which is not conducive to actual use;

[0004] In addition, in the solution of the publication number CN114372746A, every time the weight changes, the route recalculation and vehicle rescheduling will be triggered. Since the weight of the trash can may change frequently, the system needs to frequently perform global path optimization and vehicle allocation. This strategy may be feasible in small-scale pilot tests, but in large-scale urban-level deployments, it will consume excessive computing resources. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an intelligent garbage recycling station monitoring method and system based on target tracking, which forms a hierarchical scheduling mechanism through primary scheduling + secondary scheduling, and triggers scheduling when the filling state of the garbage site exceeds the threshold, rather than frequently recalculating the global path due to slight weight changes, so as to solve the problems raised in the above background art.

[0006] To achieve the above object, the present invention provides the following technical solution: An intelligent garbage recycling station monitoring method based on target tracking, including a shooting unit;

[0007] Install the shooting unit in the target area and collect image data; the target area includes fixed points and moving points; the fixed points include garbage sites; the moving points include garbage collection vehicles;

[0008] Based on the collected image data, detect garbage and personnel through YOLOv5, and continuously monitor multiple garbage targets by combining ByteTrack target tracking;

[0009] During continuous monitoring, if the garbage state is determined to be abnormal, an alarm is issued and the moving point is prompted to perform recycling;

[0010] If the garbage status of the fixed point is determined to be abnormal, a first-level scheduling signal is sent to the scheduling center;

[0011] If any fixed point's garbage status is recognized as abnormal during the movement of the moving point, a second-level scheduling signal is sent to the scheduling center;

[0012] Based on the first-level and second-level scheduling signals, the scheduling center calculates the scheduling execution recovery plan for the moving point;

[0013] In ByteTrack object tracking, the goal of object tracking is to maintain the uniqueness and continuity of the detected objects in the video stream, so that each object is correctly tracked on the time axis. Object tracking relies on Kalman filtering for object state prediction and combines the Hungarian algorithm for object matching. Finally, a confidence-based object fusion strategy is adopted to ensure tracking stability;

[0014] Object tracking includes modeling and predicting the object state based on Kalman filtering; Kalman filtering is used to predict the position of the object in the next frame; First, the object state is represented. In the multi-object tracking task, the state vector of each object is modeled as:

[0015] ;

[0016] where is the center coordinate of the object; is the scale of the object; is the aspect ratio of the object; are the velocity components of the object respectively;

[0017] Secondly, the object state is updated; Assume that the movement of the object is represented based on a linear model;

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] where is the state vector of the object at the th frame; is the state transition matrix; represents the time interval between two frames; is the control input; is the process noise; is the observation value, that is, the position of the detected object bounding box; is the observation matrix; the observation matrix represents the observed values, which include position and scale information;

[0023] Object tracking includes implementing object matching based on the Hungarian algorithm; the goal of object matching is to associate the objects in the current frame with the tracked objects in the previous item, so that the object ID remains unique in the time series. For the object matching calculation, the object matching is based on calculating the matching degree;

[0024] ;

[0025] where is the matching degree between in the current frame and in the previous frame; where respectively represent the th candidate detection box and the th candidate detection box;

[0026] In the object matching process, calculate the relationship matrix between the objects in the current frame and the objects in the previous frame, and use the Hungarian algorithm to solve the object matching, which is expressed as:

[0027] ;

[0028] where is a binary decision variable. If matches , then , otherwise it is 0;

[0029] The constraint conditions include:

[0030] ;

[0031] The constraint conditions are used to make each object match an object in the previous frame;

[0032] Object tracking includes object fusion, and the object fusion adopts a confidence fusion strategy;

[0033] For the generation of the primary scheduling signal and the secondary scheduling signal, when it is detected that the filling of the garbage site exceeds the preset threshold, the corresponding scheduling signal is triggered;

[0034] In the generation of the primary scheduling signal based on fixed points, if the filling volume of the fixed-point garbage site exceeds the threshold, then:

[0035] ;

[0036] where is the set of primary scheduling signals; is the number of the garbage site; are the volume and weight overflow thresholds respectively; are the maximum capacity and load-bearing of the garbage site respectively; is the monitoring point indicator. If , it means that this monitoring point is a fixed point. If , it means that this monitoring point is a mobile point;

[0037] In the generation of secondary scheduling signals based on mobile points, if it is detected that the garbage site is overflowing during the driving of the garbage truck, a secondary scheduling signal is triggered, which is expressed as: or ;

[0038] where is the set of secondary scheduling signals; the primary scheduling signal is triggered based on a fixed point and requires dispatching the garbage truck for cleaning. The secondary scheduling signal is triggered based on a mobile point, and the garbage truck can handle it by itself or request additional support;

[0039] Based on the dispatching center, calculate the cleaning plan for the mobile point. Based on the primary scheduling signal and the secondary scheduling signal, calculate the garbage collection path;

[0040] In the calculation of task priorities, the dispatching center calculates the cleaning priority of the garbage site and formulates indicating the task priority score of the garbage site ;

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] where is the influence factor of the overflow severity; is the influence factor of the geographical distance from the garbage site to the garbage truck; is the influence factor of the garbage category; is the influence factor of the weight of the garbage site; is the garbage site and the garbage truck the geographical distance between; is the garbage site geographical location coordinates; is the garbage site binary indicator function of whether it is abnormal; is the garbage site GPS coordinates; is the garbage truck GPS coordinates; is a garbage site Emergency level score of garbage for all candidate detection boxes within; is the category number of garbage in the is the urgent handling weight of garbage in the

[0046] In vehicle task allocation, the cleaning task of the garbage truck is expressed as:

[0047] ;

[0048] ;

[0049] where is the path of the garbage truck ; is the overload penalty item for the garbage truck; represents finding a path that can minimize the cleaning cost ; represents the total driving distance for the garbage truck to visit garbage sites in sequence; is the overload penalty factor; is the limit of the garbage truck's carrying capacity;

[0050] During the garbage truck's cleaning operation, when the garbage truck cleans the garbage site , the updated remaining garbage volume and remaining garbage weight of the garbage site are expressed as:

[0051] ;

[0052] ;

[0053] where is the garbage volume currently cleaned by the garbage truck ; is the garbage weight currently cleaned by the garbage truck ; is the garbage volume of the garbage site at time ; is the garbage weight of the garbage site at time ;

[0054] When the garbage truck reaches the carrying capacity limit, it returns to the garbage station for unloading. This condition is judged as: ; where For the current Total volume of the loaded garbage;

[0055] In the transportation status update, when the garbage truck returns to the main station for unloading, the load status of the garbage truck will be reset;

[0056] ;

[0057] Among them is the garbage truck at time moment's load volume; is the garbage truck at time moment's load weight; when the garbage truck finishes unloading, the load volume and load weight are cleared to re - execute the next round of garbage collection tasks.

[0058] In a preferred embodiment, the monitoring types of fixed points include garbage type and garbage quantity; the monitoring types of mobile points include garbage quantity;

[0059] In the detection of garbage and personnel based on YOLOv5, it includes target image pre - processing; the input image is normalized, channel - converted, resized, and data - augmented to adapt to the input layer of YOLOv5; it is proposed that represents the normalized image; ; represents the minimum and maximum pixel values;

[0060] In adapting to the input size of YOLOv5, scale adjustment is performed based on bilinear interpolation, and it is proposed that represents the adjusted image, that is, the image after scaling to meet the network input requirements, and through the scaling function the input image is adjusted to the specified target size; ; , respectively represent the expected input width and height of YOLOv5;

[0061] In the detection of garbage and personnel based on YOLOv5, it includes feature extraction; the input image is processed by the backbone feature extraction network of YOLOv5 to extract features at different levels;

[0062] ;

[0063] Among them is the input image; is the convolution kernel; is the output feature map; among which, the backbone feature extraction network Backbone is responsible for extracting features and uses residual connections to enhance the information flow ability, enabling the deep network to learn more complex features;

[0064] In the detection of garbage and people based on YOLOv5, it includes a feature fusion part; the feature fusion part based on the FPN structure combined with the PAN structure is used for multi-scale feature fusion;

[0065] ;

[0066] Among which is the feature map of the current layer; represents the feature map of the bottom layer; is the feature map of the previous layer; represents the upsampling operation; is the convolution operation;

[0067] In the detection of garbage and people based on YOLOv5, it includes an object detection and bounding box prediction part; in the object detection and bounding box prediction part, the network is responsible for outputting the class label, bounding box coordinates, and confidence of the object;

[0068] ;

[0069] Among which is the center coordinate of the object; respectively represent the width and height of the object; represents the class confidence of the object; for all the bounding boxes of object detection, the total loss function is calculated :

[0070] ;

[0071] Among which is the object classification loss; is the bounding box loss; is the object confidence loss; is the loss weight;

[0072] By introducing non-maximum suppression NMS during the object detection process, the intersection over union between the detection bounding boxes is calculated; non-maximum suppression NMS is used to remove duplicate bounding boxes and retain the object boxes;

[0073] ;

[0074] Among which respectively represent the th candidate detection box and the th candidate detection box; is the intersection over union; Indicates two candidate detection boxes and the area of the overlapping region; Indicates the area of the union of two candidate detection boxes.

[0075] In a preferred embodiment, during the monitoring of the garbage site, a weighing sensor is installed at a fixed point, and the YOLOv5 is combined with ByteTrack object tracking and the weighing sensor to provide a first feature variable and a second feature variable; the first feature variable is the garbage type , and the second feature variable is the garbage quantity, and the garbage quantity includes the total garbage volume of the current garbage site and the garbage site total weight ;

[0076] The detection result output based on YOLOv5 is expressed as:

[0077] ;

[0078] where is the garbage data set detected by the garbage site ; is the total number of candidate detection boxes detected in the th garbage site; is the bounding box coordinates of the garbage in the th candidate detection box; is the classification result of the garbage in the th candidate detection box; is the detection confidence of the garbage in the

[0079] Using the bounding box information of object detection and combining with camera depth calculation, estimate the volume of the garbage:

[0080] ;

[0081] where is the depth information of the object; is the volume conversion coefficient of the garbage type; is the volume adjustment coefficient;

[0082] For the garbage site at the fixed point, obtain the actual weight based on the installed weighing sensor:

[0083] ;

[0084] where is the total weight measured by the weighing sensor; is the time step; The density conversion coefficient for the garbage type.

[0085] An intelligent garbage recycling station monitoring system based on target tracking further includes a data acquisition module, a target detection module, an anomaly monitoring module, a primary scheduling module, a secondary scheduling module, and a scheduling execution module;

[0086] The data acquisition module is used to install the shooting unit in the target area and collect image data; the target area includes fixed points and moving points; the fixed points include garbage sites; the moving points include garbage collection vehicles;

[0087] The target detection module detects garbage and personnel based on the collected image data through YOLOv5, and combines ByteTrack target tracking to maintain continuous monitoring of multiple garbage targets;

[0088] In the continuous monitoring, if the garbage status is determined to be abnormal, the anomaly monitoring module issues an alarm and prompts the moving point to perform recycling;

[0089] The primary scheduling module includes: if the garbage status at the fixed point is determined to be abnormal, a primary scheduling signal is sent to the scheduling center;

[0090] The secondary scheduling module includes: if any fixed point's garbage status is identified as abnormal during the movement of the moving point, a secondary scheduling signal is sent to the scheduling center;

[0091] The scheduling execution module calculates the scheduling execution recycling plan for the moving point through the scheduling center based on the primary scheduling signal and the secondary scheduling signal.

[0092] The technical effects and advantages of the present invention:

[0093] 1. A hierarchical scheduling mechanism is formed through primary scheduling + secondary scheduling, triggering scheduling when the filling state of the garbage site exceeds the threshold, rather than frequently recalculating the global path due to slight weight changes, thereby reducing the consumption of computing resources;

[0094] 2. Combining double-layer monitoring of fixed points and moving points, the fixed point only triggers scheduling when there is garbage overflow, while the moving point dynamically monitors the status of the garbage site during driving and autonomously decides whether to directly clear or report to the scheduling center based on the real-time load, avoiding frequent changes in the global path;

[0095] 3. By detecting the garbage type through YOLOv5 and combining with a weighing sensor to detect the garbage volume, multi-modal fusion monitoring of garbage categories, garbage volume, and weight is achieved; the garbage collection vehicle can not only evaluate the status of the garbage site based on visual detection but also make decisions in combination with actual weighing data, thus avoiding misjudgments that may be caused by relying solely on image recognition. Description of the Drawings

[0096] Figure 1 This is a schematic diagram of the system modules of the present invention.

[0097] Figure 2 This is a flowchart of the method of the present invention. Detailed implementation manners

[0098] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0099] Referring to the attached Figure 1-2 description, an intelligent garbage recycling station monitoring method based on target tracking according to an embodiment of the present invention includes a shooting unit;

[0100] The shooting unit is installed in the target area and image data is collected; the target area includes fixed points and moving points; the fixed points include garbage stations; the moving points include garbage collection vehicles; among them, the fixed points are only used for monitoring, and the moving points are used for both monitoring and performing recycling;

[0101] Based on the collected image data, garbage and personnel are detected through YOLOv5, and continuous monitoring of multiple garbage targets is maintained by combining ByteTrack target tracking;

[0102] During continuous monitoring, if the garbage status is determined to be abnormal, an alarm is issued and the moving point is prompted to perform recycling;

[0103] If the garbage status of the fixed point is determined to be abnormal, a first-level scheduling signal is sent to the scheduling center;

[0104] If the moving point identifies that the garbage status of any fixed point is abnormal during the movement, a second-level scheduling signal is sent to the scheduling center;

[0105] The scheduling center calculates a scheduling execution recycling plan for the moving point based on the first-level scheduling signal and the second-level scheduling signal.

[0106] The monitoring types of the fixed points include garbage types and garbage quantities; the monitoring types of the moving points include garbage quantities;

[0107] During the detection of garbage and personnel through YOLOv5, it includes target image preprocessing; the input image is normalized, channel-converted, resized, and data-augmented to adapt to the input layer of YOLOv5; it is assumed that represents the normalized image; ; represent the minimum and maximum values of pixel values;

[0108] In adapting to the input size of YOLOv5, scale adjustment is performed based on bilinear interpolation, and it is planned that represents the adjusted image, that is, the image after scaling to meet the input requirements of the network, and through the scaling function , the input image is adjusted to the specified target size; ; , respectively represent the expected input width and height of YOLOv5;

[0109] In the detection of garbage and people based on YOLOv5, it includes feature extraction; the input image is processed by the backbone feature extraction network of YOLOv5 to extract features at different levels, and the features include edges, textures, and shapes;

[0110] ;

[0111] where is the input image; is the convolutional kernel; is the output feature map; where the backbone feature extraction network Backbone is responsible for extracting features and uses residual connections to enhance the information flow ability, enabling the deep network to learn more complex features;

[0112] In the detection of garbage and people based on YOLOv5, it includes a feature fusion part; the feature fusion part based on the FPN structure combined with the PAN structure is used for multi-scale feature fusion; the FPN structure constructs feature maps at different levels through a top-down feature pyramid to enhance the detection ability of small targets; while the PAN structure enhances the transmission of deep semantic information through a bottom-up path aggregation, making the high-level features have more spatial detail information, so as to simultaneously take into account the detection accuracy and robustness of targets of different sizes in the target detection task;

[0113] ;

[0114] where is the feature map of the current layer; represents the feature map of the bottom layer; is the feature map of the previous layer; represents the upsampling operation, which is used to increase the resolution of the feature map; is the convolution operation, and the convolution operation is used to process the feature map; in this way, the deep features carry more global information, while the shallow features maintain the target detail information, making the network more advantageous in detecting small targets of garbage types;

[0115] In the detection of garbage and personnel based on YOLOv5, it includes the object detection and bounding box prediction parts; in the object detection and bounding box prediction parts, the network is responsible for outputting the class label, bounding box coordinates, and confidence of the object;

[0116] ;

[0117] Among them is the center coordinate of the object; respectively represent the width and height of the object; represents the class confidence of the object; for all the bounding boxes of object detection, the total loss function :

[0118] ;

[0119] Among them is the object classification loss; is the bounding box loss; is the object confidence loss; is the loss weight;

[0120] By introducing non-maximum suppression (NMS) during the object detection process, the intersection over union (IoU) between the detection bounding boxes is calculated, redundant boxes with high overlap are filtered out, and only the best bounding box that best matches the object is retained, thereby improving the accuracy of the detection results; non-maximum suppression (NMS) is used to remove duplicate bounding boxes and retain the optimal object box;

[0121] ;

[0122] Among them respectively represent the th candidate detection box and the th candidate detection box, which are usually multiple bounding boxes predicted by the model on the same object. Since the object detection algorithm YOLOv5 may generate multiple boxes of different sizes or positions on the same object, it is necessary to calculate the IoU between them to determine the optimal box; is the intersection over union (IoU), whose value is between [0, 1]. If the value is close to 1, it means that the two boxes overlap highly and may be different prediction boxes of the same object; if the value is close to 0, it means that the two boxes have little intersection and may belong to different objects; represents the area of the overlapping region of the two bounding boxes and ; represents the union area of the two bounding boxes, that is, their total coverage area; its calculation method is as follows:

[0123] ;

[0124] Among them and are the areas of two bounding boxes respectively. Since the intersection area is double-counted once, it needs to be subtracted.

[0125] In ByteTrack object tracking, the goal of object tracking is to maintain the uniqueness and continuity of detected objects in the video stream, enabling each object to be correctly tracked over the time axis. Object tracking relies on Kalman filtering for object state prediction and combines the Hungarian algorithm for object matching. Finally, a high and low confidence object fusion strategy is adopted to ensure tracking stability;

[0126] Object tracking includes modeling and predicting the object state based on Kalman filtering; Kalman filtering is used to predict the position of the object in the next frame. Even if the object is briefly occluded, its possible motion trend can still be inferred based on the historical trajectory. First, represent the object state. In the multi-object tracking task, the state vector of each object is modeled as:

[0127] ;

[0128] where is the center coordinate of the object; is the scale of the object, which is based on the area; is the aspect ratio of the object; are the velocity components of the object respectively. The velocity components include position velocity and scale change velocity;

[0129] Secondly, update the object state; assume that the motion of the object is represented based on a linear model;

[0130] ;

[0131] ;

[0132] ;

[0133] ;

[0134] where is the state vector of the object at the th frame; is the state transition matrix, which represents how the object moves from the previous frame to the current frame; represents the time interval between two frames; is the control input, including the acceleration change of the object; is the process noise, which represents the random perturbation of the object; is the observation value, that is, the position of the detected object bounding box; is the observation matrix; the observation matrix represents the observed values, which include position and scale information but do not include velocity information, and the observed values are the detected targets;

[0135] Object tracking includes implementing object matching based on the Hungarian algorithm; the goal of object matching is to associate the objects in the current frame with the tracked objects in the previous item, so that the object ID remains unique in the time series. For the object matching calculation, the object matching is based on calculating the matching degree;

[0136] ;

[0137] where is the object in the current frame and the object in the previous frame; where respectively represent the th candidate detection box and the th candidate detection box;

[0138] In the object matching process, calculate the relationship matrix between the objects in the current frame and the objects in the previous frame, and use the Hungarian algorithm to solve the optimal object matching. The goal is to maximize the total matching score, which is expressed as:

[0139] ;

[0140] where is a binary decision variable. If matches , then , otherwise it is 0;

[0141] The constraint conditions include:

[0142] ;

[0143] The constraint conditions are used to ensure that each object can match at most one object in the previous frame;

[0144] Object tracking includes object fusion. The object fusion adopts a high-low confidence fusion strategy to ensure that more objects are effectively tracked;

[0145] In the high-confidence matching, first, ByteTrack object tracking uses high-confidence objects for matching, and the confidence :

[0146] 1. Calculate the IoU relationship matrix and perform Hungarian algorithm matching;

[0147] 2. The matched objects directly update their states and enter the next tracking cycle;

[0148] In low-confidence target supplementation, for unmatched low-confidence targets, the confidence :

[0149] 1. If the target has appeared in the previous frames, that is, there is a record of short-term loss, then tracking continues;

[0150] 2. If the target has not appeared for consecutive frames, then it is removed from the tracking list;

[0151] In target life cycle management, first, initialize, create a new trajectory for the newly detected target and assign a unique ID; second, perform updates, update the status of the successfully matched targets, which includes position and speed; finally, perform loss management. If the target is lost briefly, that is, in the case of occlusion, then use Kalman filtering to predict its position. If the target is lost for too long, then remove the target;

[0152] Regarding target monitoring and target tracking, it should be noted overall that:

[0153] Based on YOLOv5 object detection: Detect garbage, garbage sites, and people, and generate bounding box information;

[0154] Non-maximum suppression NMS: Screen the best bounding boxes and remove duplicate boxes;

[0155] Target state prediction based on Kalman filtering: Predict the position of the target in the next frame and handle the problem of short-term occlusion;

[0156] Target matching based on the Hungarian algorithm: Calculate to perform optimal target association;

[0157] The high and low confidence matching of ByteTrack target tracking includes high confidence matching and low confidence supplementation; High confidence matching: Prioritize matching targets with higher confidence; Low confidence supplementation: Try to recover low confidence targets to avoid target loss;

[0158] Target life cycle management: Maintain the ID and trajectory of the target to ensure long-term tracking;

[0159] After the execution of YOLOv5 object detection and ByteTrack target tracking, the category information (garbage, garbage sites, people, etc.), position information (bounding box coordinates), confidence score (detection credibility), unique ID (to ensure cross-frame tracking consistency), and the movement trajectory of the target (multi-frame continuous path) of each target are obtained; Based on this, abnormal states are identified, which include garbage site overflow, illegal placement, etc., and corresponding scheduling signals are generated for garbage scheduling or management alerts, thus forming a monitoring closed-loop between the garbage site and the garbage collection vehicle to achieve the effect of management or cleaning.

[0160] During the monitoring of the garbage site, a weighing sensor is installed at a fixed point, and the YOLOv5 combined with ByteTrack target tracking and the weighing sensor are used to provide the first feature variable and the second feature variable; the first feature variable is the garbage type , and the second feature variable is the garbage quantity, where the garbage quantity includes the total garbage volume of the current garbage site and the garbage site 's total weight ;

[0161] The detection result output based on YOLOv5 is expressed as:

[0162] ;

[0163] where is the garbage data set detected at the garbage site ; is the total number of candidate detection frames detected in the th garbage site; is the bounding box coordinates of the garbage in the th candidate detection frame; is the classification result of the garbage in the th candidate detection frame; is the detection confidence of the garbage in the th candidate detection frame;

[0164] Using the bounding box information of object detection and combining camera depth calculation, estimate the volume of the garbage:

[0165] ;

[0166] where is the depth information of the object, and the depth information can be measured by a camera or lidar; is the volume conversion coefficient of the garbage type. Since different garbage has different densities, such as plastics, metals, etc.; is the volume adjustment coefficient, and the volume adjustment coefficient is used to correct the detection error;

[0167] For the garbage site at the fixed point, obtain the actual weight based on the installed weighing sensor:

[0168] ;

[0169] where is the total weight measured by the weighing sensor at the garbage site at the th moment; ; is the th moment of the garbage site at the The total weight measured by the load cell; is the time step, that is, the time sequence length of the weighing record; is the density conversion coefficient of the garbage type, which is used to convert the detected garbage volume into weight. For example, the density of a plastic bottle ; while the density of wet garbage .

[0170] For the generation of the primary scheduling signal and the secondary scheduling signal, when it is detected that the filling of the garbage site exceeds the preset threshold, the corresponding scheduling signal is triggered;

[0171] In the generation of the primary scheduling signal based on fixed points, if the filling volume, volume or weight of the fixed-point garbage site exceeds the threshold, then:

[0172] ;

[0173] where is the set of primary scheduling signals; is the number of the garbage site; are the volume and weight overflow thresholds respectively; are the maximum capacity and load-bearing of the garbage site respectively; is the monitoring point indicator. If , it means that the monitoring point is a fixed point. If , it means that the monitoring point is a mobile point;

[0174] In the generation of the secondary scheduling signal based on mobile points, if it is detected that the garbage site overflows during the driving of the garbage collection vehicle, the secondary scheduling signal is triggered, which is expressed as:

[0175] or ;

[0176] where is the set of secondary scheduling signals; The primary scheduling signal is triggered based on fixed points and requires the garbage collection vehicle to be dispatched for cleaning. The secondary scheduling signal is triggered based on mobile points, and the garbage collection vehicle can handle it by itself or request additional support.

[0177] Based on the dispatching center, calculate the cleaning plan for mobile points, and calculate the optimal garbage collection path based on the primary scheduling signal and the secondary scheduling signal;

[0178] In the calculation of task priorities, the dispatching center calculates the cleaning priorities of garbage sites and formulates indicating the task priority score of garbage site ;

[0179] ;

[0180] ;

[0181] ;

[0182] ;

[0183] where is the impact factor of overflow severity. In practical applications, a relatively large value can be selected for the value of the impact factor of overflow severity to ensure that the garbage sites with garbage overflow are preferentially processed; is the impact factor of the geographical distance from the garbage site to the garbage truck. The garbage sites with a relatively short distance should be preferentially cleared; is the impact factor of garbage category. The impact factor of garbage category is used to preferentially clear certain specific types of garbage. For example, when it is identified as kitchen waste, it can be selected for preferential recycling; is the impact factor of the weight of the garbage site. The impact factor of the weight of the garbage site avoids the problem of the garbage truck being overloaded due to the excessive weight of the garbage site; is the garbage site and the garbage truck the geographical distance between; is the garbage site geographical location coordinates; is the garbage site binary indicator function of whether it is abnormal; is the garbage site GPS coordinates; is the garbage truck GPS coordinates; is the urgency score of the garbage of all candidate detection frames in the garbage site ; is the th garbage category number in the candidate detection frame; is the th emergency treatment weight of the garbage in the candidate detection frame;

[0184] In vehicle task allocation, the clearing task of the garbage truck is expressed as:

[0185] ;

[0186] ;

[0187] where is the optimal path of the garbage truck ; is the overloading penalty term of the garbage truck; represents finding a path that can minimize the clearing cost ; Denotes the total driving distance of the garbage truck visiting garbage sites in sequence, and respectively denote the location coordinates of the -th and -th garbage sites, which are used to calculate the driving distance of the garbage truck from garbage site to garbage site ; is the total number of garbage sites that the garbage truck needs to visit; is the overload penalty factor, which is used to control the impact of overload on route calculation. When takes a larger value, garbage trucks that are not overloaded are preferentially selected for cleaning to avoid overloading the vehicles; is the limit of the carrying capacity of the garbage truck;

[0188] During the garbage collection operation of the garbage truck, when the garbage truck collects the garbage at garbage site , the updated remaining garbage volume and remaining garbage weight of the garbage site are expressed as:

[0189] ;

[0190] ;

[0191] where is the garbage volume that the garbage truck can currently collect; is the garbage weight that the garbage truck can currently collect; is the garbage volume of garbage site at time ; is the remaining garbage volume of garbage site at time ; is the garbage weight of garbage site at time ; is the remaining garbage weight of garbage site at time ; Indicates that if the or collected by the garbage truck is greater than the current remaining amount at the garbage site, negative values are prevented from occurring to ensure that the remaining garbage amount at the garbage site does not fall below 0;

[0192] When the garbage truck reaches the limit of its carrying capacity, it returns to the garbage station for unloading. This condition is judged as: ; where is the current total volume of the loaded garbage; is a logical condition, indicating that when the total amount of garbage loaded by the garbage truck equals the maximum carrying capacity, it must return to the garbage station to unload the goods;

[0193] In the transportation status update, after the garbage truck returns to the main station to unload the goods, the load status of the garbage truck will be reset;

[0194] ;

[0195] where is the load volume of the garbage truck at time ; is the load weight of the garbage truck at time ; after the garbage truck finishes unloading, the load volume and load weight are cleared to restart the next round of garbage collection tasks.

[0196] An intelligent garbage recycling station monitoring system based on target tracking, including a data acquisition module, a target detection module, an anomaly monitoring module, a primary scheduling module, a secondary scheduling module, and a scheduling execution module;

[0197] The data acquisition module is used to install a shooting unit in the target area and collect image data; the target area includes fixed points and moving points; the fixed points include garbage stations; the moving points include garbage trucks;

[0198] The target detection module is based on the collected image data, detects garbage and personnel through YOLOv5, and combines ByteTrack target tracking to continuously monitor multiple garbage targets;

[0199] In the continuous monitoring, if the garbage status is determined to be abnormal, the anomaly monitoring module issues an alarm and prompts the moving point to perform recycling;

[0200] The primary scheduling module includes: if the garbage status of the fixed point is determined to be abnormal, it issues a primary scheduling signal to the scheduling center;

[0201] The secondary scheduling module includes: if the moving point identifies that the garbage status of any fixed point is abnormal during the movement, it issues a secondary scheduling signal to the scheduling center;

[0202] The scheduling execution module calculates the scheduling execution recycling plan for the moving point through the scheduling center based on the primary scheduling signal and the secondary scheduling signal.

[0203] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for monitoring an intelligent garbage collection station based on target tracking, comprising a shooting unit, characterized in that: The shooting unit is installed in the target area and image data is collected; the target area includes fixed points and moving points; the fixed points include garbage sites; Mobile points include garbage collection trucks; Based on the collected image data, garbage and people are detected through YOLOv5, and ByteTrack target tracking is combined to maintain continuous monitoring of multiple garbage targets; In continuous monitoring, if the garbage status is judged to be abnormal, an alarm will be issued and the mobile point will be prompted to perform recycling; If the garbage status at a fixed point is judged to be abnormal, a first-level dispatch signal is sent to the dispatch center; If the mobile point identifies that the garbage status of any fixed point is abnormal during the movement, it will send a secondary dispatch signal to the dispatch center; The dispatch center calculates the dispatch execution recovery plan of the mobile point based on the primary dispatch signal and the secondary dispatch signal; For the generation of primary and secondary dispatch signals, when it is detected that the garbage site filling exceeds the preset threshold, the corresponding dispatch signal is triggered; In the generation of the first-level dispatch signal based on fixed points, if the filling amount of the fixed-point garbage station exceeds the threshold, then: ； in It is a set of first-level scheduling signals; The number of the spam site; are the volume and weight overflow thresholds, respectively; They are the capacity and maximum load-bearing value of the garbage station respectively; is the monitoring point indicator, if , it means that the monitoring point is a fixed point. , it means that the monitoring point is a moving point; Indicates spam site The volume of garbage; Indicates spam site The weight of the garbage; In the generation of secondary dispatch signals based on mobile points, if the garbage collection truck detects that the garbage station is overflowing during driving, the secondary dispatch signal is triggered, which is expressed as: or ; in It is a set of secondary dispatch signals. The primary dispatch signal is triggered based on fixed points, and garbage collection trucks need to be dispatched to remove garbage. The secondary dispatch signal is triggered based on mobile points, and garbage collection trucks handle the garbage collection by themselves or request additional support. The dispatch center calculates the removal plan for the mobile point and the garbage removal path based on the primary and secondary dispatch signals; In the task priority calculation, the dispatch center calculates the removal priority of the garbage station and formulates Indicates spam site Task priority rating; ； ； ； ； in is the influencing factor of spill severity; is the geographical distance factor from the garbage station to the garbage collection truck; is the impact factor of the garbage category; is the weight impact factor of the spam site; For spam sites With garbage truck The geographical distance between For spam sites The geographic location coordinates of For spam sites A binary indicator function of whether it is abnormal; For spam sites GPS coordinates; For garbage collection trucks GPS coordinates; For spam sites The urgency score of the garbage in all candidate detection boxes; For the The category number of the garbage in the candidate detection frame; For the The emergency processing weight of garbage in the candidate detection box.

2. According to claim 1, a method for monitoring an intelligent garbage recycling station based on target tracking is characterized in that: The monitoring types at fixed points include garbage type and garbage amount; the monitoring types at mobile points include garbage amount; In the detection of garbage and people based on YOLOv5, the target image preprocessing is included; input image After normalization, channel conversion, size adjustment and data enhancement, it is adapted to the input layer of YOLOv5; represents the normalized image; ; Indicates the minimum and maximum values ​​of pixel values; In order to adapt the input size of YOLOv5, we use bilinear interpolation to adjust the scale and propose Represents the adjusted image, that is, the image that is scaled to fit the network input requirements and is scaled by the scaling function , resize the input image to the specified target size; ; , They represent the input width and height expected by YOLOv5 respectively; In the detection of garbage and people based on YOLOv5, feature extraction is included; the input image is processed by the backbone feature extraction network of YOLOv5 to extract features at different levels; ； in is the input image; is the convolution kernel; The backbone feature extraction network Backbone is responsible for extracting features and uses residual connections to improve information flow, so that the deep network can learn more complex features. In the detection of garbage and people based on YOLOv5, the feature fusion part is included; The feature fusion part based on FPN structure combined with PAN structure is used for multi-scale feature fusion; ； in is the feature map of the current layer; Represents the underlying feature map; is the feature map of the previous layer; represents an upsampling operation; is the convolution operation; In the detection of garbage and people based on YOLOv5, there are target detection and bounding box prediction parts. In the target detection and bounding box prediction parts, the network is responsible for outputting the target's category label, bounding box coordinates, and confidence. ； in is the center coordinate of the target; Respectively represent the width and height of the target; Represents the category confidence of the target; for all target detection bounding boxes, calculate the total loss function : ； in is the target classification loss; is the bounding box loss; is the target confidence loss; is the loss weight; By introducing non-maximum suppression (NMS) in the target detection process, the intersection-over-union ratio between detection bounding boxes is calculated; non-maximum suppression (NMS) is used to remove duplicate bounding boxes and retain the target box; ； in Respectively represent candidate detection boxes and the Candidate detection boxes; is the intersection and comparison; Represents two candidate detection boxes and The area of ​​the overlapping region; Represents the union area of ​​two candidate detection boxes.

3. The method for monitoring an intelligent garbage collection station based on target tracking according to claim 2 is characterized in that: In ByteTrack target tracking, the goal of target tracking is to maintain the uniqueness and continuity of the detected targets in the video stream so that each target can be correctly tracked on the time axis. Target tracking relies on Kalman filtering for target state prediction and combines it with the Hungarian algorithm for target matching. Finally, a confidence target fusion strategy is adopted to ensure tracking stability. Target tracking includes modeling and predicting the target state based on Kalman filtering; Kalman filtering is used to predict the position of the target in the next frame; first, the target state is represented. In the multi-target tracking task, the state vector of each target is modeled as: ； in is the center coordinate of the target; The scale of the goal; is the target aspect ratio; are the velocity components of the target respectively; Secondly, the target state is updated; assuming that the target's motion is represented based on a linear model; ； ； ； ； in For the goal The state vector of the frame; is the state transfer matrix; Indicates the time interval between two frames; is the control input; is the process noise; is the observed value, i.e. the position of the target bounding box detected; is the observation matrix; the observation matrix represents the observation value, which includes the position and scale information; Target tracking includes target matching based on the Hungarian algorithm; the goal of target matching is to associate the target of the current frame with the tracking target of the previous item, so that the target ID remains unique in the time series, and the target matching calculation is based on Calculate the matching degree; ； in For the current frame and previous frame The matching degree between Respectively represent candidate detection boxes and the Candidate detection boxes; In the target matching process, the current frame target and the previous frame target are calculated. Relationship matrix, and the Hungarian algorithm is used to solve the target matching, which is expressed as: ； in is a binary decision variable, if and Match, then , otherwise 0; The constraints include: ； Constraints are used to make each target match a target in the previous frame; Target tracking includes target fusion, which adopts the confidence fusion strategy.

4. The method for monitoring an intelligent garbage collection station based on target tracking according to claim 3 is characterized in that: During the garbage site monitoring process, a weighing sensor is installed at a fixed point, and the first feature variable and the second feature variable are provided by YOLOv5 combined with ByteTrack target tracking and weighing sensor; The first characteristic variable is the type of garbage The second characteristic variable is the amount of garbage, which includes the total volume of garbage at the current garbage site. and spam sites Total weight ; The detection results based on YOLOv5 output are expressed as: ； in For spam sites Detected garbage datasets; For the The total number of candidate detection boxes detected in the spam sites; For the The bounding box coordinates of the garbage of the candidate detection boxes; For the The classification results of the garbage of the candidate detection boxes; For the The detection confidence of the garbage in the candidate detection box; Using the bounding box information of the target detection and the camera depth calculation, the volume of the garbage is estimated: ； in Depth information of the target; is the volume conversion factor for the type of garbage; is the volume adjustment factor; For fixed-point garbage stations, the actual weight is obtained based on the load cells installed: ； in For the Time spam site The total weight measured by the load cell; is the time step; Density conversion factor for garbage type.

5. The method for monitoring an intelligent garbage collection station based on target tracking according to claim 4 is characterized in that: In vehicle task allocation, garbage collection vehicles The cleaning task is expressed as: ； ； in For garbage collection trucks Path; Penalties for overloading garbage collection trucks; Represents finding a path that can clear the cost ; It represents the total driving distance of the garbage collection truck visiting the garbage sites in sequence; is the overload penalty factor; The maximum carrying capacity of garbage collection trucks; In the garbage collection truck, the garbage collection truck Garbage removal site , the remaining garbage volume and remaining garbage weight of the garbage station are updated as follows: ； ； in For garbage collection trucks The volume of waste currently removed; For garbage collection trucks The weight of trash currently removed; For spam sites In time The volume of garbage at the moment; For spam sites In time The weight of the garbage at the moment; When the garbage collection truck reaches the carrying capacity limit, it returns to the garbage station for unloading. This condition is judged as: ;in For the current The total volume of garbage loaded; In the transportation status update, when the garbage truck returns to the main station for unloading, the load status of the garbage truck will be reset; ； in For garbage collection trucks In time Load volume at the time; For garbage collection trucks In time The load weight at the moment; when the garbage collection truck completes unloading, the load volume and load weight are reset to zero to resume the next round of garbage collection task.

6. A target tracking-based intelligent garbage recycling station monitoring system, comprising applying the target tracking-based intelligent garbage recycling station monitoring method according to claim 5, characterized in that: It also includes a data acquisition module, a target detection module, an anomaly monitoring module, a primary scheduling module, a secondary scheduling module, and a scheduling execution module; The data acquisition module is used to install the shooting unit in the target area and collect image data; the target area includes fixed points and mobile points; the fixed points include garbage sites; Mobile points include garbage collection trucks; The target detection module detects garbage and people based on the collected image data through YOLOv5, and combines ByteTrack target tracking to maintain continuous monitoring of multiple garbage targets; The abnormal monitoring module will issue an alarm if the garbage status is judged to be abnormal during continuous monitoring, and prompt the mobile point to perform recycling; The first-level dispatch module includes: if the garbage status of a fixed point is determined to be abnormal, a first-level dispatch signal is sent to the dispatch center; The secondary dispatch module includes: if the mobile point identifies that the garbage status of any fixed point is abnormal during the movement, it sends a secondary dispatch signal to the dispatch center; The dispatch execution module calculates the dispatch execution recovery plan of the mobile point based on the primary dispatch signal and the secondary dispatch signal through the dispatch center.

Images