Intelligent garbage recycle bin monitoring method and system based on target tracking

Abstract

The invention discloses an intelligent garbage collection station monitoring method and system based on target tracking, and particularly relates to the field of garbage collection monitoring, and the method comprises the steps: installing a shooting unit in a target area, and collecting image data; the target area comprises a fixed point and a moving point; the fixed points comprise garbage stations; the moving point comprises a garbage recycling vehicle; based on the collected image data, garbage and personnel are detected through YOLOv5, and continuous monitoring of a plurality of garbage targets is kept in combination with ByteTrack target tracking; in the continuous monitoring process, if the garbage state is judged to be abnormal, an alarm is given, and a moving point is prompted to execute recycling. A hierarchical scheduling mechanism is formed through first-level scheduling and second-level scheduling, scheduling is triggered when the filling state of the garbage can exceeds a threshold value, and the situation that a global path is frequently recalculated due to slight weight change is avoided, so that computing resource consumption is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of garbage collection monitoring, and more specifically, to a method and system for monitoring an intelligent garbage collection station based on target tracking. Background Art

[0002] The existing technology usually relies on manual monitoring and scheduling in the process of garbage station detection and scheduling, which not only increases labor costs, but is also easily affected by human errors, response delays and scheduling imbalances, resulting in low garbage removal efficiency and not conducive to garbage management needs; In the prior art, the publication number is CN114372746A, which discloses a classified garbage intelligent transportation management system and method, which relies on the dynamic monitoring of the weight of the garbage bin to optimize the transportation scheduling, and the reference variables are relatively limited, which is not conducive to practical use; In addition, every time the weight changes, the solution with publication number CN114372746A will trigger route recalculation and vehicle rescheduling. Since the weight of the trash can may change frequently, the system needs to frequently perform global path optimization and vehicle deployment. This strategy may be feasible in small-scale pilots, but in large-scale city-level deployments, it will consume excessive computing resources. Summary of the invention

[0003] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an intelligent garbage recycling station monitoring method and system based on target tracking, which forms a hierarchical scheduling mechanism through primary scheduling + secondary scheduling, and triggers scheduling when the garbage bin filling status exceeds the threshold, without frequently recalculating the global path due to slight weight changes, so as to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above-mentioned object, the present invention provides the following technical solutions: a method for monitoring an intelligent garbage recycling station based on target tracking, comprising a shooting unit; The shooting unit is installed in the target area and image data is collected; the target area includes fixed points and mobile points; the fixed points include garbage stations; the mobile points include garbage collection trucks; Based on the collected image data, garbage and people are detected through YOLOv5, and ByteTrack target tracking is combined to maintain continuous monitoring of multiple garbage targets; In continuous monitoring, if the garbage status is judged to be abnormal, an alarm will be issued and the mobile point will be prompted to perform recycling; If the garbage status at a fixed point is judged to be abnormal, a first-level dispatch signal is sent to the dispatch center; If the mobile point identifies that the garbage status of any fixed point is abnormal during the movement, it will send a secondary dispatch signal to the dispatch center; The dispatch center calculates the dispatch execution recovery plan of the mobile point based on the primary dispatch signal and the secondary dispatch signal.

[0005] In a preferred embodiment, the monitoring types at fixed points include garbage type and garbage amount; the monitoring types at mobile points include garbage amount; In the detection of garbage and people based on YOLOv5, the target image preprocessing is included; input image After normalization, channel conversion, size adjustment and data enhancement, it is adapted to the input layer of YOLOv5; represents the normalized image; ; Indicates the minimum and maximum values ​​of pixel values; In order to adapt the input size of YOLOv5, we use bilinear interpolation to adjust the scale and propose Represents the adjusted image, that is, the image that is scaled to fit the network input requirements and is scaled by the scaling function , resize the input image to the specified target size; ; , They represent the input width and height expected by YOLOv5 respectively; In the detection of garbage and people based on YOLOv5, feature extraction is included; the input image is processed by the backbone feature extraction network of YOLOv5 to extract features at different levels; ; in is the input image; is the convolution kernel; The backbone feature extraction network Backbone is responsible for extracting features and uses residual connections to improve information flow, so that the deep network can learn more complex features. In the detection of garbage and people based on YOLOv5, a feature fusion part is included; the feature fusion part based on the FPN structure combined with the PAN structure is used for multi-scale feature fusion; ; in is the feature map of the current layer; Represents the underlying feature map; is the feature map of the previous layer; represents an upsampling operation; is the convolution operation; In the detection of garbage and people based on YOLOv5, there are target detection and bounding box prediction parts. In the target detection and bounding box prediction parts, the network is responsible for outputting the target's category label, bounding box coordinates, and confidence. ; in is the center coordinate of the target; Respectively represent the width and height of the target; Represents the category confidence of the target; for all target detection bounding boxes, calculate the total loss function : ; in is the target classification loss; is the bounding box loss; is the target confidence loss; is the loss weight; By introducing non-maximum suppression (NMS) in the target detection process, the intersection-over-union ratio between detection bounding boxes is calculated; non-maximum suppression (NMS) is used to remove duplicate bounding boxes and retain the target box; ; in Respectively represent candidate detection boxes and the Candidate detection boxes; is the intersection and comparison; Represents two bounding boxes and The area of ​​the overlapping region; Represents the area of ​​the union of two bounding boxes.

[0006] In a preferred embodiment, in ByteTrack target tracking, the goal of target tracking is to maintain the uniqueness and continuity of the detected targets in the video stream so that each target is correctly tracked on the time axis. Target tracking relies on Kalman filtering for target state prediction and combines the Hungarian algorithm for target matching. Finally, a confidence target fusion strategy is adopted to ensure tracking stability. Target tracking includes modeling and predicting the target state based on Kalman filtering; Kalman filtering is used to predict the position of the target in the next frame; first, the target state is represented. In the multi-target tracking task, the state vector of each target is modeled as: ; in is the center coordinate of the target; The scale of the goal; is the target aspect ratio; are the velocity components of the target respectively; Secondly, the target state is updated; assuming that the target's motion is represented based on a linear model; ; ; ; ; in For the goal The state vector of the frame; is the state transfer matrix; Indicates the time interval between two frames; is the control input; is the process noise; is the observed value, i.e. the position of the target bounding box detected; is the observation matrix; the observation matrix represents the observation value, which includes the position and scale information; Target tracking includes target matching based on the Hungarian algorithm; the goal of target matching is to associate the target of the current frame with the tracking target of the previous item, so that the target ID remains unique in the time series, and the target matching calculation is based on Calculate the matching degree; ; in The target in the current frame and the previous frame target The degree of match between In the target matching process, the current frame target and the previous frame target are calculated. Relationship matrix, and the Hungarian algorithm is used to solve the target matching, which is expressed as: ; in is a binary decision variable. If the target With the goal match , then, otherwise 0; The constraints include: ; Constraints are used to make each target match a target in the previous frame; Target tracking includes target fusion, which adopts the confidence fusion strategy.

[0007] In a preferred embodiment, during the monitoring of garbage sites, a weighing sensor is installed at a fixed point, and YOLOv5 is combined with ByteTrack target tracking and weighing sensors to provide a first feature variable and a second feature variable; the first feature variable is the type of garbage The second characteristic variable is the amount of garbage, which includes the total volume of garbage at the current garbage site. and spam sites Total weight ; The detection results based on YOLOv5 output are expressed as: ; in For fixed point Detected garbage datasets; is the amount of garbage detected in the current frame; For the The bounding box coordinates of the garbage; For the The classification results of garbage; For the The confidence level of garbage detection; Using the bounding box information of the target detection and the camera depth calculation, the volume of the garbage is estimated: ; in Depth information of the target; is the volume conversion factor for the type of garbage; is the volume adjustment factor; For fixed-point garbage stations, the actual weight is obtained based on the load cells installed: ; in For the Time spam site The total weight measured by the load cell; is the time step; is the density conversion factor for the garbage type.

[0008] In a preferred embodiment, the primary dispatch signal and the secondary dispatch signal are generated, and when it is detected that the garbage site filling exceeds a preset threshold, the corresponding dispatch signal is triggered; In the generation of the first-level dispatch signal based on fixed points, if the filling amount of the fixed-point garbage station exceeds the threshold, then: ; in It is a set of first-level scheduling signals; Number the spam sites; are the volume and weight overflow thresholds, respectively; They are the capacity and maximum load-bearing value of the garbage station respectively; is the monitoring type indicator, if , it means that the monitoring point is a fixed point. , it means that the monitoring point is a moving point; In the generation of secondary dispatch signals based on mobile points, if the garbage collection truck detects that the garbage station is overflowing during driving, the secondary dispatch signal is triggered, which is expressed as: ; in It is a set of secondary dispatch signals. The primary dispatch signal is triggered based on fixed points, and garbage collection trucks need to be dispatched to clear the garbage. The secondary dispatch signal is triggered based on mobile points, and the garbage collection trucks handle the garbage on their own or request additional support.

[0009] In a preferred embodiment, the dispatch center calculates the removal plan of the mobile point and calculates the garbage removal path based on the primary dispatch signal and the secondary dispatch signal; In the task priority calculation, the dispatch center calculates the removal priority of the garbage station and formulates Indicates spam site The task priority rating; ; ; ; ; in is the influencing factor of spill severity; is the geographical distance factor from the garbage station to the garbage collection truck; is the impact factor of the garbage category; is the weight impact factor of the spam site; For spam sites With garbage truck The geographical distance between For spam sites The geographic location coordinates of For spam sites A binary indicator function of whether it is abnormal; For spam sites GPS coordinates; For garbage collection trucks GPS coordinates; For spam sites Urgency rating of internal waste types; For the The category number of the garbage; The emergency treatment weight of this category of waste; In vehicle task allocation, garbage collection vehicles The cleaning task is expressed as: ; ; in For garbage collection trucks Path; Penalties for overloading garbage collection trucks; Represents finding a path that can clear the cost ; It represents the total driving distance of the garbage collection truck visiting the garbage sites in sequence; is the overload penalty factor; The maximum carrying capacity of garbage collection trucks; In the garbage collection truck, the garbage collection truck Garbage removal site , the remaining garbage volume and remaining garbage weight of the garbage station are updated as follows: ; ; in For garbage collection trucks The volume of waste currently removed; For garbage collection trucks The weight of trash currently removed; For trash can In time The volume of garbage at the moment; For trash can In time The weight of the garbage at the moment; When the garbage collection truck reaches the carrying capacity limit, it returns to the garbage station for unloading. This condition is judged as: ;in For the current The total volume of garbage loaded; In the transportation status update, when the garbage truck returns to the main station for unloading, the load status of the garbage truck will be reset; ; in For garbage collection trucks In time Load volume at the time; For garbage collection trucks In time The load weight at the moment; when the garbage collection truck completes unloading, the load volume and load weight are reset to zero to resume the next round of garbage collection task.

[0010] In a preferred embodiment, a target tracking-based intelligent garbage collection station monitoring system includes a data acquisition module, a target detection module, an abnormality monitoring module, a primary scheduling module, a secondary scheduling module, and a scheduling execution module; The data acquisition module is used to install the shooting unit in the target area and collect image data; the target area includes fixed points and mobile points; the fixed points include garbage sites; the mobile points include garbage collection vehicles; The target detection module detects garbage and people based on the collected image data through YOLOv5, and combines ByteTrack target tracking to maintain continuous monitoring of multiple garbage targets; The abnormal monitoring module will issue an alarm if the garbage status is judged to be abnormal during continuous monitoring, and prompt the mobile point to perform recycling; The first-level dispatch module includes: if the garbage status of a fixed point is determined to be abnormal, a first-level dispatch signal is sent to the dispatch center; The secondary dispatch module includes: if the mobile point identifies that the garbage status of any fixed point is abnormal during the movement, it sends a secondary dispatch signal to the dispatch center; The dispatch execution module calculates the dispatch execution recovery plan of the mobile point based on the primary dispatch signal and the secondary dispatch signal through the dispatch center.

[0011] Technical effects and advantages of the present invention: A hierarchical scheduling mechanism is formed through first-level scheduling + second-level scheduling. Scheduling is triggered when the filling status of the trash can exceeds the threshold, without frequently recalculating the global path due to slight weight changes, thereby reducing computing resource consumption; Combined with dual-layer monitoring of fixed points and mobile points, fixed points trigger dispatch only when garbage overflows, while mobile points dynamically monitor the status of garbage bins during driving and independently decide whether to directly transport or report to the dispatch center based on real-time load, avoiding frequent changes in the global path; By using YOLOv5 to detect garbage type and combining it with a weighing sensor to detect garbage quantity, multimodal fusion monitoring of garbage category, volume and weight can be achieved. Garbage recycling vehicles can not only evaluate the status of garbage bins based on visual inspection, but also make decisions based on actual weighing data, thus avoiding misjudgments that may result from relying solely on image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the system module of the present invention.

[0013] Figure 2 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] Refer to the instruction manual Figure 1-2 , a method for monitoring an intelligent garbage recycling station based on target tracking according to an embodiment of the present invention includes a shooting unit; The shooting unit is installed in the target area and image data is collected; the target area includes fixed points and mobile points; the fixed points include garbage stations; the mobile points include garbage collection vehicles; the fixed points are only used for monitoring, and the mobile points are used for both monitoring and collection; Based on the collected image data, garbage and people are detected through YOLOv5, and ByteTrack target tracking is combined to maintain continuous monitoring of multiple garbage targets; In continuous monitoring, if the garbage status is judged to be abnormal, an alarm will be issued and the mobile point will be prompted to perform recycling; If the garbage status at a fixed point is judged to be abnormal, a first-level dispatch signal is sent to the dispatch center; If the mobile point identifies that the garbage status of any fixed point is abnormal during the movement, it will send a secondary dispatch signal to the dispatch center; The dispatch center calculates the dispatch execution recovery plan of the mobile point based on the primary dispatch signal and the secondary dispatch signal.

[0016] The monitoring types at fixed points include garbage type and garbage amount; the monitoring types at mobile points include garbage amount; In the detection of garbage and people based on YOLOv5, the target image preprocessing is included; input image After normalization, channel conversion, size adjustment and data enhancement, it is adapted to the input layer of YOLOv5; represents the normalized image; ; Indicates the minimum and maximum values ​​of pixel values; In order to adapt the input size of YOLOv5, we use bilinear interpolation to adjust the scale and propose Represents the adjusted image, that is, the image that is scaled to fit the network input requirements and is scaled by the scaling function , resize the input image to the specified target size; ; , They represent the input width and height expected by YOLOv5 respectively; In the detection of garbage and people based on YOLOv5, feature extraction is included; the input image is processed by the YOLOv5 backbone feature extraction network to extract features at different levels, including edges, textures, and shapes; ; in is the input image; is the convolution kernel; The backbone feature extraction network Backbone is responsible for extracting features and uses residual connections to improve information flow, so that the deep network can learn more complex features. In the detection of garbage and people based on YOLOv5, the feature fusion part is included; the feature fusion part based on the FPN structure combined with the PAN structure is used for multi-scale feature fusion; the FPN structure constructs feature maps of different levels through a top-down feature pyramid to enhance the detection ability of small targets; and the PAN structure enhances the transmission of deep semantic information through bottom-up path aggregation, making high-level features more spatially detailed, thereby taking into account the detection accuracy and robustness of targets of different sizes in the target detection task; ; in is the feature map of the current layer; Represents the underlying feature map; is the feature map of the previous layer; represents the upsampling operation, which is used to increase the feature map resolution; The convolution operation is used to process the feature map. In this way, the deep features carry more global information, while the shallow features retain the target details, making the network more advantageous in detecting small targets of garbage type. In the detection of garbage and people based on YOLOv5, there are target detection and bounding box prediction parts. In the target detection and bounding box prediction parts, the network is responsible for outputting the target's category label, bounding box coordinates, and confidence. ; in is the center coordinate of the target; Respectively represent the width and height of the target; Represents the category confidence of the target; for all target detection bounding boxes, calculate the total loss function : ; in is the target classification loss; is the bounding box loss; is the target confidence loss; is the loss weight; By introducing non-maximum suppression (NMS) in the target detection process, the intersection-over-union ratio between detection bounding boxes is calculated, redundant boxes with high overlap are screened out, and only the best bounding box that best matches the target is retained, thereby improving the accuracy of the detection results; non-maximum suppression (NMS) is used to remove duplicate bounding boxes and retain the optimal target box; ; in Respectively represent candidate detection boxes and the Candidate detection boxes, which are usually multiple bounding boxes predicted by the model on the same target. Since the target detection algorithm YOLOv5 may generate multiple boxes of different sizes or positions on the same target, it is necessary to calculate the IoU between them to determine the optimal box; is the intersection-union ratio, and its value is between [0, 1]. If its value is close to 1, it means that the two boxes are highly overlapped and may be different predicted boxes of the same target; if its value is close to 0, it means that the two boxes have almost no intersection and may belong to different targets; Represents two bounding boxes and The area of ​​the overlapping region; represents the union area of ​​two bounding boxes, that is, their total coverage area; it is calculated as follows: ; in and are the areas of the two bounding boxes respectively. Since the intersection area is It is counted twice, so it needs to be subtracted.

[0017] In ByteTrack target tracking, the goal of target tracking is to maintain the uniqueness and continuity of the detected targets in the video stream so that each target can be correctly tracked on the time axis. Target tracking relies on Kalman filtering for target state prediction and combines it with the Hungarian algorithm for target matching. Finally, a high- and low-confidence target fusion strategy is adopted to ensure tracking stability. Target tracking includes modeling and predicting the target state based on Kalman filtering; Kalman filtering is used to predict the position of the target in the next frame. Even if the target is temporarily blocked, its possible movement trend can still be inferred based on the historical trajectory; first, the target state is represented. In the multi-target tracking task, the state vector of each target is modeled as: ; in is the center coordinate of the target; is the scale of the target, which is based on area; is the target aspect ratio; They are the velocity components of the target, which include position velocity and scale change velocity; Secondly, the target state is updated; assuming that the target's motion is represented based on a linear model; ; ; ; ; in For the goal The state vector of the frame; is the state transfer matrix, which represents how the target moves from the previous frame to the current frame; Indicates the time interval between two frames; For control input, it includes the acceleration change of the target; is process noise, which represents the random disturbance of the target; is the observed value, i.e. the position of the target bounding box detected; is the observation matrix; the observation matrix represents the observation value, which includes position and scale information but does not contain speed information. The observation value is the detected target; Target tracking includes target matching based on the Hungarian algorithm; the goal of target matching is to associate the target of the current frame with the tracking target of the previous item, so that the target ID remains unique in the time series, and the target matching calculation is based on Calculate the matching degree; ; in The target in the current frame and the previous frame target The degree of match between In the target matching process, the current frame target and the previous frame target are calculated. Relationship matrix, and use the Hungarian algorithm to solve the optimal target matching, the goal is to maximize the total matching score, which is expressed as: ; in is a binary decision variable. If the target With the goal Match, then , otherwise 0; The constraints include: ; The constraint is used to ensure that each target can match at most one target in the previous frame; Target tracking includes target fusion, which uses a high- and low-confidence fusion strategy to ensure that more targets are effectively tracked; In high confidence matching, ByteTrack target tracking first uses high confidence targets for matching. : 1. Calculate the IoU relationship matrix and perform Hungarian algorithm matching; 2. The matched target directly updates its status and enters the next tracking cycle; In low confidence target supplementation, for unmatched low confidence targets, the confidence : 1. If the target is in front If it has appeared in the frame, that is, there is a short-term loss record, the tracking will continue; 2. If the target is continuous If the frame does not appear, it is removed from the tracking list; In the target lifecycle management, initialization is first performed to create a new track for the newly detected target and assign a unique ID. Next, update is performed to update the status of the successfully matched target, including position and speed. Finally, loss management is performed. If the target is temporarily lost, that is, occlusion, Kalman filtering is used to predict its position. If the target is lost for too long, the target is removed. In target monitoring and target tracking, the following should be explained as a whole: YOLOv5-based target detection: detect garbage, garbage sites, people, and generate bounding box information; Non-maximum suppression (NMS): Filter the best bounding box and remove duplicate boxes; Target state prediction based on Kalman filtering: predict the position of the target in the next frame and handle short-term occlusion problems; Target matching based on the Hungarian algorithm: Calculation Perform optimal target association; ByteTrack target tracking high and low confidence matching includes high confidence matching and low confidence supplementation; high confidence matching: prioritize matching targets with higher confidence; low confidence supplementation: try to recover low confidence targets to avoid target loss; Target lifecycle management: Maintain the target’s ID and track to ensure long-term tracking; After the target detection YOLOv5 detection and ByteTrack target tracking are completed, the category information (garbage, garbage site, personnel, etc.), location information (bounding box coordinates), confidence score (detection credibility), unique ID (to ensure cross-frame tracking consistency) and target motion trajectory (multi-frame continuous path) of each target are obtained; based on this, abnormal conditions are identified, including garbage site overflow, illegal placement, etc., and corresponding dispatch signals are generated for garbage dispatch or management alarms, thereby forming a monitoring closed loop between garbage sites and garbage collection vehicles to achieve management or transportation effects.

[0018] During the garbage site monitoring process, a weighing sensor is installed at a fixed point, and YOLOv5 combined with ByteTrack target tracking and weighing sensors are used to provide the first feature variable and the second feature variable; the first feature variable is the type of garbage. The second characteristic variable is the amount of garbage, which includes the total volume of garbage at the current garbage site. and spam sites Total weight ; The detection results based on YOLOv5 output are expressed as: ; in For fixed point Detected garbage datasets; is the amount of garbage detected in the current frame; For the The bounding box coordinates of the garbage; For the The classification results of garbage; For the The confidence level of garbage detection; Using the bounding box information of the target detection and the camera depth calculation, the volume of the garbage is estimated: ; in The depth information of the target can be measured by a camera or a lidar; It is the volume conversion coefficient of garbage type, because different garbage has different densities, such as plastic, metal, etc.; is the volume adjustment factor, which is used to correct the detection error; For fixed-point garbage stations, the actual weight is obtained based on the load cells installed: ; in For the Time spam site The total weight measured by the load cell; For the Time spam site The total weight measured by the load cell; is the time step, i.e. the time series length of the weighing record; Density conversion factor for the type of garbage, which is used to convert the detected garbage volume into weight, such as the density of plastic bottles ; The density of wet garbage .

[0019] For the generation of primary and secondary dispatch signals, when it is detected that the garbage site filling exceeds the preset threshold, the corresponding dispatch signal is triggered; In the fixed-point based primary dispatch signal generation, if the filling amount, volume or weight of the fixed-point garbage station exceeds the threshold, then: ; in It is a set of first-level scheduling signals; Number the spam sites; are the volume and weight overflow thresholds, respectively; They are the capacity and maximum load-bearing value of the garbage station respectively; is the monitoring type indicator, if , it means that the monitoring point is a fixed point. , it means that the monitoring point is a moving point; In the generation of secondary dispatch signals based on mobile points, if the garbage collection truck detects that the garbage station is overflowing during driving, the secondary dispatch signal is triggered, which is expressed as: ; in It is a set of secondary dispatch signals. The primary dispatch signal is triggered based on fixed points, and garbage collection trucks need to be dispatched to clear the garbage. The secondary dispatch signal is triggered based on mobile points, and the garbage collection trucks handle the garbage on their own or request additional support.

[0020] The dispatch center calculates the removal plan for the mobile point and calculates the optimal garbage removal route based on the primary and secondary dispatch signals; In the task priority calculation, the dispatch center calculates the removal priority of the garbage station and formulates Indicates spam site The task priority rating; ; ; ; ; in is the impact factor of the overflow severity. In practical applications, a larger value can be selected for the impact factor of the overflow severity to ensure that the garbage sites with garbage overflow are processed first. The geographical distance between the garbage station and the garbage collection truck affects the garbage collection. The garbage stations with closer distance should be cleared first. The impact factor of the garbage category is used to prioritize the removal of certain types of garbage. For example, when kitchen waste is identified, it can be selected for recycling first. The weight influencing factor of the garbage station is used to avoid the garbage station being too heavy, which may cause the garbage collection truck to have loading problems. For spam sites With garbage truck The geographical distance between For spam sites The geographic location coordinates of For spam sites A binary indicator function of whether it is abnormal; For spam sites GPS coordinates; For garbage collection trucks GPS coordinates; For spam sites Urgency rating of internal waste types; For the The category number of the garbage; The emergency treatment weight of this category of waste; In vehicle task allocation, garbage collection vehicles The cleaning task is expressed as: ; ; in For garbage collection trucks The optimal path of Penalties for overloading garbage collection trucks; Represents finding the path that minimizes the transportation cost ; represents the total driving distance of the garbage collection truck visiting the garbage sites in sequence, and Respectively represent and The location coordinates of the garbage stations are used to calculate the garbage collection truck from the garbage stations To the junk site driving distance; For garbage collection trucks The total number of spam sites to visit; is the overload penalty factor, which is used to control the impact of overload on path calculation. When a larger value is chosen, priority is given to selecting garbage collection vehicles that are not overloaded for collection and transportation to avoid overloaded vehicles; The maximum carrying capacity of garbage collection trucks; In the garbage collection truck, the garbage collection truck Garbage removal site , the remaining garbage volume and remaining garbage weight of the garbage station are updated as follows: ; ; in For garbage collection trucks The current volume of waste that can be removed; For garbage collection trucks The current weight of trash that can be removed; For trash can In time The volume of garbage at the moment; For trash can In time The remaining garbage volume at the time; For trash can In time The weight of the garbage at the moment; For trash can In time The weight of the remaining garbage at the moment; It means that if the garbage collection truck removes or Bigger than a trash can The current remaining amount prevents negative values ​​from appearing and ensures that the trash can The amount of remaining garbage will not be less than 0; When the garbage collection truck reaches the carrying capacity limit, it returns to the garbage station for unloading. This condition is judged as: ;in For the current The total volume of garbage loaded; is a logical condition, indicating that when the total amount of garbage loaded by the garbage collection truck is equal to the maximum carrying capacity, it must return to the garbage station to unload; In the transportation status update, when the garbage truck returns to the main station for unloading, the load status of the garbage truck will be reset; ; in For garbage collection trucks In time Load volume at the time; For garbage collection trucks In time The load weight at the moment; when the garbage collection truck completes unloading, the load volume and load weight are reset to zero to resume the next round of garbage collection task.

[0021] An intelligent garbage collection station monitoring system based on target tracking includes a data acquisition module, a target detection module, an abnormality monitoring module, a primary scheduling module, a secondary scheduling module, and a scheduling execution module; The data acquisition module is used to install the shooting unit in the target area and collect image data; the target area includes fixed points and mobile points; the fixed points include garbage sites; the mobile points include garbage collection vehicles; The target detection module detects garbage and people based on the collected image data through YOLOv5, and combines ByteTrack target tracking to maintain continuous monitoring of multiple garbage targets; The abnormal monitoring module will issue an alarm if the garbage status is judged to be abnormal during continuous monitoring, and prompt the mobile point to perform recycling; The first-level dispatch module includes: if the garbage status of a fixed point is determined to be abnormal, a first-level dispatch signal is sent to the dispatch center; The secondary dispatch module includes: if the mobile point identifies that the garbage status of any fixed point is abnormal during the movement, it sends a secondary dispatch signal to the dispatch center; The dispatch execution module calculates the dispatch execution recovery plan of the mobile point based on the primary dispatch signal and the secondary dispatch signal through the dispatch center.

[0022] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring an intelligent garbage collection station based on target tracking, comprising a shooting unit, characterized in that: The shooting unit is installed in the target area and image data is collected; the target area includes fixed points and moving points; the fixed points include garbage sites; Mobile points include garbage collection trucks; Based on the collected image data, garbage and people are detected through YOLOv5, and ByteTrack target tracking is combined to maintain continuous monitoring of multiple garbage targets; In continuous monitoring, if the garbage status is judged to be abnormal, an alarm will be issued and the mobile point will be prompted to perform recycling; If the garbage status at a fixed point is judged to be abnormal, a first-level dispatch signal is sent to the dispatch center; If the mobile point identifies that the garbage status of any fixed point is abnormal during the movement, it will send a secondary dispatch signal to the dispatch center; The dispatch center calculates the dispatch execution recovery plan of the mobile point based on the primary dispatch signal and the secondary dispatch signal.

2. According to claim 1, a method for monitoring an intelligent garbage recycling station based on target tracking is characterized in that: The monitoring types at fixed points include garbage type and garbage amount; the monitoring types at mobile points include garbage amount; In the detection of garbage and people based on YOLOv5, the target image preprocessing is included; input image After normalization, channel conversion, size adjustment and data enhancement, it is adapted to the input layer of YOLOv5; represents the normalized image; ; Indicates the minimum and maximum values ​​of pixel values; In order to adapt the input size of YOLOv5, we use bilinear interpolation to adjust the scale and propose Represents the adjusted image, that is, the image that is scaled to fit the network input requirements and is scaled by the scaling function , resize the input image to the specified target size; ; , They represent the input width and height expected by YOLOv5 respectively; In the detection of garbage and people based on YOLOv5, feature extraction is included; the input image is processed by the backbone feature extraction network of YOLOv5 to extract features at different levels; ； in is the input image; is the convolution kernel; The backbone feature extraction network Backbone is responsible for extracting features and uses residual connections to improve information flow, so that the deep network can learn more complex features. In the detection of garbage and people based on YOLOv5, the feature fusion part is included; The feature fusion part based on FPN structure combined with PAN structure is used for multi-scale feature fusion; ； in is the feature map of the current layer; Represents the underlying feature map; is the feature map of the previous layer; represents an upsampling operation; is the convolution operation; In the detection of garbage and people based on YOLOv5, there are target detection and bounding box prediction parts. In the target detection and bounding box prediction parts, the network is responsible for outputting the target's category label, bounding box coordinates, and confidence. ； in is the center coordinate of the target; Respectively represent the width and height of the target; Represents the category confidence of the target; for all target detection bounding boxes, calculate the total loss function : ； in is the target classification loss; is the bounding box loss; is the target confidence loss; is the loss weight; By introducing non-maximum suppression (NMS) in the target detection process, the intersection-over-union ratio between detection bounding boxes is calculated; non-maximum suppression (NMS) is used to remove duplicate bounding boxes and retain the target box; ； in Respectively represent candidate detection boxes and the Candidate detection boxes; is the intersection and comparison; Represents two bounding boxes and The area of ​​the overlapping region; Represents the area of ​​the union of two bounding boxes.

3. The method for monitoring an intelligent garbage collection station based on target tracking according to claim 2 is characterized in that: In ByteTrack target tracking, the goal of target tracking is to maintain the uniqueness and continuity of the detected targets in the video stream so that each target can be correctly tracked on the time axis. Target tracking relies on Kalman filtering for target state prediction and combines it with the Hungarian algorithm for target matching. Finally, a confidence target fusion strategy is adopted to ensure tracking stability. Target tracking includes modeling and predicting the target state based on Kalman filtering; Kalman filtering is used to predict the position of the target in the next frame; first, the target state is represented. In the multi-target tracking task, the state vector of each target is modeled as: ； in is the center coordinate of the target; The scale of the goal; is the target aspect ratio; are the velocity components of the target respectively; Secondly, the target state is updated; assuming that the target's motion is represented based on a linear model; ； ； ； ； in For the goal The state vector of the frame; is the state transfer matrix; Indicates the time interval between two frames; is the control input; is the process noise; is the observed value, i.e. the position of the target bounding box detected; is the observation matrix; the observation matrix represents the observation value, which includes the position and scale information; Target tracking includes target matching based on the Hungarian algorithm; the goal of target matching is to associate the target of the current frame with the tracking target of the previous item, so that the target ID remains unique in the time series, and the target matching calculation is based on Calculate the matching degree; ； in The target in the current frame and the previous frame target The degree of match between In the target matching process, the current frame target and the previous frame target are calculated. Relationship matrix, and the Hungarian algorithm is used to solve the target matching, which is expressed as: ； in is a binary decision variable. If the target With the goal match , then, otherwise 0; The constraints include: ； Constraints are used to make each target match a target in the previous frame; Target tracking includes target fusion, which adopts the confidence fusion strategy.

4. The method for monitoring an intelligent garbage collection station based on target tracking according to claim 3 is characterized in that: During the garbage site monitoring process, a weighing sensor is installed at a fixed point, and the first feature variable and the second feature variable are provided by YOLOv5 combined with ByteTrack target tracking and weighing sensor; The first characteristic variable is the type of garbage The second characteristic variable is the amount of garbage, which includes the total volume of garbage at the current garbage site. and spam sites Total weight ; The detection results based on YOLOv5 output are expressed as: ； in For fixed point Detected garbage datasets; is the amount of garbage detected in the current frame; For the The bounding box coordinates of the garbage; For the The classification results of garbage; For the The confidence level of garbage detection; Using the bounding box information of the target detection and the camera depth calculation, the volume of the garbage is estimated: ； in Depth information of the target; is the volume conversion factor for the type of garbage; is the volume adjustment factor; For fixed-point garbage stations, the actual weight is obtained based on the load cells installed: ； in For the Time spam site The total weight measured by the load cell; is the time step; Density conversion factor for garbage type.

5. The method for monitoring an intelligent garbage collection station based on target tracking according to claim 4 is characterized in that: For the generation of primary and secondary dispatch signals, when it is detected that the garbage site filling exceeds the preset threshold, the corresponding dispatch signal is triggered; In the generation of the first-level dispatch signal based on fixed points, if the filling amount of the fixed-point garbage station exceeds the threshold, then: ； in It is a set of first-level scheduling signals; Number the spam sites; are the volume and weight overflow thresholds, respectively; They are the capacity and maximum load-bearing value of the garbage station respectively; is the monitoring type indicator, if , it means that the monitoring point is a fixed point. , it means that the monitoring point is a moving point; In the generation of secondary dispatch signals based on mobile points, if the garbage collection truck detects that the garbage station is overflowing during driving, the secondary dispatch signal is triggered, which is expressed as: ； in It is a set of secondary dispatch signals. The primary dispatch signal is triggered based on fixed points, and garbage collection trucks need to be dispatched to clear the garbage. The secondary dispatch signal is triggered based on mobile points, and the garbage collection trucks handle the garbage on their own or request additional support.

6. The method for monitoring an intelligent garbage collection station based on target tracking according to claim 5 is characterized in that: The dispatch center calculates the removal plan for the mobile point and the garbage removal path based on the primary and secondary dispatch signals; In the task priority calculation, the dispatch center calculates the removal priority of the garbage station and formulates Indicates spam site Task priority rating; ； ； ； ； in is the influencing factor of spill severity; is the geographical distance factor from the garbage station to the garbage collection truck; is the impact factor of the garbage category; is the weight impact factor of the spam site; For spam sites With garbage truck The geographical distance between For spam sites The geographic location coordinates of For spam sites A binary indicator function of whether it is abnormal; For spam sites GPS coordinates; For garbage collection trucks GPS coordinates; For spam sites Urgency rating of internal waste types; For the The category number of the garbage; The emergency treatment weight of this category of waste; In vehicle task allocation, garbage collection vehicles The cleaning task is expressed as: ； ； in For garbage collection trucks Path; Penalties for overloading garbage collection trucks; Represents finding a path that can clear the cost ; It represents the total driving distance of the garbage collection truck visiting the garbage sites in sequence; is the overload penalty factor; The maximum carrying capacity of garbage collection trucks; In the garbage collection truck, the garbage collection truck Garbage removal site , the remaining garbage volume and remaining garbage weight of the garbage station are updated as follows: ； ； in For garbage collection trucks The volume of waste currently removed; For garbage collection trucks The weight of trash currently removed; For trash can In time The volume of garbage at the moment; For trash can In time The weight of the garbage at the moment; When the garbage collection truck reaches the carrying capacity limit, it returns to the garbage station for unloading. This condition is judged as: ;in For the current The total volume of garbage loaded; In the transportation status update, when the garbage truck returns to the main station for unloading, the load status of the garbage truck will be reset; ； in For garbage collection trucks In time Load volume at the time; For garbage collection trucks In time The load weight at the time; when the garbage truck After unloading is completed, the load volume and load weight are reset to zero to resume the next round of garbage removal tasks.

7. An intelligent garbage collection station monitoring system based on target tracking, comprising a data acquisition module, a target detection module, an abnormality monitoring module, a primary scheduling module, a secondary scheduling module, and a scheduling execution module, characterized in that: The data acquisition module is used to install the shooting unit in the target area and collect image data; the target area includes fixed points and mobile points; the fixed points include garbage sites; Mobile points include garbage collection trucks; The target detection module detects garbage and people based on the collected image data through YOLOv5, and combines ByteTrack target tracking to maintain continuous monitoring of multiple garbage targets; The abnormal monitoring module will issue an alarm if the garbage status is judged to be abnormal during continuous monitoring, and prompt the mobile point to perform recycling; The first-level dispatch module includes: if the garbage status of a fixed point is determined to be abnormal, a first-level dispatch signal is sent to the dispatch center; The secondary dispatch module includes: if the mobile point identifies that the garbage status of any fixed point is abnormal during the movement, it sends a secondary dispatch signal to the dispatch center; The dispatch execution module calculates the dispatch execution recovery plan of the mobile point based on the primary dispatch signal and the secondary dispatch signal through the dispatch center.

Images