Sanitation video garbage detection method based on uncertainty guide hierarchical retrieval

Abstract

The invention belongs to the technical field of artificial intelligence and computer vision, relates to an environmental sanitation video garbage detection method based on uncertainty guide hierarchical retrieval, and aims to solve the problem that an existing environmental sanitation video garbage detection method is high in labeling cost and weak in generalization ability or lacks historical context support, so that complex scenes are missed and missed. The method comprises the following steps: firstly, acquiring an environmental sanitation video frame sequence and dividing the sequence into fragments, extracting visual and garbage category language features through a pre-training encoder, and calculating a negative cosine distance to obtain category probability distribution; secondly, estimating the uncertainty of frames and fragments by using normalized Renyi entropy, retrieving high-confidence reference fragments from short-time, scene and global levels of a clean and junk double-memory library when the uncertainty is high, and directly inputting the high-confidence reference fragments into a pre-trained VLM model when the uncertainty is low; and finally, obtaining a final result through evidence perception time sequence attention fusion, and dynamically updating the memory bank. According to the scheme, additional training is not needed, the detection robustness and accuracy in complex scenes such as low illumination and shielding are effectively improved, and reliable technical support is provided for intelligent environmental sanitation operation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computer vision technology, and in particular relates to a method for detecting garbage in sanitation videos based on uncertainty-guided hierarchical retrieval. Background Technology

[0002] Intelligent sanitation is a core support for the modernization of urban environmental governance. Sanitation video-based waste detection, as a key technology, can identify waste disposal and accumulation in real time in streets, back alleys, and municipal squares, providing data support for cleaning scheduling and violation monitoring. However, in practical applications, waste detection often faces complex challenges such as low light, object obstruction, and changing viewing angles. Waste targets are often blurry, diverse in shape, and sparsely distributed, significantly increasing the difficulty of detection. Therefore, accurate and efficient waste detection technology is of great practical significance for improving sanitation operation efficiency and enhancing the urban environment.

[0003] Existing garbage detection methods in sanitation videos are mainly divided into two categories: one relies on large-scale labeled data to train models and achieve detection by learning the discrimination boundary between garbage and non-garbage. Although it has a certain accuracy in standard scenarios, the labeling cost is high, and its generalization ability is limited when there are insufficient training samples or when the scene changes; the other relies on pre-trained visual language models to achieve detection without additional training, which has stronger transferability and deployment flexibility. However, this type of method usually only makes judgments based on the current short video clips, lacks effective use of historical context information, is difficult to deal with complex scenarios with insufficient visual cues, and has obvious shortcomings in dynamically supplementing high-confidence reference information.

[0004] The aforementioned existing technologies have failed to effectively address the issues of false positives and false negatives caused by insufficient visual cues and lack of contextual support in complex sanitation scenarios. To overcome this bottleneck, there is an urgent need for a sanitation video garbage detection method that requires no additional training, can dynamically assess scenario uncertainty, and supplement historical reference information through hierarchical retrieval, thereby improving the system's robustness and accuracy in complex environments. Summary of the Invention

[0005] To address the problems existing in the background art, one aspect of the present invention provides a method for garbage detection in sanitation videos based on uncertainty-guided hierarchical retrieval, comprising:

[0006] S1: Obtain the sanitation video frame sequence of the target area and divide it into several non-overlapping video segments. Use a pre-trained visual encoder to extract the visual features of the video frames in the video segments.

[0007] S2: Extract the linguistic features of the garbage category prompt texts from m predefined garbage category prompt texts using a pre-trained visual encoder;

[0008] S3: Calculate the negative cosine distance between the visual features of the video frame and the linguistic features of the text prompts for each garbage category; and use Softmax to predict the category probability distribution of the video frame based on the negative cosine distance.

[0009] S4: Calculate the uncertainty of video frames using normalized Rényi entropy based on the category probability distribution of video frames, and then calculate the uncertainty of video segments.

[0010] S5: If the uncertainty of a video segment is greater than a set threshold, then query the memory bank for a historical video segment similar to the video segment as its reference video segment; input the video segment, its reference video segment, and the constructed prompt word template into the pre-trained VLM model to make predictions and obtain the initial prediction results of the video segment;

[0011] S6: If the uncertainty of the video segment is less than or equal to the set threshold, then input the video segment and the constructed prompt word template into the pre-trained VLM model to make predictions and obtain the initial prediction results of the video segment.

[0012] S7: The initial prediction result of the video segment is weighted, fused, and corrected based on the prediction results of the adjacent video segments before and after the video segment to obtain the final prediction result of the video segment;

[0013] S8: Update the memory bank based on the final prediction results of the video segment.

[0014] Another aspect of the present invention provides a garbage detection system for sanitation videos based on uncertainty-guided hierarchical retrieval. The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the aforementioned garbage detection method for sanitation videos based on uncertainty-guided hierarchical retrieval.

[0015] Another aspect of the present invention provides a computer storage medium storing a remote monitoring program, which, when executed by a processor, implements the aforementioned method for detecting garbage in sanitation videos based on uncertainty-guided hierarchical retrieval.

[0016] The present invention has at least the following beneficial effects

[0017] This invention avoids the problems of high annotation costs and weak scene generalization ability of traditional sanitation video garbage detection methods that rely on large-scale labeled data. It also overcomes the shortcomings of existing visual language model methods that rely only on short-term segments and lack historical context, making them prone to false positives and false negatives in complex scenes such as low light and occlusion. Through an uncertainty evaluation mechanism based on normalized Rényi entropy, it can accurately identify high-difficulty scenes with insufficient visual cues and trigger retrieval. Then, through short-term, scene-level, and global-level hierarchical retrieval, it retrieves high-confidence historical reference segments from dual cleanliness and garbage memory banks, providing sufficient spatiotemporal context for the current detection. Combined with evidence-aware temporal attention fusion, it can effectively suppress instantaneous interference such as pedestrian occlusion and vehicle passage, improving the temporal stability of the detection results. Meanwhile, the dynamically updated memory bank can retain typical cleanliness and garbage scene samples for a long time, ensuring the reliability and representativeness of the reference information. The overall solution significantly improves the robustness and accuracy of garbage detection in complex sanitation scenarios without additional training, providing more reliable technical support for real-time scheduling and precise supervision of intelligent sanitation operations, and effectively contributing to the improvement of urban environmental governance efficiency. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the method flow framework of the present invention. Detailed Implementation

[0019] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0020] Please see Figure 1 One aspect of the present invention provides a training method for garbage detection in sanitation videos based on uncertainty-guided hierarchical retrieval, comprising:

[0021] S1: Obtain the sanitation video frame sequence of the target area and divide it into several non-overlapping video segments. Use a pre-trained visual encoder to extract the visual features of the video frames in the video segments.

[0022] S2: Extract the linguistic features of the garbage category prompt texts from m predefined garbage category prompt texts using a pre-trained visual encoder;

[0023] In this embodiment, the sanitation video frame sequence of the target area is first acquired. These videos can come from the front-facing camera of the road sweeper, fixed monitoring at garbage disposal points, or devices worn by sanitation inspectors, covering typical sanitation scenarios such as streets, back alleys, and municipal squares. Then, the video frame sequence is divided into several non-overlapping video segments according to a fixed number of frames η (e.g., 5 frames / segment, taking into account both temporal continuity and computational efficiency) to avoid information loss caused by isolated judgment of single frames. Next, a pre-trained visual encoder (such as the InternViT architecture visual encoder of OpenGVLab / InternVL-14B-224px) is used to extract features from the single frame image in each segment. This encoder adapts to the 224px input size through dynamic block technology, and can capture detailed features such as the local contour of plastic bottles and the shape of scattered paper scraps while retaining global information such as the position of obstructions in the overall road environment, and finally outputs a high-dimensional visual feature vector for each frame. In S2, a predefined set of waste category prompt texts is constructed for m typical waste categories of concern to sanitation operations (such as plastic bottles on sidewalks, paper scraps overflowing from trash cans, glass shards on driveways, medical waste at collection points, etc.). Then, a pre-trained text encoder (such as the text module trained across modal alignment in InternVL-14B-224px) sharing the embedding space with the visual encoder is used to transform each prompt text into a linguistic feature vector that matches the semantics of the visual features. This ensures that the association between the visual image and the waste category can be established through feature similarity calculation. These two steps, through standardized video processing and cross-modal feature extraction, avoid the cost of additional annotation and training, and lay a unified feature foundation for subsequent feature similarity calculation and uncertainty assessment, ensuring the accuracy and reliability of the subsequent detection process from the source.

[0024] S3: Calculate the negative cosine distance between the visual features of the video frame and the linguistic features of the text prompts for each garbage category; and use Softmax to predict the category probability distribution of the video frame based on the negative cosine distance.

[0025] Preferably, the category probability distribution of the video frame includes:

[0026]

[0027]

[0028] in, Indicates the first in the video clip The video frame and the first The negative cosine distance between the text prompts for each garbage category; Indicates the first in the video clip Visual features of each video frame; Indicates the first The linguistic features of the text indicating each type of waste; Indicates the first in the video clip The video frame belongs to the first The probability of garbage.

[0029] In this embodiment, the semantic similarity between visual images and waste categories is transformed into quantifiable and comparable probability data, establishing a direct association between visual information in sanitation scenarios and waste categories, providing a core input for calculating uncertainty through probability distribution.

[0030] S4: Calculate the uncertainty of video frames using normalized Rényi entropy based on the category probability distribution of video frames, and then calculate the uncertainty of video segments.

[0031] Preferably, the uncertainty of the video segment includes:

[0032]

[0033]

[0034] in, Indicates the first in the video clip The uncertainty of each video frame; Indicates the number of video frames in a video clip; Indicates video clip Uncertainty; This represents an adjustable hyperparameter that is greater than 0 and not equal to 1; Indicates the first in the video clip The video frame belongs to the first The probability of garbage.

[0035] In this embodiment, step S4 standardizes the detection difficulty of complex scenes such as low light, occlusion, and changing viewpoints in sanitation videos. This avoids false triggering of retrieval for low-difficulty segments such as clear garbage frames and completely clean frames (reducing computational overhead), while accurately identifying high-difficulty segments such as blurry objects and semantically confused segments (providing accurate trigger signals for subsequent hierarchical retrieval). From a decision-making perspective, this lays the foundation for balancing detection accuracy and operational efficiency in the solution. At the same time, normalization ensures the consistency of uncertainty assessment under different sanitation scenarios and different garbage categories and quantities.

[0036] S5: If the uncertainty of a video segment is greater than a set threshold, then query the memory bank for a historical video segment similar to the video segment as its reference video segment; input the video segment, its reference video segment, and the constructed prompt word template into the pre-trained VLM model to make predictions and obtain the initial prediction results of the video segment;

[0037] Preferably, the step of querying the memory bank for historical video clips similar to the video clip as reference video clips includes: the memory bank includes: a clean scene memory bank. and garbage scene memory bank Each record in the memory bank contains visual features of the video clip. garbage probability Model confidence With uncertainty ;

[0038] S51: Memory library for cleaning scenarios and garbage scene memory bank They are divided into three different levels:

[0039] ={ }

[0040] ={ }

[0041] ={ }

[0042] in, Represents the cleaning scene memory bank Or garbage scene memory bank A collection of video clips; Indicates video clip The average time of all video frames in the video; Indicates video clip The average time of all video frames in the video; This indicates the set threshold.

[0043] S52: Calculate video segments Visual features and video clips Similarity of visual features:

[0044]

[0045] in, Indicates video clip visual features With video clips visual features Similarity;

[0046] S53: Memory library for cleaning scenarios and garbage scene memory bank Perform a layer-by-layer Top-K search, prioritizing from... Selecting samples with similarity greater than or equal to a set threshold video clip Expand to when quantity is insufficient If that's still not enough, expand further to Continue this process until the required number of elements is selected or all levels are traversed; this will yield the clean reference set. and garbage reference set Merge Cleaning Reference Set and garbage reference collection The final reference set is obtained. Final reference set The video clips in the video are video clips. Reference video clip.

[0047] In this embodiment, step S5 is the core supplementary reasoning step for difficult-to-judge sanitation video clips. It is only activated when the uncertainty of the clip is determined to be higher than the threshold in S4—for example, in scenarios such as blurred objects on the road under low light, electric vehicles obscuring part of the garbage, or rain reflecting light causing loss of image details. In these cases, it is difficult to accurately determine whether there is garbage based solely on the current clip information, and historical reference information needs to be introduced to assist in decision-making. First, the system will call two pre-maintained memory banks: one is a clean scene memory bank, which stores high-confidence clips that are clearly clean in the past, such as clear footage of garbage-free roads on sunny days and street scenes after cleaning. Each record also includes the visual characteristics and confidence level of these clips; the other is a garbage scene memory bank, which stores high-confidence clips that clearly contain garbage, such as clearly visible piles of plastic bottles and scattered paper scraps outside garbage cans, also with detailed judgment information. Then, the system will search for reference clips layer by layer according to the logic of near to far and local to global: prioritizing searching from short-term memory— This refers to the most recent high-confidence clip captured at the time of the current clip, such as footage taken 5 minutes ago by the same street sweeper. Scenes with similar times and environments are more similar and therefore have higher reference value. If short-term memory doesn't have enough suitable clips, it expands to scene-level memory, which includes all high-confidence clips from the same sanitation scene (e.g., the same backstreet or the same municipal square), covering typical states of that scene at different times. If scene-level memory still isn't sufficient, it finally retrieves global memory, which contains all historically processed high-confidence clips from different sanitation scenes, ensuring a usable sample can be found. During the retrieval process, the system compares the visual similarity between the current clip and historical clips, selecting only clips that look similar (e.g., similar road layouts or object outlines). Finally, it selects suitable cleaning reference clips and garbage reference clips from the two memory banks respectively, merging them into a complete reference set.Finally, the system inputs the current segment to be judged, a reference set labeled with clean or garbage attributes, and preset detection prompts (such as judging whether the current segment contains garbage like plastic bottles and paper scraps by combining reference images of clean roads and garbage) into a pre-trained vision-language model. This allows the model to reason with reference and comparison, outputting the initial detection result of the current segment—including the probability of garbage presence, the credibility of the judgment, and natural language explanation (e.g., the current blurry object is similar in shape to a plastic bottle in a reference garbage segment, indicating the probability of garbage presence). The core effect of this step is that it supplements the crucial historical context for complex sanitation scenarios with insufficient information, solving the problem of existing methods that rely solely on blind judgment based on the current segment without training. For example, when it is difficult to distinguish between plastic bags and fallen leaves under low light, referring to clear images of plastic bags and fallen leaves in the reference memory can accurately differentiate them. At the same time, the layer-by-layer retrieval logic ensures the relevance of reference information (prioritizing the most recent and most similar images) while avoiding the inefficiency caused by blind retrieval, making the model's judgment more accurate and reliable in complex scenarios.

[0048] S6: If the uncertainty of the video segment is less than or equal to the set threshold, then input the video segment and the constructed prompt word template into the pre-trained VLM model to make predictions and obtain the initial prediction results of the video segment.

[0049] Preferably, when the pre-trained VLM model predicts a video segment, if the uncertainty of the video segment is greater than a set threshold, the prediction of the video segment by the pre-trained VLM model is expressed as follows:

[0050]

[0051]

[0052] If the uncertainty of a video segment is less than or equal to a set threshold, the prediction made by the pre-trained VLM model for the video segment is as follows:

[0053]

[0054] in, Indicates the constructed prompt word template; This represents a pre-trained VLM model; , and These represent video clips. The garbage probability, model confidence, and natural language interpretation, i.e., the initial prediction results for the video segment; the final reference set. Each reference video in the document has an explicit label indicating whether it is clean or junk.

[0055] In this embodiment, step S6 is the efficient reasoning step for sanitation video clips with sufficient information and easy judgment. It is only activated when the uncertainty of the clip determined in S4 is less than or equal to a preset threshold. These scenarios are mostly where visual cues are clear, such as streets with ample sunlight at noon on a sunny day (where the outlines of garbage such as plastic bottles and paper scraps are clearly visible), or clean roads without obstructions after sweeping (where there are no suspected garbage objects). In these cases, the visual information of the current clip is sufficient for accurate judgment, and there is no need to call historical references. In specific operation, the system does not need to start a hierarchical retrieval process (nor does it access the cleaning or garbage scene memory database). It directly inputs the current video clip to be judged, along with preset garbage detection prompts (such as judging whether there are plastic bottles, paper scraps, glass shards, etc. in this sanitation video clip, and outputting the judgment result, credibility, and reason) into the pre-trained visual-language model. The model will directly analyze the matching relationship between the visual features of the current clip and the semantics of the garbage category based on its existing cross-modal alignment capability (without additional training), and output the initial prediction result: including the probability that there is garbage in the current clip, the credibility of the model's judgment, and a natural language explanation. The core effect of this step is to skip redundant retrieval processes in simple and clear sanitation scenarios, significantly reducing computational overhead (no need to retrieve or compare historical memory data) and improving detection efficiency. At the same time, because the scene has sufficient visual information, the results of the model's direct inference are already highly reliable and will not be interfered with by introducing extra reference information. This ensures both detection speed in easily judged scenarios and high accuracy, allowing the solution to be flexibly adapted to the complexity of sanitation scenarios, balancing efficiency and accuracy.

[0056] S7: The initial prediction result of the video segment is weighted, fused, and corrected based on the prediction results of the adjacent video segments before and after the video segment to obtain the final prediction result of the video segment;

[0057] Preferably, the weighted fusion correction of the initial prediction results for the video segment includes:

[0058] S71: Define Time Window , Indicates the preset length. Indicates video clip The average time of all video frames in the video;

[0059] S72: For each video segment within the time window Calculate video segments Evidence feature vector :

[0060]

[0061] in, Indicates video clip Model confidence; Indicates video clip Uncertainty;

[0062] S73: Define the query vector Weight matrix Calculate evidence score :

[0063]

[0064] in, Temperature coefficient;

[0065] S74: Attention weights are derived from normalized evidence scores.

[0066]

[0067] S75: Fusion Time Window The probability of garbage content for all video clips is used to obtain the video clips. Final detection probability:

[0068]

[0069] in, Indicates video clip The final detection probability.

[0070] In this embodiment, step S7 is a key optimization step to improve the temporal stability of the sanitation video detection results. Regardless of whether it is a high uncertainty scenario or a low uncertainty scenario, after the initial prediction result is output, it must be corrected through this step. This is because sanitation videos are shot continuously, and the initial result of a single segment may be affected by instantaneous interference, such as pedestrians briefly obscuring the road surface, the sweeper brush temporarily entering the frame, or the shadow of a tree quickly passing by, which may lead to fluctuations or misjudgments in the results. In practice, the system first sets a time window covering several adjacent segments before and after the current segment (e.g., two before and two after, for a total of five segments). These segments were captured at similar times and have coherent scene environments, reflecting the true context of the current segment. Next, for each segment within the window, evidence features are constructed based on its initial prediction confidence and uncertainty. Simply put, segments with high confidence and low uncertainty (such as clear, unobstructed, clean road frames) have stronger evidence, while segments with low confidence and high uncertainty (such as frames misjudged due to momentary occlusion) have weaker evidence. Subsequently, the system assigns attention weights to each segment based on its evidence strength, with reliable segments receiving higher weights and interfering segments receiving lower weights. Finally, the initial garbage probabilities of all segments within the window are fused according to their weights to obtain the final detection probability of the current segment.

[0071] For example, if a segment is mistakenly identified as containing litter due to pedestrian occlusion (initial probability 70%, confidence 50%), but the four adjacent segments are all clean road surfaces (probability ≤ 10%, confidence ≥ 90%), the high weight of the adjacent segments during fusion will significantly lower the final probability, correcting it to a litter probability of 15%. The core effect of this step is that it effectively suppresses single-segment misjudgments caused by momentary occlusion and temporary interference in sanitation scenarios, making the detection results more stable in the time dimension. This avoids false alarms triggered by accidental interference (such as misjudging litter due to pedestrians passing by) and also prevents the failure to detect persistent litter due to brief occlusion (such as plastic bottles being confirmed by the preceding and following frames even after brief occlusion), significantly improving the reliability and practicality of the detection results.

[0072] S8: Update the memory bank based on the final prediction results of the video segment.

[0073] Preferably, updating the memory includes:

[0074] S81: After completing the video clip After the detection, for the video clip Construct candidate memory entries :

[0075]

[0076] in, Indicates video clip Visual features; Indicates video clip Uncertainty; Indicates video clip Model confidence; Indicates video clip Detection probability;

[0077] S82: Determine whether to write to the memory bank according to the following rules:

[0078] like and Then Write to the cleaning scene memory bank ;

[0079] like and Then Write to the garbage scene memory bank ;

[0080] S83: If the scene memory bank is cleaned and garbage scene memory bank The size exceeds the defined capacity limit Calculate the priority score for each memory entry and eliminate the entry with the lowest priority score:

[0081]

[0082] in, To weigh the parameters; Indicates video clip Model confidence; This indicates a video clip. Uncertainty.

[0083] In this embodiment, step S8 is a dynamic maintenance step that maintains the long-term reliability of the memory bank and provides high-quality references for subsequent detection. It is initiated after each video segment completes its final prediction. The core is to ensure that the clean scene memory bank and the garbage scene memory bank always store highly reliable and representative samples, avoiding redundant or low-quality data from affecting subsequent retrieval results. In specific operation, the system first constructs a candidate memory entry for the current segment, which includes the segment's visual features (such as road environment, object outline characteristics), the uncertainty (degree of ambiguity) during final detection, the confidence level (reliability of judgment), and the final garbage probability (conclusion of whether it is garbage). Then, it determines whether to write it into the memory bank according to the rule of high confidence + clear conclusion: only when the segment's confidence level is higher than a set threshold (ensuring reliable judgment) and the final garbage probability is lower than the clear clean threshold, will the candidate entry be written into the clean scene memory bank (such as a clean road after sweeping); if the final garbage probability is higher than the clear garbage threshold, it will be written into the garbage scene memory bank (such as a clear pile of plastic bottles); while blurry scenes (such as light-colored objects that are difficult to distinguish) or low-confidence segments will be directly discarded to avoid introducing noisy data.

[0084] Simultaneously, the system sets capacity limits for both memory banks to prevent storage overload after long-term operation. When one memory bank exceeds its capacity, the lowest priority entries are discarded based on the rule that higher confidence and lower uncertainty lead to higher priority. For example, early stored, low-confidence, blurry cleaning frames are replaced by new, high-confidence, clear cleaning frames. The core effect of this step is that, through dynamic filtering and priority discarding, the memory banks always retain high-value samples of typical cleaning and typical waste in sanitation scenarios (such as clean road surfaces under different lighting and road sections, and typical forms of various types of waste). This provides a reliable reference for subsequent S5 hierarchical retrieval (avoiding the retrieval of low-quality or irrelevant samples), while controlling storage and retrieval overhead, ensuring that the memory banks adapt to changes in sanitation scenarios in the long term and supporting the continuous and stable operation of the solution.

[0085] Another aspect of the present invention provides a garbage detection system for sanitation videos based on uncertainty-guided hierarchical retrieval. The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the aforementioned garbage detection method for sanitation videos based on uncertainty-guided hierarchical retrieval.

[0086] Another aspect of the present invention provides a computer storage medium storing a remote monitoring program, which, when executed by a processor, implements the aforementioned method for detecting garbage in sanitation videos based on uncertainty-guided hierarchical retrieval.

[0087] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0088] In summary, this invention avoids the problems of high annotation costs and weak scene generalization ability of traditional sanitation video garbage detection methods that rely on large-scale labeled data. It also makes up for the shortcomings of existing visual language model methods that do not require training, which rely only on short-term segments and lack historical context support, making them prone to false positives and false negatives in complex scenes such as low light and occlusion. By using an uncertainty evaluation mechanism based on normalized Rényi entropy, it can accurately identify high-difficulty scenes with insufficient visual cues and trigger retrieval. Then, through short-term, scene-level, and global-level hierarchical retrieval, it retrieves high-confidence historical reference segments from the dual cleanliness and garbage memory banks, providing sufficient spatiotemporal context for the current detection. Combined with evidence-aware temporal attention fusion, it can effectively suppress instantaneous interference such as pedestrian occlusion and vehicle passing, improve the temporal stability of detection results, and the dynamically updated memory bank can retain typical cleanliness and garbage scene samples for a long time, ensuring the reliability and representativeness of reference information. The overall solution significantly improves the robustness and accuracy of garbage detection in complex sanitation scenarios without requiring additional training, providing more reliable technical support for real-time scheduling and precise supervision of intelligent sanitation operations, and effectively helping to improve the efficiency of urban environmental governance.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for garbage detection in a video based on an uncertainty-guided hierarchical search, characterized in that, The method comprises the following steps: S1: obtaining a sequence of sanitation video frames of a target area, dividing the sequence into a plurality of non-overlapping video clips, and extracting visual features of the video frames in the video clips using a pre-trained visual encoder; S2: extracting language features of garbage category prompt texts for m pre-defined garbage categories using the pre-trained visual encoder; S3: calculating negative cosine distances between the visual features of the video frames and the language features of the garbage category prompt texts, and generating a category probability distribution of the video frames by predicting the negative cosine distances using Softmax; S4: calculating the uncertainty of the video frames according to the category probability distribution of the video frames using normalized Rényi entropy, and then calculating the uncertainty of the video clips; S5: if the uncertainty of the video clip is greater than a set threshold, querying a historical video clip similar to the video clip from a memory bank as a reference video clip of the video clip, and inputting the video clip, the reference video clip, and a constructed prompt word template into a pre-trained VLM model to obtain an initial prediction result of the video clip; S6: if the uncertainty of the video clip is less than or equal to the set threshold, inputting the video clip and the constructed prompt word template into the pre-trained VLM model to obtain the initial prediction result of the video clip; S7: weighting and fusing the initial prediction result of the video clip according to the prediction results of adjacent video clips before and after the video clip to obtain a final prediction result of the video clip; S8: updating the memory bank according to the final prediction result of the video clip. 2.The sanitation video garbage detection method based on uncertainty-guided hierarchical retrieval according to claim 1, wherein, The category probability distribution of the video frame comprises: in, Indicates the first in the video clip The video frame and the first The negative cosine distance between the text prompts for each garbage category; Indicates the first in the video clip Visual features of each video frame; Indicates the first The linguistic features of the text indicating each type of waste; Indicates the first in the video clip The video frame belongs to the first The probability of garbage. 3.The sanitation video garbage detection method based on uncertainty-guided hierarchical retrieval according to claim 1, characterized in that, The uncertainty of the video clip comprises: in, Indicates the first in the video clip The uncertainty of each video frame; Indicates the number of video frames in a video clip; Indicates video clip Uncertainty; This represents an adjustable hyperparameter that is greater than 0 and not equal to 1; Indicates the first in the video clip The video frame belongs to the first The probability of garbage.

4. The sanitation video garbage detection method based on uncertainty-guided hierarchical retrieval according to claim 1, characterized in that, The querying the memory base for historical video clips similar to the video clip as reference video clips thereof comprises that the memory base comprises a clean scene memory base and a garbage scene memory base ; each record in the memory base contains visual features , garbage probability , model confidence and uncertainty of the video clip. S51: For cleaning scene memory library and garbage scene memory library are divided into three different levels respectively: ={ } ={ } ={ } wherein, represents a set of video segments in a clean scene memory bank or a garbage scene memory bank ; represents an average time of all video frames in a video segment ; represents an average time of all video frames in a video segment ; represents a set threshold value; S52: Calculate visual features of the video segment S54: Calculate similarity between visual features of the video segment and the visual features of the video segment wherein, representing a visual feature of a video segment representing a similarity of a visual feature of a video segment representing a visual feature of a video segment​​ S53: Memory library for cleaning scenarios and garbage scene memory bank Perform a layer-by-layer Top-K search, prioritizing from... Selecting samples with similarity greater than or equal to a set threshold video clip Expand to when quantity is insufficient If that's still not enough, expand further to Continue this process until the required number of elements is selected or all levels are traversed; this will yield the clean reference set. and garbage reference set Merge Cleaning Reference Set and garbage reference collection The final reference set is obtained. Final reference set The video clips in the video are video clips. Reference video clip.

5. The method according to claim 4, wherein, When the pre-trained VLM model predicts the video clip, if the uncertainty of the video clip is greater than the set threshold, the prediction of the pre-trained VLM model on the video clip is represented as: If the uncertainty of the video clip is less than or equal to the set threshold, the prediction of the pre-trained VLM model on the video clip is represented as: wherein, represents a constructed prompt word template; represents a pre-trained VLM model; , and respectively represent the garbage probability, model confidence and natural language explanation of the video segment , i.e. the initial prediction result of the video segment; each reference video in the final reference set has an explicit labeled clean or garbage attribute.

6. The sanitation video garbage detection method based on uncertainty-guided hierarchical retrieval according to claim 1, characterized in that, The weighting and fusion correction of the initial prediction result of the video clip comprises: S71: define a time window , denotes a preset length, denotes the average time of all video frames of the video segment . S72: For each video segment within the time window , compute the evidence feature vector of the video segment : wherein, represents a model confidence of a video segment ; represents an uncertainty of a video segment ; S73: define query vector , weight matrix , compute evidence score : wherein is the temperature coefficient; S74: normalizing the evidence score to obtain an attention weight: S75: Fusion Time Window The probability of garbage content for all video clips is used to obtain the video clips. Final detection probability: wherein, represents the final detection probability of a video segment .

7. The method according to claim 4, wherein, The updating of the memory bank comprises: S81 : upon completion of the detection of the video segment , the candidate memory entry is constructed for the video segment : wherein, represents a visual feature of a video segment ; represents an uncertainty of a video segment ; represents a model confidence of a video segment ; represents a detection probability of a video segment ; S82: judging whether to write into the memory bank according to the following rules: If and , then write clean scene memory ; If and , then write to the junk scene memory bank ; S83: If the size of the cleaning scene memory bank and the trash scene memory bank exceeds the defined upper capacity limit , calculate a priority score for each memory entry, and eliminate the entry with the lowest priority score: wherein, is a trade-off parameter; denotes a model confidence of the video segment denotes an uncertainty of the video segment denotes a model confidence of the video segment denotes an uncertainty of the video segment 8.A sanitation video garbage detection system based on uncertainty-guided hierarchical retrieval, characterized in that, The system comprises a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the sanitation video garbage detection method based on uncertainty-guided hierarchical retrieval according to any one of claims 1 to 7.

9. A computer storage medium, characterized in that The computer storage medium stores a remote monitoring program, and the remote monitoring program is executed by the processor to implement the sanitation video garbage detection method based on uncertainty-guided hierarchical retrieval according to any one of claims 1 to 7.

Images