Picture recognition-based household garbage traceability method, system, device and medium

Abstract

The application belongs to the technical field of garbage traceability, and particularly relates to a household garbage traceability method, system, device and medium based on picture recognition, which comprises the following steps: collecting multi-source monitoring data based on a monitoring device, processing the multi-source monitoring data to obtain target image data; constructing an optimized Mask R-CNN architecture model, performing garbage recognition based on the optimized Mask R-CNN architecture model and the target image data to obtain a garbage recognition result; determining a monitoring blind area based on the multi-source monitoring data, judging whether the monitoring blind area has a vehicle entering a blind area associated section or new garbage based on the multi-source monitoring data and the garbage recognition result, and performing blind area traceability based on a judgment result; determining a potential responsible vehicle based on a traceability result, calculating an association probability of the potential responsible vehicle and the new garbage, and determining a target responsible vehicle based on the association probability. The application improves the efficiency of rural household garbage management.

Description

Technical Field

[0001] This application belongs to the field of waste tracing technology, and in particular relates to methods, systems, equipment and media for tracing municipal solid waste based on image recognition. Background Technology

[0002] With the acceleration of urbanization and the enhancement of environmental awareness, the management of rural domestic waste has an increasingly significant impact on environmental sanitation, residents' health, and ecological balance. Rural domestic waste not only affects aesthetics but may also trigger a series of social problems such as environmental pollution and the spread of diseases.

[0003] Traditional waste monitoring relies primarily on manual reporting or fixed-location camera footage, which has significant limitations. On one hand, manual reporting is often delayed and inaccurate; on the other hand, fixed cameras, limited by their installation location and field of view, struggle to capture waste in narrow alleyways or remote corners. Therefore, efficiently and accurately acquiring waste information has become a major challenge.

[0004] Existing technologies are clearly insufficient to provide effective solutions to the problems existing in rural domestic waste management. Therefore, an integrated and intelligent approach is needed to comprehensively optimize aspects such as data collection, waste identification, and the probability of associating waste with specific vehicle sources. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method, system, equipment, and medium for tracing the source of domestic waste based on image recognition. Employing integrated and intelligent technologies, through multi-source monitoring data collection, optimized model construction, and multi-dimensional source analysis, it achieves accurate identification of rural domestic waste and efficient location of responsible vehicles. This effectively compensates for the shortcomings of traditional waste monitoring methods and significantly improves the efficiency and quality of rural domestic waste management.

[0006] Firstly, this application provides a method for tracing the source of household waste based on image recognition, the method comprising: Based on the collection of multi-source monitoring data by monitoring equipment, the multi-source monitoring data is processed to obtain target image data; An optimized Mask R-CNN architecture model is constructed, and garbage identification is performed based on the optimized Mask R-CNN architecture model and the target image data to obtain garbage identification results; Based on the multi-source monitoring data, a monitoring blind spot is determined. Based on the multi-source monitoring data and the garbage identification result, it is determined whether there are vehicles entering the road section associated with the blind spot or new garbage in the monitoring blind spot. Based on the judgment result, the source of the blind spot is traced. Based on the source tracing results, potential responsible vehicles are identified, the correlation probability between potential responsible vehicles and newly added waste is calculated, and the target responsible vehicle is determined based on the correlation probability.

[0007] Furthermore, Processing multi-source monitoring data specifically includes: A spatiotemporal dual-threshold dynamic frame capture algorithm is used to extract keyframes from video streams captured by a fixed camera. A time offset compensation model is used to perform time synchronization processing on the keyframes and image data collected from UAV inspections to obtain target image data.

[0008] Furthermore, Building and optimizing the Mask R-CNN architecture model specifically includes: A differentiable homography transformation layer is introduced before the backbone network of the initial Mask R-CNN architecture model to obtain an updated Mask R-CNN architecture model; The updated Mask R-CNN architecture model is trained, and the model is optimized based on a multi-task loss function that includes a physical constraint loss term to obtain an optimized Mask R-CNN architecture model.

[0009] Furthermore, Blind spot tracing based on the judgment results specifically includes: Based on the preset coverage area of ​​the fixed cameras and the actual coverage area of ​​the drone inspection, identify the monitoring blind spots that are not effectively monitored; Determine whether a vehicle has entered the blind spot-related road section based on real-time monitoring data from fixed surveillance cameras near the nearby iron tower; The presence of new garbage is determined based on real-time images collected by the drone and the garbage identification results of the optimized Mask R-CNN architecture model. If new garbage is found in the monitoring blind spot or a vehicle is detected entering the road section associated with the blind spot, the blind spot tracing process will be initiated.

[0010] Furthermore, Based on the source tracing results, potential liable vehicles were identified, specifically including: Calculate the backtracking time window based on the waste identification results; Extract all vehicles that entered the associated road segment of the monitoring blind zone detected by the nearby camera within the backtracking time window, and perform license plate recognition and vehicle type classification on all vehicles. Based on the license plate recognition results and vehicle type classification results, filter out garbage dumping vehicles and use the garbage dumping vehicles as candidate vehicles. An entrance-exit matching analysis is performed on the candidate vehicles. If any candidate vehicle is detected at the blind spot entrance but not at the exit within a preset time, the vehicle is marked as a potential entry vehicle. Based on the historical dumping speed threshold corresponding to the candidate vehicle model and the amount of newly added waste, the actual dwell time threshold is calculated. If the actual dwell time threshold exceeds the preset dwell time threshold, the candidate vehicle is identified as a potential responsible vehicle.

[0011] Furthermore, Calculate the probability of association between potentially liable vehicles and newly generated waste, specifically including: Calculate the spatial distance between any of the aforementioned potentially liable vehicles and the newly added waste; Calculate the time difference between the timestamp of the vehicle passing through the associated road segment and the timestamp of the newly added waste identification; Determine the degree of path matching between the vehicle's travel path and the newly added garbage collection point; Based on the spatial distance, the time difference, and the path matching degree, the probability of association between the vehicle and the newly added waste is calculated.

[0012] Furthermore, This also includes tracing the source and strengthening the chain of evidence for potential offending vehicles illegally dumping, specifically including: Based on the target image data, a dual-branch neural network model is used to identify the cargo loading status of potentially responsible vehicles. Based on the vehicle re-identification algorithm, the license plate recognition results, vehicle type classification results and color features are integrated to match the images of the same potentially responsible vehicle between different cameras, and the matching results are used to determine whether illegal dumping has occurred. Based on the driving path and timestamp information of the potentially responsible vehicle, the location of the suspected illegal dumping is estimated, and the correlation probability is corrected based on the suspected illegal dumping.

[0013] Secondly, based on the same inventive concept, this application provides a waste tracing system based on image recognition, the system comprising: The image acquisition module is used to collect multi-source monitoring data based on monitoring equipment, process the multi-source monitoring data, and obtain target image data; The identification module is used to construct an optimized Mask R-CNN architecture model, and to perform garbage identification based on the optimized Mask R-CNN architecture model and the target image data to obtain garbage identification results; The source tracing module is used to determine the monitoring blind zone based on the multi-source monitoring data, and to determine whether there are vehicles entering the blind zone or new garbage in the monitoring blind zone based on the multi-source monitoring data and the garbage identification results, and to perform source tracing of the blind zone based on the judgment results. The target determination module is used to identify potential responsible vehicles based on the source tracing results, calculate the association probability between potential responsible vehicles and newly added waste, and determine the target responsible vehicle based on the association probability.

[0014] Thirdly, this application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When the processor executes the program stored in the memory, it implements the steps of any of the image recognition-based methods for tracing the source of household waste as described above.

[0015] Fourthly, this application also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the image recognition-based household waste tracing methods described above.

[0016] Compared with the prior art, this application has the following advantages: 1. This application integrates multi-source monitoring data from fixed cameras and drone inspections, and combines a spatiotemporal dual-threshold dynamic frame capture algorithm with a time offset compensation model to achieve comprehensive and efficient collection and synchronous processing of garbage conditions in a wide rural area, solving the problems of limited coverage and incomplete data collection of traditional fixed cameras.

[0017] 2. This application introduces a differentiable homography transformation layer into the Mask R-CNN architecture and combines it with physical constraint loss for model optimization. This effectively corrects the distortion of wide-angle cameras, enhances the model's ability to identify garbage under complex optical conditions such as low light, rain, and fog, and improves the recognition accuracy and system stability.

[0018] 3. This application, through a dynamic judgment and triggering mechanism based on monitoring blind spots, combined with license plate recognition, vehicle type classification, entrance-exit matching, and dwell time analysis, can achieve lightweight backtracking of vehicles entering blind spots and accurate identification of potentially responsible vehicles even in the absence of complete vehicle trajectory data. This effectively solves the problem of traceability failure caused by the lack of blind spot data in traditional methods.

[0019] 4. This application constructs a correlation probability model by integrating spatial distance, time difference, and path matching degree, and combines Shannon entropy to quantitatively assess the dispersion of vehicle sources at garbage collection points, making the determination of responsible vehicles more objective and accurate, and providing a reliable basis for priority decision-making in garbage management.

[0020] 5. This application achieves proactive detection and location of illegal dumping by recognizing cargo status and matching vehicles across cameras, and dynamically adjusts the vehicle association probability based on the history of illegal dumping, so that the evidence chain can be continuously improved, thereby enhancing the adaptability and reliability of the traceability system.

[0021] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a method for tracing the source of household waste based on image recognition according to an embodiment of this application is shown. Figure 2 A schematic diagram of a Mask R-CNN model processing flow according to an embodiment of this application is shown; Figure 3 A schematic diagram of a feature map coordinate matrix transformation according to an embodiment of this application is shown; Figure 4 A schematic diagram illustrating a vehicle-garbage association result according to an embodiment of this application is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Figure 1 A flowchart illustrating a method for tracing the source of household waste based on image recognition according to an embodiment of this application is shown, as follows: Figure 1 As shown in the figure, the image recognition-based method for tracing the source of household waste in this application includes, S1, based on the monitoring equipment, collects multi-source monitoring data, processes the multi-source monitoring data, and obtains target image data; In this embodiment of the application, step S1 specifically includes: S11 uses a spatiotemporal dual-threshold dynamic frame capture algorithm to extract key frames from the video stream captured by a fixed camera. S12, a time offset compensation model is used to perform time synchronization processing on the key frame and the image data collected from the UAV inspection to obtain the target image data.

[0026] In this embodiment of the application, the sources of multi-source monitoring data are shown in Table 1 below: Table 1. Sources of Monitoring Data

[0027] In this embodiment, the video stream is first converted into an image, and a spatiotemporal dual threshold detection method is used to avoid mechanical truncation at fixed time intervals.

[0028]

[0029]

[0030]

[0031] in, As a condition for interception, For inter-frame motion energy based on optical flow, The target spatial displacement (calculated using GPS data). and These are the threshold for the amount of movement in the image area and the threshold for the amount of displacement in the target space, respectively. Image width, Image height, For pixels The velocity component on the x-axis, For pixels The velocity component on the y-axis, and Represents the coordinates at time t. and This represents the coordinates at time t-1.

[0032] In this embodiment, a time offset compensation model is used to achieve time synchronization between devices.

[0033] in, For the k-th synchronization event (such as the moment a drone flies over the top of a tower), the optimal time offset is obtained through maximum likelihood estimation. , This is a hypothetical time offset. This method of synchronizing spatial events between devices can achieve sub-second synchronization without requiring a GPS clock signal.

[0034] In this embodiment, considering that natural conditions such as backlighting or rain / fog can cause image quality degradation, the image is fused and enhanced before being input into the model.

[0035] in, For experience fusion coefficient, For adaptive enhancement operators for waste materials, To merge the pixel values ​​at coordinates (a, b) in the image, This represents the intensity value at pixel (a, b) of the original RGB image. This represents the intensity value of pixel (a, b) in a near-infrared image.

[0036] S2, construct an optimized Mask R-CNN architecture model, perform garbage identification based on the optimized Mask R-CNN architecture model and the target image data, and obtain the garbage identification result; In this embodiment of the application, step S2 specifically includes: S21, A differentiable homography transformation layer is introduced before the backbone network of the initial Mask R-CNN architecture model to obtain an updated Mask R-CNN architecture model; S22, the updated Mask R-CNN architecture model is trained, and the model is optimized based on a multi-task loss function that includes a physical constraint loss term to obtain an optimized Mask R-CNN architecture model.

[0037] In this embodiment, Mask R-CNN is an instance segmentation model whose core components include: a backbone network, typically ResNet, which extracts multi-scale features; a Region Proposal Network (RPN), which generates candidate regions of interest (RoIs); and RoI Align, which aligns candidate regions of different scales to a fixed resolution. The multi-task output heads include: a classification head, which determines the target category (garbage / non-garbage); a regression head, which optimizes the target bounding box coordinates; and a mask head, which outputs pixel-level segmentation, summarizing the results of solid waste target recognition. Figure 2 A schematic diagram of a Mask R-CNN model processing flow according to an embodiment of this application is shown. See also: Figure 2 As shown, the model compensates for wide-angle distortion through homography transformation by adding a differentiable homography layer before the backbone. This involves using matrix transformations to achieve wide-angle distortion compensation and multi-view alignment of the feature map coordinates. Figure 3 A schematic diagram of a feature map coordinate matrix transformation according to an embodiment of this application is shown. See also: Figure 3As shown, as the transformation Then g1 and h1 are learned from camera pose metadata and geometric priors are learned from the input image.

[0038] In the network model of this scheme, the Homography matrix parameter H is obtained in the following way.

[0039]

[0040] in, These are geometric features (such as edges and texture orientation) extracted from the shallow features of the backbone. To initialize with an identity matrix and ensure training stability, This is a lightweight subnetwork (a 3-layer MLP) designed. Bilinear interpolation is used to obtain the transformed feature values ​​from the output coordinates.

[0041]

[0042] in Given the value of the input feature map at integer coordinates (i,j), Floating-point coordinates after transformation The interpolation result at the location, These are floating-point coordinates after Homography transformation. The index is the coordinate of the four surrounding integers.

[0043] This operation maintains the continuity of gradient propagation. In Mask R-CNN, backpropagation is achieved through the chain rule.

[0044]

[0045] in, The gradient of the loss with respect to the Homography parameters, The gradient of the loss with respect to the output feature values, The gradient of the eigenvalues ​​with respect to the transformed x-coordinate. This represents the gradient of the transformed x-coordinate with respect to the parameters. The gradient of the eigenvalues ​​with respect to the transformed y-coordinate. This represents the gradient of the transformed y-coordinate with respect to the parameters. This represents the element in the i-th row and j-th column of the Homography matrix H. It is the loss function.

[0046] Finally, the model output is a binary mask image, with the data type being a matrix (0 / 1 values), marking the pixel-level positions of the garbage heap.

[0047] In this embodiment, a multi-task loss function is used for model training optimization.

[0048] in, The cross-entropy loss is for mask segmentation. For depth estimation, a smoothed L1 loss is used. This is the physical constraint loss (Laplace second derivative penalty term). To balance the hyperparameters (determined through grid search).

[0049] The physical constraint loss is obtained from the second-order partial derivative of the depth map.

[0050]

[0051] in, The second-order partial derivative of the depth map in the x-direction (discrete calculation using the Sobel operator). It is an L1 norm, which penalizes discontinuous deep mutations.

[0052] S3, determine the monitoring blind zone based on the multi-source monitoring data, determine whether there are vehicles entering the blind zone-related road sections or new garbage based on the multi-source monitoring data and the garbage identification results, and trace the source of the blind zone based on the judgment results. In this embodiment of the application, step S3 specifically includes: S31. Based on the preset coverage area of ​​the fixed camera and the actual coverage area of ​​the drone inspection, determine the monitoring blind spots that are not effectively monitored. S32 determines whether a vehicle has entered the blind spot-related road section based on real-time monitoring data from fixed surveillance cameras near nearby iron towers; S33, Based on the real-time images collected by the drone and the garbage identification results of the optimized Mask R-CNN architecture model, determine whether there is any new garbage; S34. If new garbage appears in the monitoring blind spot or a vehicle is detected entering the road section associated with the blind spot, the blind spot tracing process is initiated.

[0053] In this embodiment, based on the current coverage of drone cameras and tower cameras (such as the coverage radius of tower cameras), unmonitored blind spots are marked in real time. If a nearby fixed camera detects a vehicle entering a road segment (entrance / exit) associated with the blind spot, or if a drone discovers new litter in the blind spot, the blind spot tracing process is initiated. Here, a nearby fixed camera refers to a tower camera that is spatially adjacent to the blind spot and whose monitoring range covers the entrance or exit of the blind spot. Specifically, when the target area is a blind spot, vehicle activity on the road segments before and after it is traced back. The triggering conditions are as follows: (1) The nearby camera detected the vehicle entering the blind spot associated road segment (entrance / exit); (2) The drone discovered new garbage in the blind spot through image recognition.

[0054] When any condition is met, the blind spot tracing process is automatically started, and the backtracking time window is the duration set in the previous step.

[0055] In this embodiment of the application, determining whether a vehicle has entered a blind spot-related road segment based on real-time monitoring data from a fixed surveillance camera near a nearby tower specifically includes: Real-time reception and processing of video streams from nearby fixed surveillance cameras on towers that capture images of road segments in the blind spots of the surveillance system; Vehicle detection and tracking are performed on the video stream to identify and record vehicles entering the entrance area of ​​the associated road segment; The identified vehicle entry event is used as one of the triggering conditions to initiate the subsequent blind spot tracing process.

[0056] In this embodiment of the application, determining whether new spam exists based on real-time images collected by the UAV and the spam identification results of the optimized Mask R-CNN architecture model specifically includes: The system acquires real-time images collected by the drone within the monitoring blind zone at the current moment, and inputs the real-time images into the optimized Mask R-CNN architecture model. The optimized Mask R-CNN architecture model outputs the garbage mask segmentation results in the real-time image. The garbage mask segmentation result of the real-time image is compared with the garbage distribution mask in the historical reference image of the same monitoring blind area that is stored in advance; If the comparison results show that there are newly added garbage mask areas in the real-time image that do not appear in the historical reference image, then it is determined that there is newly added garbage in the monitoring blind spot.

[0057] S4. Based on the source tracing results, identify potential responsible vehicles, calculate the association probability between potential responsible vehicles and newly added waste, and determine the target responsible vehicle based on the association probability.

[0058] In this embodiment of the application, step S4 specifically includes: S41, Calculate the backtracking time window based on the garbage identification results; S42, extract all vehicles that have entered the road segment associated with the monitoring blind spot detected by the nearby camera within the retrospective time window, and perform license plate recognition and vehicle type classification on all vehicles. Based on the license plate recognition results and vehicle type classification results, filter out garbage dumping vehicles and use the garbage dumping vehicles as candidate vehicles. S43, perform entrance-exit matching analysis on the candidate vehicles. If any candidate vehicle is detected at the blind spot entrance but not at the exit within a preset time, then mark the vehicle as a potential entry vehicle. S44. Calculate the actual dwell time threshold based on the historical dumping speed threshold corresponding to the model of the candidate vehicle and the amount of newly added waste. If the actual dwell time threshold exceeds the preset dwell time threshold, then the candidate vehicle is identified as a potential responsible vehicle.

[0059] In this embodiment of the application, the formula for the dynamic backtracking duration is as follows.

[0060]

[0061] in, The garbage identification result calculated in the previous step. The average load capacity of rural domestic waste trucks, The daily average collection frequency is indicated by 2 hours, which means a minimum of 2 hours of backtracking.

[0062] In this embodiment, image processing and OCR technology are combined to identify garbage dumping vehicles based on license plate numbers, as shown in the following formula.

[0063]

[0064] in, , For Sobel horizontal / vertical gradient operators, For the input image, For element-wise multiplication, This refers to the edge features of the license plate.

[0065] The located license plate area is binarized and segmented. An open-source OCR tool is used to directly recognize the segmented characters, and the recognition results are corrected by combining the license plate format (such as province abbreviation + letter + number).

[0066] In this embodiment, potential garbage dumping vehicle types (such as tricycles, trucks, lorries, and dump trucks) are screened based on a vehicle classification algorithm. A transfer learning model based on EfficientNet-B3 is used to support vehicle classification. The model input is a surveillance image, and the output is the vehicle category label and confidence score (Softmax probability).

[0067] A classification network is constructed using a compound scaling strategy, as shown in the following formula.

[0068]

[0069] in, This is the scaling factor. It is a balancing factor.

[0070] Cross-entropy loss is used, and the loss function is as follows.

[0071]

[0072] in, One-hot encoding of the real label. To predict class probabilities, This represents the number of vehicle model categories.

[0073] In this embodiment of the application, the vehicle type or license plate number detected by the blind spot entrance camera within the statistical ΔT time window is counted; If a vehicle appears at the entrance but not at the exit (time difference ≤ 10 minutes), it is marked as "may enter blind spot".

[0074] In this embodiment of the application, the rules for calculating the dwell time are as follows.

[0075]

[0076] in, For the historical tipping speed threshold of the specified vehicle model, This is used to calculate the amount of new waste. For example, if the historical dumping speed threshold for trucks is 0.5 t / min, and V = 2 t, then the dwell time threshold is 4 minutes. If the threshold is exceeded, the vehicle is marked as a potentially responsible vehicle.

[0077] In this embodiment of the application, step S4 further includes: S45, calculate the spatial distance between any of the potential responsible vehicles and the newly added waste; S46, calculate the time difference between the timestamp of the vehicle passing through the associated road segment and the timestamp of the newly added waste identification; S47, determine the matching degree between the vehicle's driving path and the path of the newly added garbage point; S48, Calculate the association probability between the vehicle and the newly added waste based on the spatial distance, the time difference, and the path matching degree.

[0078] In this embodiment of the application, for each detected potentially responsible vehicle, its correlation with each piece of waste is calculated:

[0079] in, The shortest distance from the waste center to the location where the image of the waste transport vehicle was detected. The time frame for the images of garbage trucks For garbage image time, This is the distance attenuation coefficient. The time decay coefficient, For vehicle routes and vehicles The degree of matching, ,in, =1 indicates that the vehicle's direction of travel is the same as the vehicle's direction of travel. If the position matches, return 0; otherwise, return 0. This is for junk image time.

[0080] In this embodiment, for vehicles with a high correlation (i.e., those clearly identified as illegally dumping waste), a violation warning (such as a village committee notification or broadcast reminder) can be triggered for that vehicle. A schematic diagram illustrating the correlation between vehicles and waste is shown below. Figure 4 As shown.

[0081] In this embodiment of the application, the correlation probabilities calculated for all potential liable vehicles are sorted, and the potential liable vehicle with the highest correlation probability in the sorting results is determined as the target liable vehicle.

[0082] In this embodiment of the application, the method of tracing the source and strengthening the chain of evidence for the illegal dumping behavior of potentially responsible vehicles is also included, specifically: Based on the target image data, a dual-branch neural network model is used to identify the cargo loading status of potentially responsible vehicles. Based on the vehicle re-identification algorithm, the license plate recognition results, vehicle type classification results and color features are integrated to match the images of the same potentially responsible vehicle between different cameras, and the matching results are used to determine whether illegal dumping has occurred. Based on the driving path and timestamp information of the potentially responsible vehicle, the location of the suspected illegal dumping is estimated, and the correlation probability is corrected based on the suspected illegal dumping.

[0083] In this embodiment of the application, the vehicle source entropy value is calculated, representing the number of vehicles originating from the garbage collection point, using the following formula:

[0084] in, For the number of vehicles, This represents the association probability of each vehicle with the garbage collection point calculated in the previous step. Shannon entropy for vehicle origin.

[0085] Finally, the vehicle source index is calculated as a comprehensive evaluation index of the vehicle source at the garbage collection point.

[0086]

[0087] in, The maximum possible entropy value, This represents the number of vehicles with an association probability exceeding 0.1.

[0088] In this embodiment, it is first necessary to accurately determine whether the vehicle is fully loaded, empty, or in an unknown state. A dual-branch neural network structure is adopted as follows.

[0089] Branch 1 (Global Features): ResNet-50 extracts overall vehicle features (cargo box height, tire sinkage, etc.).

[0090] Branch 2 (Local Features): PSPNet segments the cargo compartment area and calculates the garbage filling rate.

[0091] Fusion Output: The features extracted from both branches are synthesized through a fully connected layer to output a state label and confidence level. The features extracted from the two branches are processed by the fully connected layer to finally output the vehicle cargo status label and its confidence level. A specific state determination formula is used here to ensure the accuracy of state recognition.

[0092] The state determination formula is as follows.

[0093]

[0094] in, The percentage of pixels representing garbage in the cargo container. Calculate the confidence level for model prediction.

[0095] Secondly, suspicious illegal dumping behavior is identified through vehicle re-identification and state change analysis. This part uses the DeepSORT algorithm combined with multiple features (such as license plate OCR recognition results, vehicle type classification results YOLOv8, color histogram features, etc.) to achieve vehicle re-identification across cameras. When a change in the loading status of the same vehicle is detected between different cameras (e.g., from fully loaded to empty), and the time interval meets the set criteria, it is marked as suspected illegal dumping. This step not only relies on feature similarity calculations but also needs to consider the specific conditions of vehicle status and time difference in order to more accurately capture suspicious activities. The DeepSORT algorithm is used to fuse features of license plate number (OCR), vehicle model (YOLOv8), and color histogram for vehicle re-identification. The feature similarity calculation formula is as follows.

[0096]

[0097] If the feature similarity is higher than a threshold, they are determined to be the same vehicle. Indicates the degree of license plate matching. Indicates the confidence level for vehicle type classification. Represents the cosine similarity of color histograms. , , These are the weight coefficients corresponding to each feature, which are summed to 1.

[0098] The illegal dumping behavior is determined again by the vehicle's garbage status and the time difference between the two cameras. When the vehicle is fully loaded at camera A and empty at camera B, and the time difference meets the following conditions:

[0099] It is then marked as suspicious illegal dumping.

[0100] Finally, the system infers the specific locations where illegal dumping might occur and adjusts the previously established vehicle association probabilities accordingly. By analyzing vehicle travel routes and timestamps, the most likely locations for illegal dumping can be estimated, and the risk assessment values ​​of relevant vehicles are updated based on actual illegal dumping incidents. Furthermore, for vehicles with records of illegal dumping, the system increases their association probability, strengthening the completeness and persuasiveness of the entire evidence chain.

[0101] The source of the illegal dumping location is inferred, and the vehicle association probability in step three is adjusted. For vehicles involved in illegal dumping, the association probability is increased. The formula for locating illegal dumping locations is as follows.

[0102]

[0103] in Indicates the last full-load timestamp. Represents the path point timestamp. Indicates the maximum permissible time difference. Indicates the maximum allowed distance.

[0104] For vehicles that engage in illegal dumping, the association probability is increased, and the association probability correction formula is as follows.

[0105]

[0106] in This represents the original association probability (from step three). This represents the corrected association probability. This represents the impact coefficient of illegal dumping. Indicates illegal dumping. This indicates the number of times the vehicle was involved in illegal dumping. This indicates the total number of times the vehicle has been active.

[0107] Based on the above method, this application also provides a waste tracing system based on image recognition, corresponding to the above method, the system comprising: The image acquisition module is used to collect multi-source monitoring data based on monitoring equipment, process the multi-source monitoring data, and obtain target image data. The identification module is used to construct an optimized Mask R-CNN architecture model, and to perform garbage identification based on the optimized Mask R-CNN architecture model and the target image data to obtain garbage identification results; The source tracing module is used to determine the monitoring blind zone based on the multi-source monitoring data, and to determine whether there are vehicles entering the blind zone or new garbage in the monitoring blind zone based on the multi-source monitoring data and the garbage identification results, and to perform source tracing of the blind zone based on the judgment results. The target determination module is used to identify potential responsible vehicles based on the source tracing results, calculate the association probability between potential responsible vehicles and newly added waste, and determine the target responsible vehicle based on the association probability.

[0108] Based on the same inventive concept disclosed above, this application also provides an electronic device. The electronic device of this application includes at least one processor and at least one memory electrically connected to the processor. The memory is electrically connected to the processor, wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0109] It should be noted that the electrical connection between the above-mentioned units does not necessarily mean the connection between lines. The indirect connection method can be applied to the embodiments of this application as long as it achieves the purpose of this application.

[0110] Based on the same inventive concept, this application also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the above method.

[0111] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for tracing the source of household waste based on image recognition, characterized in that, The method includes, Based on the collection of multi-source monitoring data by monitoring equipment, the multi-source monitoring data is processed to obtain target image data; An optimized Mask R-CNN architecture model is constructed, and garbage identification is performed based on the optimized Mask R-CNN architecture model and the target image data to obtain garbage identification results; Based on the multi-source monitoring data, a monitoring blind spot is determined. Based on the multi-source monitoring data and the garbage identification result, it is determined whether there are vehicles entering the road section associated with the blind spot or new garbage in the monitoring blind spot. Based on the judgment result, the source of the blind spot is traced. Based on the source tracing results, potential responsible vehicles are identified, the correlation probability between potential responsible vehicles and newly added waste is calculated, and the target responsible vehicle is determined based on the correlation probability.

2. The method according to claim 1, characterized in that, Processing multi-source monitoring data specifically includes: A spatiotemporal dual-threshold dynamic frame capture algorithm is used to extract keyframes from video streams captured by a fixed camera. A time offset compensation model is used to perform time synchronization processing on the keyframes and image data collected from UAV inspections to obtain target image data.

3. The method according to claim 1, characterized in that, Building and optimizing the Mask R-CNN architecture model specifically includes: A differentiable homography transformation layer is introduced before the backbone network of the initial Mask R-CNN architecture model to obtain an updated Mask R-CNN architecture model; The updated Mask R-CNN architecture model is trained, and the model is optimized based on a multi-task loss function that includes a physical constraint loss term to obtain an optimized Mask R-CNN architecture model.

4. The method according to claim 1, characterized in that, Blind spot tracing based on the judgment results specifically includes: Based on the preset coverage area of ​​the fixed cameras and the actual coverage area of ​​the drone inspection, identify the monitoring blind spots that are not effectively monitored; Determine whether a vehicle has entered the blind spot-related road section based on real-time monitoring data from fixed surveillance cameras near the nearby iron tower; The presence of new garbage is determined based on real-time images collected by the drone and the garbage identification results of the optimized Mask R-CNN architecture model. If new garbage is found in the monitoring blind spot or a vehicle is detected entering the road section associated with the blind spot, the blind spot tracing process will be initiated.

5. The method according to claim 1, characterized in that, Based on the source tracing results, potential liable vehicles were identified, specifically including: Calculate the backtracking time window based on the waste identification results; Extract all vehicles that entered the associated road segment of the monitoring blind zone detected by the nearby camera within the backtracking time window, and perform license plate recognition and vehicle type classification on all vehicles. Based on the license plate recognition results and vehicle type classification results, filter out garbage dumping vehicles and use the garbage dumping vehicles as candidate vehicles. An entrance-exit matching analysis is performed on the candidate vehicles. If any candidate vehicle is detected at the blind spot entrance but not at the exit within a preset time, the vehicle is marked as a potential entry vehicle. Based on the historical dumping speed threshold corresponding to the candidate vehicle model and the amount of newly added waste, the actual dwell time threshold is calculated. If the actual dwell time threshold exceeds the preset dwell time threshold, the candidate vehicle is identified as a potential responsible vehicle.

6. The method according to claim 1, characterized in that, Calculate the probability of association between potentially liable vehicles and newly generated waste, specifically including: Calculate the spatial distance between any of the aforementioned potentially liable vehicles and the newly added waste; Calculate the time difference between the timestamp of the vehicle passing through the associated road segment and the timestamp of the newly added waste identification; Determine the degree of path matching between the vehicle's travel path and the newly added garbage collection point; Based on the spatial distance, the time difference, and the path matching degree, the probability of association between the vehicle and the newly added waste is calculated.

7. The method according to claim 1, characterized in that, This also includes tracing the source and strengthening the chain of evidence for potential offending vehicles illegally dumping, specifically including: Based on the target image data, a dual-branch neural network model is used to identify the cargo loading status of potentially responsible vehicles. Based on the vehicle re-identification algorithm, the license plate recognition results, vehicle type classification results and color features are integrated to match the images of the same potentially responsible vehicle between different cameras, and the matching results are used to determine whether illegal dumping has occurred. Based on the driving path and timestamp information of the potentially responsible vehicle, the location of the suspected illegal dumping is estimated, and the correlation probability is corrected based on the suspected illegal dumping.

8. A waste tracing system based on image recognition, characterized in that, The system includes: The image acquisition module is used to collect multi-source monitoring data based on monitoring equipment, process the multi-source monitoring data, and obtain target image data; The identification module is used to construct an optimized Mask R-CNN architecture model, and to perform garbage identification based on the optimized Mask R-CNN architecture model and the target image data to obtain garbage identification results; The source tracing module is used to determine the monitoring blind zone based on the multi-source monitoring data, and to determine whether there are vehicles entering the blind zone or new garbage in the monitoring blind zone based on the multi-source monitoring data and the garbage identification results, and to perform source tracing of the blind zone based on the judgment results. The target determination module is used to identify potential responsible vehicles based on the source tracing results, calculate the association probability between potential responsible vehicles and newly added waste, and determine the target responsible vehicle based on the association probability.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in a memory, it implements the steps of the image recognition-based waste tracing method according to any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of the image recognition-based waste tracing method according to any one of claims 1-7.

Images