Unmanned garbage collection method based on multi-sensor fusion and edge intelligence

Abstract

The application discloses an unmanned garbage collection method based on multi-sensor fusion and edge intelligence, and belongs to the technical field of garbage treatment. The method comprises the following steps: collecting environmental visual information by using a high-definition camera array, constructing a three-dimensional point cloud map by using a laser radar and performing obstacle ranging, and fusing data collected by each sensor; performing real-time inference on multi-source data by using a deep learning model, and identifying the category, material characteristics, position coordinates and confidence of road surface garbage; integrating perception data and garbage identification results, overall planning global task scheduling, and generating a globally optimal cleaning path; a flexible grabbing end effector automatically switches the grabbing mode according to the garbage category; and a garbage bin automatic door is automatically opened when the mechanical arm is detected to be close. The application has the integrated functions of autonomous navigation, accurate target identification, flexible grabbing and garbage classification, and significantly improves the automation level and operation efficiency of urban environmental sanitation operation.

Description

Technical Field

[0001] This invention relates to the field of waste treatment technology, and in particular to an unmanned waste collection method based on multi-sensor fusion and edge intelligence. Background Technology

[0002] With the acceleration of urbanization, the amount of urban waste is growing exponentially. Traditional sanitation operations mainly rely on manually driven sanitation vehicles or fixed-point disposal, facing severe challenges such as high labor intensity, high operating costs, and uneven coverage. Although some driverless sweepers have appeared on the market, existing technical solutions mostly use a single-sensor mode, and their environmental perception capabilities are easily affected by changes in lighting, rain, fog, and occlusion, resulting in poor robustness of the system in complex sanitation scenarios. In addition, existing systems often rely on basic algorithms to identify waste targets, making it difficult to accurately classify waste with varying shapes and materials, often resulting in missed detections or misidentifications. The level of intelligence is insufficient to meet the needs of refined cleaning.

[0003] At the decision-making and execution levels, traditional driverless garbage trucks typically operate along pre-set fixed routes, lacking the ability to dynamically adjust to garbage distribution density. This results in frequent empty runs in sparsely populated areas and low efficiency in densely populated areas. Their path planning strategies exhibit significant rigidity, making it difficult to respond in real-time to obstacles or task changes in dynamic traffic environments. Furthermore, existing collection and execution mechanisms are mostly rigid gripping devices, lacking adaptive force adjustment mechanisms. This can easily cause garbage breakage or spillage during operation, leading to secondary environmental pollution. Moreover, the lack of closed-loop coordination between system modules severely limits the automation level and resource utilization efficiency of urban sanitation operations. Summary of the Invention

[0004] Purpose of the invention: To address the above problems, the purpose of this invention is to provide an unmanned waste collection method based on multi-sensor fusion and edge intelligence.

[0005] Technical solution: The unmanned waste collection method based on multi-sensor fusion and edge intelligence of the present invention includes the following steps:

[0006] Step 1: Use a high-definition camera array to collect environmental visual information, use lidar to construct a 3D point cloud map and measure obstacle distance, use ultrasonic sensors for near-range obstacle avoidance protection, use infrared sensors to assist in detecting the heat source characteristics of garbage, and fuse the data collected by each sensor to obtain multi-source data.

[0007] Step 2: Use a deep learning model to perform real-time reasoning on multi-source data to identify the type, material properties, location coordinates, and confidence level of road debris;

[0008] Step 3: Integrate the sensing data and waste identification results, coordinate the overall task scheduling, and generate the globally optimal cleaning path;

[0009] Step 4: The flexible gripping end effector automatically switches the gripping mode according to the type of waste; the automatic door of the waste bin opens automatically when it detects the robotic arm approaching and closes automatically after collection is completed.

[0010] Preferably, step 1 includes:

[0011] Two-dimensional color image information of the working environment is acquired by a high-definition camera array, and the two-dimensional color image information is processed by an image enhancement algorithm to obtain visual image features;

[0012] A 3D point cloud map is constructed by emitting a laser beam and receiving the reflected signal using a lidar system. This allows for the localization and distance measurement of obstacles. The spatial distribution of obstacles is quantified using a point cloud density function, which is then used to obtain obstacle features. The point cloud density function is expressed as follows:

[0013] ,

[0014] In the formula, The spatial coordinates of the waste distribution This represents the waste distribution density value. The total number of data points. For the first The location coordinates of the garbage target An adaptive bandwidth parameter related to garbage size;

[0015] The infrared radiation characteristics of objects are detected by infrared sensors to distinguish between biological and non-biological waste; heat source characteristics are generated by aligning thermal imaging and visual images.

[0016] The data collected by each sensor are aligned in time and space to eliminate timing errors, and real-time calibration is performed based on Kalman filtering. The processed data are then adaptively weighted and fused, as follows:

[0017] ,

[0018] In the formula, This represents the fused multi-source data. Visual image features acquired from a high-definition camera array, The heat source characteristics are collected by the infrared sensor. Obstacle features derived from lidar point clouds; The weights are dynamically adjusted based on ambient light intensity. As an occlusion compensation factor;

[0019] The fused multi-source data is then compressed and sensed.

[0020] Preferably, step 2 includes:

[0021] Multi-source data is input into the feature extraction layer of a deep learning model, and shallow detail features of the input image are extracted layer by layer through convolutional layers. and deep semantic features ;

[0022] The feature pyramid is used to fuse deep semantic features with shallow detail features to obtain fused features; the fusion formula is as follows:

[0023] ,

[0024] In the formula, This is a convolutional layer operation used to smooth the fused features. For upsampling operation, This is an element-wise addition operation;

[0025] Based on fusion features An adaptive attention mechanism is introduced to analyze spatial and channel importance, generate a weight mask, and then generate optimized features. The calculation formula is:

[0026] ,

[0027] In the formula, These are weighting coefficients, dynamically adjusted based on the entropy value of the feature map;

[0028] feature map The input is fed into the region proposal module, which slides preset anchor boxes on the feature map and uses a convolutional network to perform binary classification and initial bounding box adjustment on each anchor box, outputting a series of preliminary coordinates of candidate boxes and their confidence scores for being garbage targets;

[0029] feature map Input the data into the classification regression head, and output the bounding box coordinates, class probability, and final confidence score of the garbage target.

[0030] Preferably, step 2 further includes:

[0031] The garbage target identification results output by the deep learning model are compared with multi-source data. Together with observational evidence, Bayesian inference is performed to calculate the posterior probability of the existence of the garbage target. The calculation formula is as follows:

[0032] ,

[0033] In the formula, With the goal, For multi-source data ; Let be the likelihood probability. As a priori probability, it is dynamically updated using historical job data. This is the normalization constant.

[0034] Preferably, during the training process, the multi-task loss function of the deep learning model includes classification loss and regression loss, expressed as:

[0035] ,

[0036] In the formula, For the total loss, This is for classification loss, used to ensure the reliability of waste category identification; The regression loss is used to achieve pixel-level precise localization of garbage boundaries; These are the balancing weighting coefficients.

[0037] Preferably, the deep learning model is built based on the YOLO algorithm. During the model training phase, the backpropagation algorithm combined with gradient descent strategy is used to optimize the multi-task loss function. The process includes:

[0038] Calculate the total loss in batches based on the training data. Gradients of deep learning model parameters The gradient is propagated through the backpropagation algorithm; the gradient calculation formula is:

[0039] ,

[0040] In the formula, Represents the weight parameters of a deep learning model.

[0041] Losses according to classification and regression loss The relative size of the dynamic adjustment of the balance weight coefficient ;

[0042] An exponentially decaying learning rate strategy is adopted, with an initial learning rate of... The decay rate per training cycle is The learning rate update formula is: Simultaneously, a gradient clipping mechanism is introduced to limit the gradient norm from exceeding a threshold. The formula is:

[0043] ,

[0044] Based on the recognition results output by the deep learning model, predefined waste classification rules are applied to filter out detection results with confidence scores below the threshold, determine the disposal category of the waste based on the maximum category probability, and output the classification label.

[0045] Preferably, step 3 includes:

[0046] Based on real-time environmental data collected by multiple sensors, starting from the current position of the unmanned garbage truck, an improved A* search algorithm is used to dynamically search for intermediate nodes and generate a globally optimal cleaning path; this is achieved by minimizing the cost function. The cost function in the A* search algorithm, which dynamically determines intermediate nodes, is expressed as:

[0047] ,

[0048] In the formula, From the starting point to the node The actual path cost For nodes Heuristic value of Euclidean distance to the destination. This is the adjustment coefficient; For nodes Garbage density weight at each location.

[0049] Preferably, step 4 includes:

[0050] The gripping force of the flexible gripper end effector is adjusted by a PID controller, and the gripping force optimization function is defined as follows:

[0051] ,

[0052] In the formula, To set the real-time gripping force value, For gripping force error, when If the safety threshold is exceeded continuously, a grabbing stop command is triggered, and the grabbing trajectory is replanned; , , These are the proportional, integral, and derivative control coefficients.

[0053] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0054] 1. This invention realizes the real-time detection and identification of garbage targets and dynamic path planning functions. It can autonomously adjust the operation strategy and cleaning path according to the garbage distribution density and traffic environment, forming a closed-loop architecture of perception, decision-making and execution. This reduces the need for manual inspection and intervention, lowers operating costs, and improves the continuity of operation and cleaning coverage.

[0055] 2. This invention uses a YOLO-based deep learning algorithm for waste detection, which can identify waste classification in real time, accurately locate coordinates, and generate the optimal grasping strategy. This intelligent identification method significantly improves the accuracy of waste detection and classification, effectively solving the problems of high missed detection rate and rough classification in traditional operations, thereby improving waste collection efficiency and resource utilization rate.

[0056] 3. Compared with traditional rigid gripping devices, this invention uses a flexible gripping end effector and an automatic garbage bin door for intelligent collaborative control. The system can automatically switch gripping modes when different types of garbage are identified to avoid damage. After collection, the bin door is closed in time to prevent odor spread and garbage spillage. At the same time, the coordinated operation of low-flow pressure holding and high-density area repeated cleaning modes significantly reduces energy consumption and secondary pollution, achieving the dual goals of precise operation and environmentally friendly operation. Attached Figure Description

[0057] Figure 1 This is a flowchart of the present invention;

[0058] Figure 2 This is a flowchart of the YOLO algorithm optimization process of the present invention;

[0059] Figure 3 This is a comparison chart of the garbage detection accuracy of the present invention;

[0060] Figure 4 This is a comparison chart of the 24-hour operating efficiency of the present invention;

[0061] Figure 5 A diagram showing the cost structure of a traditional system;

[0062] Figure 6 This is a cost breakdown diagram of the intelligent system of the present invention;

[0063] Figure 7 This is a cost-saving analysis diagram of the intelligent system of the present invention;

[0064] Figure 8 This is a comparison chart of path planning optimization according to the present invention. Detailed Implementation

[0065] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the embodiments of the present invention, and not all structures.

[0066] In the following description, specific details such as target system architecture and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0067] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0068] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0069] Furthermore, in the description of this application and the appended claims, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0070] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include the target features, structures, or characteristics described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0071] Combination Figure 1 As shown in this embodiment, the unmanned waste collection method based on multi-sensor fusion and edge intelligence includes the following steps:

[0072] Step 1: Use a high-definition camera array to collect environmental visual information, use lidar to construct a 3D point cloud map and measure obstacle distances, use ultrasonic sensors for near-range obstacle avoidance protection, and use infrared sensors to assist in detecting the heat source characteristics of garbage. The data collected by each sensor are fused to obtain multi-source data.

[0073] Further, step 1 includes:

[0074] Two-dimensional color image information of the working environment is collected by a high-definition camera array. It is mainly used for the preliminary identification of garbage targets, road marking detection, traffic signal recognition and dynamic monitoring of pedestrians and vehicles. Through the collaborative work of multiple cameras, visual coverage of a wide area environment is achieved. Image enhancement algorithms are used to process the two-dimensional color image information, including histogram equalization and adaptive gamma correction, to reduce recognition errors at night or in strong light and obtain visual image features.

[0075] By emitting a laser beam and receiving the reflected signal using a lidar system, a 3D point cloud map is constructed to obtain the location and distance measurement of obstacles. The spatial distribution of obstacles is quantified using a point cloud density function to obtain obstacle features, providing reliable spatial geometric information support for vehicle navigation and obstacle avoidance. The point cloud density function is expressed as:

[0076] ,

[0077] In the formula, The spatial coordinates of the waste distribution This represents the waste distribution density value. The total number of data points. For the first The location coordinates of the garbage target An adaptive bandwidth parameter related to the size of the waste is dynamically adjusted based on the sparsity of the point cloud to optimize the detection of occluded areas;

[0078] Ultrasonic sensors utilize the principle of ultrasonic ranging to achieve accurate obstacle detection and obstacle avoidance at close range, compensating for the blind spots of vision and lidar in extremely close-range perception, preventing collisions during robotic arm grasping operations, and ensuring operational safety; they also compensate for ranging errors of moving objects through the Doppler effect, improving reliability under obstructed conditions.

[0079] By detecting the infrared radiation characteristics of objects using infrared sensors, the thermal characteristics of waste can be identified, enabling waste target detection at night or in low-light conditions, and distinguishing between biological and non-biological waste. A thermal imaging and visual image alignment algorithm is used to generate thermal source features to enhance the identification of waste with diverse morphologies.

[0080] The data collected by each sensor are aligned in time and space to eliminate timing errors, and real-time calibration is performed based on Kalman filtering. The processed data are then adaptively weighted and fused, as follows:

[0081] ,

[0082] In the formula, This represents the fused multi-source data. Visual image features acquired from a high-definition camera array, The heat source characteristics are collected by the infrared sensor. Obstacle features derived from lidar point clouds; The weights are dynamically adjusted based on ambient light intensity. As an occlusion compensation factor;

[0083] The fused multi-source data is then compressed and sensed.

[0084] In one example, weights The ambient light intensity is dynamically adjusted; for example, when the light intensity is less than 50 lux, it can be set... Emphasis is placed on infrared data; settings can be configured when illumination exceeds 1000 lux. Emphasis is placed on visual data. Occlusion compensation factor. The obstruction density is calculated based on the output of the ultrasonic sensor, using the following formula:

[0085] ,

[0086] in, To obscure the distance, the distance is directly obtained using an ultrasonic sensor based on the principle of ultrasonic ranging.

[0087] Step 2: Use a deep learning model to perform real-time reasoning on multi-source data to identify the type, material characteristics, location coordinates, and confidence level of road debris.

[0088] In one example, the deep learning model includes a feature extraction layer, a region proposal module, and a classification and regression head.

[0089] Combination Figure 2 As shown, step 2 further includes:

[0090] Multi-source data is input into the feature extraction layer of a deep learning model, and shallow detail features of the input image are extracted layer by layer through convolutional layers. and deep semantic features Among them, shallow detail features Shallow features originate from early convolutional layers (such as layers 1-3). These layers are close to the input and have a small receptive field, mainly capturing local details of the image, such as edges, textures, and color variations. Due to their high resolution, shallow features retain rich spatial details but have weak semantic information. Deep semantic features come from deep convolutional layers (such as layers 4-6). These layers, through multiple pooling and convolution operations, increase the receptive field and can capture more global, semantic-level information, such as object shape and category, but have lower resolution and lose some detail.

[0091] To address the significant scale diversity of waste targets in complex urban sanitation scenarios (e.g., the coexistence of small cigarette butts and large cardboard boxes), a feature pyramid network structure is introduced. This structure fuses deep semantic features with shallow detail features to obtain fused features. The fused feature map effectively enhances the detection capability of small-scale waste targets and improves the model's robustness to scale changes. The fusion formula is as follows:

[0092] ,

[0093] In the formula, This is a convolutional layer operation used to smooth the fused features. For upsampling operation, This is an element-wise addition operation;

[0094] To address the false detection problem caused by similarly shaped litter and non-litter interference, a feature fusion approach is used. An adaptive attention mechanism is introduced to analyze spatial and channel importance. The weight mask is dynamically adjusted based on the feature map entropy value to ensure that key visual features of the material dominate the fusion process. This significantly improves the recognition and recall rate for waste with varied shapes and materials. A weight mask is then generated to produce optimized features. The calculation formula is:

[0095] ,

[0096] In the formula, The weighting coefficients are dynamically adjusted based on the entropy value of the feature map, where the entropy value is... ,in The entropy value represents the normalized distribution probability of the feature map; a higher entropy value indicates a more complex feature or a greater amount of information. The attention network calculates the entropy value in real time. For example, when When, set By emphasizing high-level semantic features and conversely emphasizing low-level detailed features, this mechanism ensures that key features dominate in the fusion process, significantly improving the recall and accuracy of spam object detection.

[0097] feature map The input is fed into the region proposal module, which generates candidate target regions based on the anchor point mechanism. By sliding preset anchor boxes on the feature map, and using a convolutional network to perform binary classification and initial bounding box adjustment on each anchor box, a series of candidate boxes are output with their preliminary coordinates and confidence scores for being garbage targets, ensuring a high recall rate.

[0098] feature map The data is input into the classification regression head, which outputs the bounding box coordinates, class probability, and final confidence score of the garbage target, forming a complete garbage target recognition result.

[0099] Furthermore, step 2 also includes:

[0100] The garbage target identification results output by the deep learning model are compared with multi-source data. Together with observational evidence, Bayesian inference is performed to calculate the posterior probability of the existence of the garbage target. The calculation formula is as follows:

[0101] ,

[0102] In the formula, With the goal, For multi-source data ; Let be the likelihood probability. As a priori probability, it is dynamically updated using historical job data. This is the normalization constant.

[0103] The more reliable probability of garbage existence, as the output of Bayesian inference, is the garbage density weight in the path planning cost function. The direct basis is precisely based on the fact that... Aggregate calculation This allows for dynamic path adjustment, prioritizing the cleaning of high-probability areas, thereby achieving a closed loop from reliable perception to intelligent decision-making.

[0104] Furthermore, during the training process, the multi-task loss function of a deep learning model includes classification loss and regression loss, expressed as:

[0105] ,

[0106] In the formula, For the total loss, This is for classification loss, used to ensure the reliability of waste category identification; The regression loss is used to achieve pixel-level precise localization of garbage boundaries; To balance the weighting coefficients, they are usually set as follows: =1.0.

[0107] The expression for classification loss is:

[0108] ,

[0109] in, To predict probabilities, This is a real label.

[0110] The expression for regression loss is:

[0111] ,

[0112] in, To predict coordinate offset, This is the actual offset; The function is defined as: when Time output Otherwise output To enhance regression stability.

[0113] Furthermore, the deep learning model is built based on the YOLO algorithm. During the model training phase, the backpropagation algorithm combined with gradient descent strategy is used to optimize the multi-task loss function. The process includes:

[0114] Calculate the total loss in batches based on the training data. Gradients of deep learning model parameters The gradient is propagated through the backpropagation algorithm; the gradient calculation formula is:

[0115] ,

[0116] In the formula, Represents the weight parameters of a deep learning model;

[0117] Losses according to classification and regression loss The relative size of the dynamic adjustment of the balance weight coefficient The adjustment rule can be: when Time (indicating the dominant category task), settings To enhance regression loss; when When the regression task is dominant, λ=2.0 is set to strengthen the classification loss; this adjustment ensures that the loss components are optimized in a balanced manner, improving the model's adaptability to changes in lighting and occlusion conditions;

[0118] An exponentially decaying learning rate strategy is adopted, with an initial learning rate of... The decay rate per training cycle is The learning rate update formula is: Simultaneously, a gradient clipping mechanism is introduced to limit the gradient norm from exceeding a threshold. The formula is:

[0119] ,

[0120] This optimization ensures the loss function It converges smoothly to the global minimum.

[0121] The training data for the model is mainly based on multi-source sensor data (including visual images, LiDAR point clouds, and infrared thermal imaging) collected synchronously in actual urban sanitation scenarios. The data is finely labeled (bounding boxes and category labels). At the same time, large-scale data augmentation and the introduction of negative samples and interference samples are used to form a diversified dataset that can cover complex environmental changes and improve the model's generalization and robustness.

[0122] Based on the recognition results output by the deep learning model, predefined waste classification rules are applied to filter out detection results with confidence scores below the threshold, determine the disposal category of the waste based on the maximum category probability, and output the classification label.

[0123] In one example, the predefined waste sorting rules are based on the physical characteristics and composition of waste, classifying waste into recyclables, hazardous waste, kitchen waste and other waste. For example, the detection results with confidence scores below the threshold (e.g., 0.7) are first filtered out, and then the disposal category of the waste is determined according to the maximum category probability, and the classification label is output.

[0124] Step 3: Integrate the sensing data and waste identification results, coordinate the overall task scheduling, and generate the globally optimal cleaning path.

[0125] Furthermore, step 3 includes:

[0126] Based on real-time environmental data collected by multiple sensors, starting from the current position of the unmanned garbage truck, an improved A* search algorithm is used to dynamically search for intermediate nodes and generate a globally optimal cleaning path; this is achieved by minimizing the cost function. The cost function in the A* search algorithm, which dynamically determines intermediate nodes, is expressed as:

[0127] ,

[0128] In the formula, From the starting point to the node The actual path cost For nodes Heuristic value of Euclidean distance to the destination. This is the adjustment coefficient; For nodes Garbage density weight at each location.

[0129] Step 4: The flexible gripping end effector automatically switches the gripping mode according to the type of waste; the automatic door of the waste bin opens automatically when it detects the robotic arm approaching and closes automatically after collection is completed.

[0130] Furthermore, step 4 includes:

[0131] The gripping force of the flexible gripper end effector is adjusted by a PID controller, and the gripping force optimization function is defined as follows:

[0132] ,

[0133] In the formula, To set the real-time gripping force value, For gripping force error, when When the safety threshold is continuously exceeded (e.g., | |>3N), triggering a grab abort command and replanning the grab trajectory; , , These are the proportional, integral, and derivative control coefficients. The ideal grasping force mapping table corresponding to the type of garbage is determined, as shown in Table 1. Real-time data is collected via distributed pressure sensors. A PID controller enables smooth adjustment of the gripping force, preventing sudden force changes that could damage the waste.

[0134] Table 1 Grasping Force Mapping Table

[0135]

[0136] The automatic door of the garbage bin uses infrared and ultrasonic sensors to detect the approach signal of the robotic arm, enabling intelligent triggering of door control. Based on the principle of multi-sensor data fusion, the Euclidean distance between the end position of the robotic arm and the reference position of the garbage bin door is dynamically calculated. When this distance is less than or equal to a preset distance threshold (e.g., 0.5 meters), an opening command is automatically sent. After collection is completed, a closing action is triggered by a timer or a robotic arm position feedback signal to prevent odor diffusion and garbage spillage. Door control status and behavior decision coordination: During high-frequency operations... Automatic door opening and closing cycle shortened to 3 seconds, improving grasping efficiency; during low-load periods Enable energy-saving mode to extend the shutdown time to 10 seconds and reduce energy consumption.

[0137] To further demonstrate the effectiveness and superiority of the waste collection method described in this invention, a comparison of waste detection accuracy between this invention and traditional single-sensor systems (such as pure vision systems) was conducted. The results are as follows: Figure 3 As shown, experimental data demonstrate that the intelligent system's robustness in complex scenarios far surpasses that of traditional vision systems: in small-target waste detection, the accuracy of the intelligent system improved from 0.45 to 0.85; in occluded and densely packed waste scenarios, the accuracy improved from approximately 0.35 and 0.40 to over 0.78 and 0.82, respectively; and the most significant improvement was observed under extremely low light conditions at night, with the accuracy jumping dramatically from 0.25 to 0.82. The experiment collected data in diverse sanitation scenarios (day-night transition, occluded environments, and varying waste forms), and statistically analyzed the recognition accuracy of the two systems for categories such as recyclables and hazardous waste.

[0138] Figure 4 Simulating 24-hour continuous urban sanitation operations, this invention compares the efficiency of traditional fixed-route garbage trucks with the dynamic path planning system of this invention. The invention utilizes a garbage density field (…). The system optimizes routes in real time and records the amount of garbage collected, empty mileage, and task completion rate per unit time. Experiments covered peak and off-peak periods to verify the adaptability of the behavioral decision-making module. The efficiency curve comparison shows that the traditional manual mode is significantly affected by fatigue and environmental factors, reaching a peak of 0.80 at midday and then dropping to a trough of 0.40 at night. In contrast, the intelligent unmanned collection system maintains extremely high stability throughout the day, with efficiency consistently between 0.85 and 0.95, eliminating operational peaks and troughs.

[0139] Figure 5 This study statistically analyzes the cost structure of traditional manual cleaning methods, including labor costs, fuel consumption, equipment maintenance, and management expenses. Data is sourced from annual operational reports of sanitation departments in multiple cities, and the breakdown of each cost component is presented after normalization.

[0140] Figure 6 Based on the actual deployment data of this invention, its cost composition (such as sensor procurement, edge computing equipment, power consumption and maintenance costs) is quantified.

[0141] Figure 7 Calculate the unit cost savings rate of this invention compared to the traditional model, combined with Figure 7 , Figure 8 The data analyzes the total savings over a 5-year period. The chart quantifies the changes in various costs, with labor costs decreasing by approximately 75%. Although maintenance costs increased by about 200% due to hardware and software maintenance, the overall operating costs still showed a significant saving trend over the 5-year period thanks to the large reduction in the number of employees.

[0142] Figure 8 pass A comparison of the heatmaps of the regions shows that traditional fixed paths only travel along the periphery of the region without reaching the high-density waste areas inside; while the improved A* algorithm of this invention is based on waste density weights. Dynamically adjusted, the path accurately covers high-density "garbage hotspots" such as coordinates (3,3) and (7,7), effectively reducing invalid traversals.

Claims

1. A method for unmanned waste collection based on multi-sensor fusion and edge intelligence, characterized in that, Includes the following steps: Step 1: Use a high-definition camera array to collect environmental visual information, use lidar to construct a 3D point cloud map and measure obstacle distance, use ultrasonic sensors for near-range obstacle avoidance protection, use infrared sensors to assist in detecting the heat source characteristics of garbage, and fuse the data collected by each sensor to obtain multi-source data. Step 2: Use a deep learning model to perform real-time reasoning on multi-source data to identify the type, material properties, location coordinates, and confidence level of road debris; Step 3: Integrate the sensing data and waste identification results, coordinate the overall task scheduling, and generate the globally optimal cleaning path; Step 4: The flexible gripping end effector automatically switches the gripping mode according to the type of waste; the automatic door of the waste bin opens automatically when it detects the robotic arm approaching and closes automatically after collection is completed.

2. The unmanned waste collection method based on multi-sensor fusion and edge intelligence according to claim 1, characterized in that, Step 1 includes: Two-dimensional color image information of the working environment is acquired by a high-definition camera array, and the two-dimensional color image information is processed by an image enhancement algorithm to obtain visual image features; A 3D point cloud map is constructed by emitting a laser beam and receiving the reflected signal using a lidar system. This allows for the localization and distance measurement of obstacles. The spatial distribution of obstacles is quantified using a point cloud density function, which is then used to obtain obstacle features. The point cloud density function is expressed as follows: ， In the formula, The spatial coordinates of the waste distribution This represents the waste distribution density value. The total number of data points. For the first The location coordinates of the garbage target An adaptive bandwidth parameter related to garbage size; The infrared radiation characteristics of objects are detected by infrared sensors to distinguish between biological and non-biological waste; heat source characteristics are generated by aligning thermal imaging and visual images. The data collected by each sensor are aligned in time and space to eliminate timing errors, and real-time calibration is performed based on Kalman filtering. The processed data are then adaptively weighted and fused, as follows: ， In the formula, This represents the fused multi-source data. Visual image features acquired from a high-definition camera array, The heat source characteristics are collected by the infrared sensor. Obstacle features derived from lidar point clouds; The weights are dynamically adjusted based on ambient light intensity. As an occlusion compensation factor; The fused multi-source data is then compressed and sensed.

3. The unmanned waste collection method based on multi-sensor fusion and edge intelligence according to claim 2, characterized in that, Step 2 includes: Multi-source data is input into the feature extraction layer of a deep learning model, and shallow detail features of the input image are extracted layer by layer through convolutional layers. and deep semantic features ; The feature pyramid is used to fuse deep semantic features with shallow detail features to obtain fused features; the fusion formula is as follows: ， In the formula, This is a convolutional layer operation used to smooth the fused features. For upsampling operation, This is an element-wise addition operation; Based on fusion features An adaptive attention mechanism is introduced to analyze spatial and channel importance, generate a weight mask, and then generate optimized features. The calculation formula is: ， In the formula, These are weighting coefficients, dynamically adjusted based on the entropy value of the feature map; feature map The input is fed into the region proposal module, which slides preset anchor boxes on the feature map and uses a convolutional network to perform binary classification and initial bounding box adjustment on each anchor box, outputting a series of preliminary coordinates of candidate boxes and their confidence scores for being garbage targets; feature map Input the data into the classification regression head, and output the bounding box coordinates, class probability, and final confidence score of the garbage target.

4. The unmanned waste collection method based on multi-sensor fusion and edge intelligence according to claim 3, characterized in that, Step 2 also includes: The garbage target identification results output by the deep learning model are compared with multi-source data. Together with observational evidence, Bayesian inference is performed to calculate the posterior probability of the existence of the garbage target. The calculation formula is as follows: ， In the formula, With the goal, For multi-source data ; Let be the likelihood probability. As a priori probability, it is dynamically updated using historical job data. This is the normalization constant.

5. The unmanned waste collection method based on multi-sensor fusion and edge intelligence according to any one of claims 1 to 4, characterized in that, During the training process, the multi-task loss function of a deep learning model includes classification loss and regression loss, expressed as follows: ， In the formula, For the total loss, This is for classification loss, used to ensure the reliability of waste category identification; The regression loss is used to achieve pixel-level precise localization of garbage boundaries; These are the balancing weighting coefficients.

6. The unmanned waste collection method based on multi-sensor fusion and edge intelligence according to claim 5, characterized in that, The deep learning model is built based on the YOLO algorithm. During the model training phase, the backpropagation algorithm combined with gradient descent is used to optimize the multi-task loss function. The process includes: Calculate the total loss in batches based on the training data. Gradients of deep learning model parameters The gradient is propagated through the backpropagation algorithm; the gradient calculation formula is: ， In the formula, Represents the weight parameters of a deep learning model. Losses according to classification and regression loss The relative size of the dynamic adjustment of the balance weight coefficient ; An exponentially decaying learning rate strategy is adopted, with an initial learning rate of... The decay rate per training cycle is The learning rate update formula is: Simultaneously, a gradient clipping mechanism is introduced to limit the gradient norm from exceeding a threshold. The formula is: ， Based on the recognition results output by the deep learning model, predefined waste classification rules are applied to filter out detection results with confidence scores below the threshold, determine the disposal category of the waste based on the maximum category probability, and output the classification label.

7. The unmanned waste collection method based on multi-sensor fusion and edge intelligence according to any one of claims 1 to 4, characterized in that, Step 3 includes: Based on real-time environmental data collected by multiple sensors, starting from the current position of the unmanned garbage truck, an improved A* search algorithm is used to dynamically search for intermediate nodes and generate a globally optimal cleaning path; this is achieved by minimizing the cost function. The cost function in the A* search algorithm, which dynamically determines intermediate nodes, is expressed as: ， In the formula, From the starting point to the node The actual path cost For nodes Heuristic value of Euclidean distance to the destination. This is the adjustment coefficient; For nodes Garbage density weight at each location.

8. The unmanned waste collection method based on multi-sensor fusion and edge intelligence according to claim 1, characterized in that, Step 4 includes: The gripping force of the flexible gripper end effector is adjusted by a PID controller, and the gripping force optimization function is defined as follows: ， In the formula, To set the real-time gripping force value, For gripping force error, when If the safety threshold is exceeded continuously, a grabbing stop command is triggered, and the grabbing trajectory is replanned; , , These are the proportional, integral, and derivative control coefficients.

Images