Intelligent cruise garbage picking and classifying vehicle, method, equipment and medium

Abstract

The invention provides an intelligent cruise garbage picking and classifying vehicle, method, equipment and medium to solve the problem that in the prior art, a laser radar can only obtain two-dimensional point cloud data and cannot recognize the characteristics of garbage materials, the size and the like. The garbage picking and classifying vehicle comprises an upper computer, a lower computer, a laser radar, a depth camera, a speedometer, an IMU (Inertial Measurement Unit), a mechanical arm and a wireless communication port. Through heterogeneous data cooperation of the laser radar and the depth camera, the limitation of a single sensor is effectively eliminated. The laser radar provides high-precision two-dimensional space information, and the depth camera supplements three-dimensional geometric and visual semantic features, so that the comprehensive perception capability of garbage materials, shapes and volumes is remarkably enhanced. According to the fusion, the problem of misjudgment of garbage materials in a traditional scheme is solved, the self-adaptive grabbing strategy of the mechanical arm is supported, and the grabbing failure risk caused by environment complexity is reduced. And a dual-bus communication redundancy design is adopted, so that stable transmission of key signals is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and particularly relates to an intelligent cruise garbage pickup and classification vehicle, method, device and medium. Background Art

[0002] The intelligent cruise garbage pickup and classification vehicle realizes the full-automatic pickup and classification management of garbage in a closed area (such as a park, a square, etc.) through multi-sensor collaborative perception, autonomous navigation and path planning, intelligent grasping of the robotic arm and garbage automatic classification module, and provides data support for environmental sanitation management.

[0003] The existing intelligent cruise garbage pickup and classification vehicle with a single lidar has the following problems: The lidar can only obtain two-dimensional point cloud data and cannot identify features such as the material and volume of garbage, resulting in a high failure rate of the robotic arm to grasp (for example, plastic bags and cardboard boxes are easily confused due to similar reflectivity). This requires relying on manual pre-labeling of garbage categories, and an additional image processing unit needs to be deployed for the classification function, resulting in a hardware cost increase of more than 30% (such as adding a Jetson series host computer). Summary of the Invention

[0004] The present invention provides an intelligent cruise garbage pickup and classification vehicle, method, device and medium to solve the problem in the prior art that the lidar can only obtain two-dimensional point cloud data and cannot identify features such as the material and volume of garbage, resulting in a high failure rate of the robotic arm to grasp.

[0005] The technical solution adopted by the present invention is as follows:

[0006] An intelligent cruise garbage pickup and classification vehicle, comprising:

[0007] A host computer, the host computer runs ROS based on the Jetson Orin Nano embedded platform; The ROS deploys a mapping algorithm, a garbage recognition and classification model YOLOv8, an inner spiral coverage algorithm, a search algorithm, and a DWA algorithm;

[0008] A slave computer, the slave computer is connected to the host computer;

[0009] A lidar, the lidar is connected to the host computer;

[0010] A depth camera, the depth camera is connected to the host computer;

[0011] An odometer, the odometer is connected to the slave computer, the odometer is used to collect the moving speed of the garbage pickup and classification vehicle body, the odometer transmits the moving speed of the garbage pickup and classification vehicle body to the slave computer, and the slave computer processes the moving speed of the garbage pickup and classification vehicle body to obtain the XY-axis coordinates P of the garbage pickup and classification vehicle body in the three-dimensional coordinate system of the global map XYand the angle θ around the Z axis 里程计 ;

[0012] Among them, in the three-dimensional coordinate system, the axis extending along the front of the vehicle is the X axis, the axis perpendicular to the front of the vehicle is the Y axis, and the plane perpendicular to the X axis and the Y axis is the Z axis; the origin of the three-dimensional coordinate system is the starting point for building the global map;

[0013] IMU, the IMU is connected to the lower computer, and the IMU is used to collect the XYZ axis angular acceleration and angular velocity of the garbage picking and sorting vehicle body in the three-dimensional coordinate system, and obtain the angle θ of the garbage picking and sorting vehicle body rotating around the Z axis IMU ;

[0014] A mechanical arm, which is installed on the garbage picking and sorting vehicle body and is used for picking up and sorting garbage;

[0015] A wireless communication port is connected to the host computer, and is used to receive remote control signals, so that an operator can remotely control the garbage picking and sorting vehicle body.

[0016] Furthermore, the lower computer is connected to the upper computer via a serial bus and a CAN bus to achieve communication redundancy, wherein the emergency stop signal is transmitted via dual channels of the serial bus and the CAN bus to reduce the packet loss rate during the communication process.

[0017] Based on the same inventive concept, the present invention also provides a control method for an intelligent cruising garbage picking and sorting vehicle, which is implemented based on the aforementioned intelligent cruising garbage picking and sorting vehicle, and includes the steps of:

[0018] Step S1, the laser radar obtains 360° two-dimensional point cloud data;

[0019] Obtaining an RGB-D point cloud through the depth camera;

[0020] The movement speed of the garbage picking and sorting vehicle body is obtained through the odometer, and the movement speed of the garbage picking and sorting vehicle body is transmitted to the lower computer, and the lower computer processes the movement speed of the garbage picking and sorting vehicle body to obtain the XY axis coordinate P of the garbage picking and sorting vehicle body in the three-dimensional coordinate system. XY and the angle θ around the Z axis 里程计 ;

[0021] The IMU is used to obtain the rotation angle θ of the garbage collection and sorting vehicle body around the Z axis. IMU ,

[0022] Step S2, the two-dimensional point cloud data, the RGB-D point cloud, and the lower computer process the movement speed of the garbage collection and sorting vehicle body input by the odometer to obtain the XY axis coordinates P of the garbage collection and sorting vehicle body XYThe angle θ of rotation about the Z-axis 里程计 and the angle θ of rotation of the garbage collection and sorting vehicle body about the Z-axis obtained by the IMU IMU Perform data processing to generate a filtered two-dimensional grid map and filtered pose data;

[0023] Step S3, when the operator remotely controls the garbage collection and sorting vehicle body to move in the site, the movement trajectory covers the site at least twice; during the movement of the garbage collection and sorting vehicle body in the site, based on the filtered two-dimensional grid map and the filtered pose data, execute a mapping algorithm to generate the global map;

[0024] Step S4, the garbage collection and sorting vehicle performs autonomous navigation based on the global map using the inner spiral coverage algorithm, the DWA algorithm, and the search algorithm, and executes the task of garbage collection and sorting.

[0025] Further, the mapping algorithm in step S3 is the Cartographer algorithm.

[0026] Further, step S2 specifically performs the following steps:

[0027] Step S21, use the point cloud library PCL in ROS to project the 3D depth data onto the initial two-dimensional grid map to obtain two-dimensional plane data;

[0028] Step S22, perform mean filtering on the two-dimensional plane data and the initial two-dimensional grid map to eliminate isolated noise points and obtain the filtered two-dimensional grid map;

[0029] Step S23, perform extended Kalman filtering on the θ 里程计 and the θ IMU to generate the angle θ of rotation of the garbage collection and sorting vehicle body about the Z-axis after filtering; 滤波后 Combine the angle θ of rotation of the garbage collection and sorting vehicle body about the Z-axis after filtering with the P 滤波后 to generate the pose data of the garbage collection and sorting vehicle body after filtering. XY

[0030] Further, step S3 specifically performs the following steps:

[0031] The pose of the garbage collection and sorting vehicle body is used as a node in the Cartographer algorithm;

[0032] The transformation relationship between the nodes is used as an edge in the Cartographer algorithm;

[0033] Introduce a loop closure constraint in the backend of the Cartographer algorithm to eliminate the cumulative error and solve the problem of mapping in large-scale scenes;​

[0034] The large-scale scene mapping refers to a relatively large mapping scene;

[0035] In the back-end optimization of the Cartographer algorithm, a closed-loop constraint is established by real-time detection of the spatial matching of historical nodes and current nodes. When the garbage truck repeatedly passes through the mapped area, the Cartographer algorithm will generate edges connecting new and old nodes, and use graph optimization technology to globally adjust the posture of all nodes; the closed-loop constraint eliminates the accumulated error of the motion trajectory by fusing the real-time filtered posture data and the real-time filtered two-dimensional grid map;

[0036] The accumulated error is the error of the filtered posture data and the filtered two-dimensional grid map caused by the slippage of the garbage picking and sorting vehicle body.

[0037] Furthermore, the step S4 specifically performs the following steps:

[0038] Step S41, importing the global map;

[0039] Step S42, the garbage picking and sorting vehicle body executes an inner spiral covering algorithm, and the garbage picking and sorting vehicle body takes a corner of the periphery of the site as the starting point of the inner spiral covering algorithm, and forms a first U-shaped path along the outer boundary;

[0040] After the garbage collection and sorting vehicle completes each layer of the U-shaped path, it shrinks inward by a certain distance according to the minimum safe passing distance between obstacles in the venue, forming a new nested U-shaped layer;

[0041] Wherein, the path of the garbage picking and sorting vehicle body executing the inner spiral covering algorithm movement is the path composed of the nested back-shaped layers;

[0042] Wherein, the path composed of the nested back-word layers is a global path;

[0043] Among them, when the garbage pickup and sorting vehicle changes its direction of movement, the path smoothness is optimized through B-spline interpolation to eliminate jagged corners in the path;

[0044] When the garbage picking and sorting vehicle body approaches the blind spot of the corner, the host computer calls the search algorithm to generate an escape path;

[0045] During the movement of the garbage picking and sorting vehicle, the depth camera updates the local cost map in the ROS in real time, expands the obstacle area, and ensures that the safety distance between the path and the obstacle area is not less than 20 cm;

[0046] When a new obstacle is detected compared to the global map, the global path is paused, and the DWA algorithm is called to generate an obstacle avoidance trajectory, which is a local path; after the local path planning is completed, the garbage picking and sorting vehicle body moves along the local path, avoids the obstacle, and then moves along the global path again;

[0047] When the garbage pickup and sorting vehicle detects a suspected garbage target, it stops moving and makes a judgment on the suspected garbage target. In the host computer, YOLOv8 fuses the RGB image and the depth map to output whether the suspected garbage target is garbage.

[0048] If the suspected garbage target is garbage, the category and three-dimensional coordinates of the garbage are output; wherein the RGB image and the depth map are the RGB-D point cloud output by the depth camera; if the suspected garbage target is not garbage, it is processed according to the newly added obstacle;

[0049] The upper computer checks the size of the garbage. If the size of the garbage exceeds the upper limit of the gripper of the robotic arm, the upper computer performs obstacle avoidance. After avoiding the obstacle, the upper computer moves again according to the global path. If the size of the garbage does not exceed the upper limit of the gripper of the robotic arm, the upper computer transmits back the three-dimensional coordinates of the garbage, and the robotic arm grabs the garbage and returns it to the warehouse.

[0050] When the garbage picking and sorting vehicle finally reaches the innermost center point of the area and completes full coverage of the site, the mission is terminated when the garbage in the site is picked up and sorted.

[0051] Furthermore, the escape path is generated by using a backtracking algorithm in the search algorithm.

[0052] Based on the same inventive concept, the present invention also provides a computer device, including: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the aforementioned intelligent cruising garbage picking and classification method by executing the computer instructions.

[0053] Based on the same inventive concept, the present invention also provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the aforementioned intelligent cruising garbage picking and sorting method.

[0054] The beneficial effects of the present invention are:

[0055] Through the heterogeneous data collaboration of lidar and depth camera, combined with Cartographer and the extended Kalman filter algorithm, the limitations of a single sensor are effectively eliminated. The lidar provides high-precision two-dimensional spatial information, and the depth camera supplements three-dimensional geometry and visual semantic features, significantly enhancing the comprehensive perception ability of the garbage material, shape, and volume. This fusion not only solves the problem of misjudgment of garbage materials in traditional solutions but also supports the adaptive grasping strategy of the robotic arm, reducing the risk of grasping failure caused by environmental complexity. The dual-bus communication redundancy design ensures the stable transmission of key signals (such as emergency stop instructions) and avoids the risk of single-point failure. The hierarchical data processing architecture decouples the underlying motion control (lower computer) from the high-level decision-making and planning (upper computer), ensuring both real-time performance and improving the utilization efficiency of computing resources. This design enables the system to maintain stable operation in complex dynamic environments. The combination of the inner spiral coverage algorithm, DWA, and backtracking algorithm realizes more efficient full-coverage path planning through path smoothing and dynamic adjustment of safety distances, reducing the repeated traversal area. Description of the Drawings

[0056] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0057] Figure 1 It is a system architecture block diagram of an intelligent cruise garbage pickup and classification vehicle;

[0058] Figure 2 It is a schematic diagram of the movement path of the intelligent cruise garbage pickup and classification vehicle body. Detailed Implementation Modes

[0059] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0060] The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present invention.

[0061] The embodiments of the invention will be described in detail below with reference to the accompanying drawings.

[0062] An intelligent cruise garbage pickup and classification vehicle disclosed in this embodiment includes the following components: a host computer 2, a slave computer 1, a lidar 3, a depth camera 4, an odometer 5, an IMU 6, a robotic arm 7, and a wireless communication port 8; as shown in the attached Figure 1 figure. The functions, selections, and connection relationships of the above components will be described in detail below.

[0063] The host computer 2 runs ROS based on the Jetson Orin Nano embedded platform; the ROS (Robot Operating System) deploys a mapping algorithm, a garbage recognition and classification model YOLOv8, an inner spiral coverage algorithm, a search algorithm, and a DWA algorithm;

[0064] The host computer 2 is connected to a PC display screen 9 for real-time display of the position information of the garbage pickup and classification vehicle body 10 and the status information of the garbage pickup and classification vehicle body 10.

[0065] The slave computer 1 selects an STM32F407VET6 microcontroller, and the slave computer 1 is connected to the host computer 2;

[0066] The lidar 3 selects the Slan A1 lidar 3, and the lidar 3 is connected to the host computer 2;

[0067] The depth camera 4 selects an Intel Real Sense D435i depth camera 4, and the depth camera 4 is connected to the host computer 2;

[0068] The odometer 5 is connected to the slave computer 1. The odometer 5 is used to collect the moving speed of the garbage pickup and classification vehicle body 10. The odometer transmits the moving speed of the garbage pickup and classification vehicle body 10 to the slave computer 1, and the slave computer 1 processes the moving speed of the garbage pickup and classification vehicle body 10 to obtain the XY-axis coordinates P XY in the three-dimensional coordinate system of the global map of the garbage pickup and classification vehicle body 10 and the angle θ of rotation around the Z-axis 里程计 ;

[0069] Among them, in the three-dimensional coordinate system, the direction along the vehicle head extension is the X-axis, the direction perpendicular to the vehicle head is the Y-axis, and the plane perpendicular to the plane formed by the X-axis and the Y-axis is the Z-axis; the origin of the three-dimensional coordinate system is the mapping starting point of the global map;

[0070] The IMU 6 selects the MPU6050. The IMU 6 is connected to the lower computer. The IMU 6 is used to collect the XYZ-axis angular accelerations and angular velocities of the garbage pickup and sorting vehicle body 10 in the three-dimensional coordinate system, and obtain the angle θ of the garbage pickup and sorting vehicle body 10 rotating around the Z-axis. IMU ;

[0071] The robotic arm 7 is installed on the garbage pickup and sorting vehicle body 10 and is used for garbage pickup and sorting. In this embodiment, the robotic arm 7 has six degrees of freedom, corresponding to three translational degrees of freedom (movement in the X, Y, and Z-axis directions) and three rotational degrees of freedom (pitching, yawing, and rolling motions around the X, Y, and Z axes) at the end of the robotic arm 7, which are achieved through the coordinated movement of six independent joints.

[0072] The wireless communication port 8 selects a WIFI communication port. The wireless communication port 8 is connected to the upper computer 2. The wireless communication port 8 is used to receive remote control signals for an operator to remotely control the garbage pickup and sorting vehicle body 10.

[0073] Specifically, the lower computer 1 and the upper computer 2 are connected through a serial bus and a CAN bus to achieve communication redundancy. Among them, the emergency stop signal is transmitted through both the serial bus and the CAN bus channels to reduce the packet loss rate during communication.

[0074] Based on the same inventive concept, this embodiment also provides a control method for an intelligent cruise garbage pickup and sorting vehicle. This method is implemented based on the aforementioned intelligent cruise garbage pickup and sorting vehicle, and specifically includes the following steps:

[0075] Step S1, start the garbage pickup and sorting vehicle body 10, and the garbage pickup and sorting vehicle body 10 starts to move in the site 11; start the upper computer 2 and the lower computer 1;

[0076] Activate the lidar 3 (measurement radius 12m, scanning frequency 8000 times / second). The lidar 3 generates 360° two-dimensional point cloud data (resolution 5cm×5cm) and transmits it to the upper computer 2;

[0077] Start the depth camera 4. The depth camera 4 outputs RGB-D point clouds (resolution is full high definition 1080p, 30 frames per second) and transmits them to the upper computer 2;

[0078] Start the odometer 5. The odometer 5 collects the moving speed of the garbage pickup and sorting vehicle body 10. The odometer 5 transmits the moving speed of the garbage pickup and sorting vehicle body 10 to the lower computer 1. The lower computer 1 processes the moving speed of the garbage pickup and sorting vehicle body 10 to obtain the XY-axis coordinates P of the garbage pickup and sorting vehicle body 10 in the three-dimensional coordinate system XY and the angle θ of rotation around the Z-axis 里程计;The lower computer 1 transmits the XY-axis coordinates P of the garbage pickup and classification vehicle body 10 in the three-dimensional coordinate system XY and the angle θ of rotation about the Z-axis 里程计 to the upper computer 2;

[0079] Start the IMU 6. The IMU 6 collects the angular accelerations and angular velocities of the XYZ axes of the garbage pickup and classification vehicle body 10 in the three-dimensional coordinate system, and obtains the angle θ of rotation of the garbage pickup and classification vehicle body 10 about the Z-axis IMU , and transmits it to the lower computer 1. The lower computer 1 transmits the angle θ of rotation of the garbage pickup and classification vehicle body 10 about the Z-axis obtained by the IMU 6 IMU to the upper computer 2;

[0080] Step S2, in the upper computer 2, perform data processing on the two-dimensional point cloud data, RGB-D point cloud, and the XY-axis coordinates P of the garbage pickup and classification vehicle body 10 obtained by processing the odometer 5 by the lower computer 1 to obtain the garbage pickup and classification vehicle body 10 XY 、the angle θ of rotation about the Z-axis 里程计 and the angle θ of rotation of the garbage pickup and classification vehicle body 10 about the Z-axis obtained by the IMU 6 IMU to generate a filtered two-dimensional grid map and filtered pose data;

[0081] Among them, the two-dimensional point cloud data is the initial two-dimensional grid map; the RGB-D point cloud is 3D depth data;

[0082] Step S3, when the operator remotely controls the garbage pickup and classification vehicle body 10 to move in the site 11, the movement trajectory covers the site 11 at least twice; during the movement of the garbage pickup and classification vehicle body 10 in the site 11, perform a mapping algorithm based on the filtered two-dimensional grid map and filtered pose data to generate a global map;

[0083] Step S4, the garbage pickup and classification vehicle performs autonomous navigation based on the global map using an inner spiral coverage algorithm, DWA algorithm, and search algorithm to perform the task of garbage pickup and classification;

[0084] Specifically, the mapping algorithm in step S3 is the Cartographer algorithm.

[0085] Specifically, step S2 specifically performs the following steps:

[0086] Step S21, in the upper computer 2, use the point cloud library PCL in ROS to project the 3D depth data onto the initial two-dimensional grid map to obtain two-dimensional plane data; among them, the two-dimensional plane data is only converted from the 3D depth data, and the two-dimensional plane data does not include the initial two-dimensional grid map;

[0087] Step S22: Perform mean filtering on the two-dimensional plane data and the initial two-dimensional grid map to eliminate isolated noise points and obtain the filtered two-dimensional grid map.

[0088] Among them, eliminating isolated noise means that through the averaging of the neighborhood across data sources (a concept in mean filtering), if there are isolated outliers in a certain data source, the normal values in the same neighborhood of another data source will dilute the abnormal influence, ultimately reducing noise interference and generating a smooth (filtered) grid map.

[0089] Step S23: The lower computer 1 processes the movement speed of the garbage collection and classification vehicle body 10 transmitted by the odometer 5 to obtain the angle θ of the garbage collection and classification vehicle body 10 rotating around the Z-axis. 里程计 And the angle θ of the garbage collection and classification vehicle body 10 rotating around the Z-axis obtained by the IMU 6. IMU Perform extended Kalman filtering to generate the filtered angle θ of the garbage collection and classification vehicle body 10 rotating around the Z-axis. 里程计 ; Combine the filtered angle θ of the garbage collection and classification vehicle body 10 rotating around the Z-axis. 里程计 With the XY-axis coordinates P of the garbage collection and classification vehicle body 10 obtained by the lower computer 1 processing the movement speed of the garbage collection and classification vehicle body 10 transmitted by the odometer 5. XY To generate the filtered pose data of the garbage collection and classification vehicle body 10.

[0090] Specifically, the specific method of step S3 is as follows:

[0091] The pose of the garbage collection and classification vehicle body 10 serves as a node in the Cartographer algorithm, and the transformation relationship between each node serves as an edge in the Cartographer algorithm. Introduce loop closure constraints in the backend of the Cartographer algorithm to eliminate cumulative errors and solve the problem of large-scale scene mapping.

[0092] Among them, large-scale scene mapping refers to a relatively large mapping scene; compared with other mapping algorithms such as gmapping, the Cartographer algorithm is applicable to a larger usage scenario.

[0093] During the execution of step 2 and step 3, when the operator remotely controls the garbage picking and sorting vehicle body 10 to move in the site 11, the motion trajectory covers the site 11 at least twice; in the back-end optimization of the Cartographer algorithm, a closed-loop constraint (Loop Closure) is established by real-time detection of the spatial matching of historical nodes and current nodes. When the garbage truck repeatedly passes through the mapped area, the Cartographer algorithm will generate edges connecting new and old nodes, and use graph optimization technology (such as SPA or pose graph optimization) to globally adjust the poses of all nodes; the closed-loop constraint eliminates the accumulated error of the motion trajectory by fusing real-time filtered pose data and real-time filtered two-dimensional grid maps;

[0094] The accumulated error is the error of the filtered posture data and the filtered two-dimensional grid map caused by the slipping of the garbage picking and sorting vehicle body 10.

[0095] Specifically, step S4 specifically performs the following steps:

[0096] Step S41, importing a global map;

[0097] Step S42, the garbage picking and sorting vehicle body 10 executes the inner spiral covering algorithm. The garbage picking and sorting vehicle body 10 takes a certain corner of the outer periphery of the site 11 as the starting point of the inner spiral covering algorithm and forms the first U-shaped path along the outer boundary; as shown in the attached figure Figure 2 shown.

[0098] After the garbage picking and sorting vehicle body 10 completes each layer of the U-shaped path, it shrinks inward by a certain distance according to the minimum safe passing distance between obstacles in the site 11 to form a new nested U-shaped layer (the distance between adjacent U-shaped layers is 10 cm);

[0099] Among them, the global path 12 is the path composed of all nested word layers; Figure 2 As shown, the arrow direction is the direction of movement of the garbage picking and sorting vehicle body 10;

[0100] Among them, when the garbage picking and sorting vehicle body 10 changes its moving direction, the path smoothness is optimized by B-spline interpolation to eliminate jagged corners in the path;

[0101] When the garbage picking and sorting vehicle body 10 approaches the corner blind spot 14, the host computer calls the search algorithm to generate an escape path; the escape path is the path for the garbage picking and sorting vehicle body 10 to return to the original path when it encounters a dead end; Figure 2 shown.

[0102] During the movement of the garbage picking and sorting vehicle 10, the depth camera updates the local cost map (costmap_2d) in ROS in real time, expands the obstacle area 15, and ensures that the safety distance between the path and the obstacle area 15 is not less than 20 cm; as shown in the attached figure Figure 2 shown.

[0103] When an obstacle is detected that is new compared to the global map, the global path 12 is paused, and the DWA algorithm is called to generate an obstacle avoidance trajectory 13 (with a minimum turning radius of 15 cm), which is a local path. After the local path planning is completed, the garbage picking and sorting vehicle body 10 moves along the local path, avoids the obstacle, and then moves along the global path 12 again. Figure 2 shown.

[0104] When the garbage picking and sorting vehicle body 10 detects a target suspected to be garbage, it stops moving and makes a judgment on the target suspected to be garbage. In the host computer 2, YOLOv8 fuses the RGB image and the depth map and outputs whether the target suspected to be garbage is garbage.

[0105] If the suspected garbage target is garbage, the category and three-dimensional coordinates of the garbage are output;

[0106] Among them, the RGB image and depth map are the RGB-D point clouds output by the depth camera 4; the categories of garbage include "recyclable", "kitchen waste", "hazardous" and "others"; the three-dimensional coordinates of the garbage are converted from the TF coordinate system in ROS;

[0107] If the object suspected to be garbage is not garbage, it will be treated as a newly added obstacle;

[0108] The upper computer 2 checks the size of the garbage (the upper limit of the grabbing is ≤ 20 cm³). If the size of the garbage exceeds the upper limit of the gripping of the gripper of the robot arm 7, it avoids obstacles and moves again according to the global path 12 after avoiding the obstacles. If the size of the garbage does not exceed the upper limit of the gripping of the gripper of the robot arm 7, the three-dimensional coordinates of the garbage are transmitted back, and the robot arm 7 grabs the garbage and returns it to the warehouse.

[0109] When the garbage picking and sorting vehicle body 10 finally reaches the innermost center point of the area and completes full coverage of the site 11, at this time, when the garbage in the site 11 is picked up and sorted, the task is terminated.

[0110] Specifically, the backtracking algorithm in the search algorithm is used to generate the escape path. The backtracking algorithm is a kind of search algorithm.

[0111] Based on the same inventive concept, this embodiment also provides a computer device, which includes a memory and a processor. The memory and the processor are communicatively connected to each other. Computer instructions are stored in the memory. The processor executes the aforementioned intelligent cruising garbage picking and sorting method by executing the computer instructions.

[0112] Based on the same inventive concept, this embodiment also provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the aforementioned intelligent cruising garbage picking and classification method.

[0113] The intelligent cruising garbage picking and sorting vehicle and method provided in this embodiment have the following advantages over the prior art:

[0114] Multimodal perception fusion improves environmental understanding capabilities

[0115] Through the heterogeneous data collaboration of LiDAR 3 and Depth Camera 4, combined with Cartographer and extended Kalman filter algorithm, the limitations of a single sensor are effectively eliminated. LiDAR 3 provides high-precision two-dimensional spatial information, and Depth Camera 4 supplements three-dimensional geometric and visual semantic features, significantly enhancing the comprehensive perception of garbage material, shape and volume. This fusion not only solves the problem of misjudgment of garbage material in traditional solutions, but also supports the adaptive grasping strategy of the robot arm 7, reducing the risk of grasping failure due to environmental complexity.

[0116] Redundant architecture ensures system reliability

[0117] The dual-bus communication redundant design ensures the stable transmission of key signals (such as emergency stop commands) and avoids the risk of single-point failure. The hierarchical data processing architecture decouples the underlying motion control (lower computer 1) from the high-level decision-making planning (upper computer 2), which not only ensures real-time performance but also improves the efficiency of computing resource utilization. This design enables the system to maintain stable operation in a complex dynamic environment.

[0118] Adaptive algorithms to optimize operational performance

[0119] The inner spiral coverage algorithm, DWA and backtracking algorithm are combined to achieve more efficient full coverage path planning and reduce repeated traversal areas through path smoothing and dynamic adjustment of safety spacing. The mapping method combining closed-loop constraint mechanism and multi-source data fusion can effectively suppress the accumulated errors in long-term operations and improve the accuracy of large-scale scene mapping. The linkage design of local obstacle avoidance and escape strategy further enhances the system's adaptability to sudden obstacles.

[0120] Integrated design reduces deployment costs

[0121] The functional complementarity between the depth camera 4 and the laser radar 3 avoids the redundant configuration of the dedicated image processing unit in the traditional solution, and can realize multi-modal data processing under the architecture of a single host computer 2. The six-degree-of-freedom robot arm 7 supports the precise grasping of various garbage forms through the coordinated control of coordinate system conversion and visual guidance, significantly reducing the need for manual intervention, and the overall hardware architecture is more compact and cost-effective.

Claims

1. An intelligent cruise garbage pickup and sorting vehicle, characterized in that, Including: A host computer that runs ROS based on the Jetson Orin Nano embedded platform; the ROS deploys a mapping algorithm, a garbage recognition and classification model YOLOv8, an inner spiral coverage algorithm, a search algorithm, and a DWA algorithm; A slave computer that is connected to the host computer; A lidar that is connected to the host computer; A depth camera that is connected to the host computer; An odometer, which is connected to the lower computer. The odometer is used to collect the moving speed of the garbage pickup and sorting vehicle body. The odometer transmits the moving speed of the garbage pickup and sorting vehicle body to the lower computer, and the lower computer processes the moving speed of the garbage pickup and sorting vehicle body to obtain the XY-axis coordinates P of the garbage pickup and sorting vehicle body in the three-dimensional coordinate system of the global map XY and the angle θ of rotation around the Z axis 里程计 ; Among them, in a three-dimensional coordinate system, the direction along the vehicle head is the X-axis, the direction perpendicular to the vehicle head is the Y-axis, and the plane perpendicular to the plane formed by the X-axis and the Y-axis is the Z-axis; the origin of the three-dimensional coordinate system is the mapping starting point of the global map; IMU, the IMU is connected to the lower computer, and the IMU is used to collect the XYZ-axis angular accelerations and angular velocities of the garbage pickup and sorting vehicle body in a three-dimensional coordinate system, and obtain the angle θ of the garbage pickup and sorting vehicle body rotating around the Z-axis IMU ; A robotic arm that is installed on the garbage pickup and classification vehicle body and is used for garbage pickup and classification; A wireless communication port that is connected to the host computer and is used to receive remote control signals for an operator to remotely control the garbage pickup and classification vehicle body.

2. The intelligent cruise garbage pickup and sorting vehicle according to claim 1, wherein The slave computer and the host computer are connected through a serial bus and a CAN bus to achieve communication redundancy, where the emergency stop signal is transmitted through both the serial bus and the CAN bus channels to reduce the packet loss rate during communication.

3. A control method for an intelligent cruise garbage pickup and sorting vehicle, which is implemented based on the intelligent cruise garbage pickup and sorting vehicle described in claim 1 or 2, characterized in that, Including the steps: Step S1, the lidar obtains 360° two-dimensional point cloud data; Obtain RGB-D point cloud through the depth camera; Obtain the moving speed of the garbage pickup and sorting vehicle body through the odometer, and input the moving speed of the garbage pickup and sorting vehicle body into the lower computer. The lower computer processes the moving speed of the garbage pickup and sorting vehicle body to obtain the XY-axis coordinates P of the garbage pickup and sorting vehicle body in the three-dimensional coordinate system XY and the angle θ of rotation around the Z axis 里程计 ; Obtain the angle θ of the garbage collection and sorting vehicle body rotating around the Z-axis through the IMU IMU , Step S2, perform data processing on the two-dimensional point cloud data, the RGB-D point cloud, the XY-axis coordinates P of the garbage collection and classification vehicle body obtained by the lower computer processing the movement speed of the garbage collection and classification vehicle body transmitted by the odometer XY , the angle θ of rotation about the Z-axis 里程计 , and the angle θ of rotation of the garbage collection and classification vehicle body about the Z-axis obtained by the IMU IMU to generate a filtered two-dimensional grid map and filtered pose data; Step S3, when the operator remotely controls the garbage pickup and classification vehicle body to move in the site, the movement trajectory covers the site at least twice; during the movement of the garbage pickup and classification vehicle body in the site, execute the mapping algorithm based on the filtered two-dimensional grid map and the filtered pose data to generate the global map; Step S4, the garbage pickup and classification vehicle performs autonomous navigation using the inner spiral coverage algorithm, the DWA algorithm, and the search algorithm based on the global map and executes the task of garbage pickup and classification.

4. The intelligent cruise garbage pickup and classification method according to claim 3, characterized in that The mapping algorithm in step S3 is the Cartographer algorithm.

5. The intelligent cruise garbage pickup and classification method according to claim 4, wherein, Step S2 specifically executes the following steps: Step S21, use the Point Cloud Library (PCL) in ROS to project the 3D depth data onto the initial two-dimensional grid map to obtain two-dimensional plane data; Step S22, perform mean filtering on the two-dimensional plane data and the initial two-dimensional grid map to eliminate isolated noise points and obtain the filtered two-dimensional grid map; Step S23, take the said θ 里程计 and the said θ IMU to perform extended Kalman filtering to generate the angle θ 滤波后 by which the garbage pickup and sorting vehicle body rotates around the Z-axis after filtering; combine the angle θ 滤波后 by which the garbage pickup and sorting vehicle body rotates around the Z-axis after filtering with the said P XY to generate the pose data of the garbage pickup and sorting vehicle body after filtering.

6. The intelligent cruise garbage pickup and classification method according to claim 5, wherein Step S3 specifically executes the following steps: The pose of the garbage pickup and classification vehicle body is used as a node in the Cartographer algorithm; The conversion relationship between each node is used as an edge in the Cartographer algorithm; Introduce a loop closure constraint in the backend of the Cartographer algorithm to eliminate the cumulative error to solve the problem of large-scale scene mapping; Among them, the large-scale scene mapping refers to a relatively large mapping scene; In the back-end optimization of the Cartographer algorithm, a closed-loop constraint is established by real-time detection of the spatial matching of historical nodes and current nodes. When the garbage truck repeatedly passes through the mapped area, the Cartographer algorithm will generate edges connecting new and old nodes, and use graph optimization technology to globally adjust the posture of all nodes; the closed-loop constraint eliminates the accumulated error of the motion trajectory by fusing the real-time filtered posture data and the real-time filtered two-dimensional grid map; The accumulated error is the error of the filtered posture data and the filtered two-dimensional grid map caused by the slippage of the garbage picking and sorting vehicle body.

7. The intelligent cruise garbage pickup and classification method according to claim 6, wherein The step S4 specifically performs the following steps: Step S41, importing the global map; Step S42, the garbage picking and sorting vehicle body executes an inner spiral covering algorithm, and the garbage picking and sorting vehicle body takes a corner of the periphery of the site as the starting point of the inner spiral covering algorithm, and forms a first U-shaped path along the outer boundary; After the garbage collection and sorting vehicle completes each layer of the U-shaped path, it shrinks inward by a certain distance according to the minimum safe passing distance between obstacles in the venue, forming a new nested U-shaped layer; Wherein, the path of the garbage picking and sorting vehicle body executing the inner spiral covering algorithm movement is the path composed of the nested back-shaped layers; Wherein, the path composed of the nested back-word layers is a global path; Among them, when the garbage pickup and sorting vehicle changes its direction of movement, the path smoothness is optimized through B-spline interpolation to eliminate jagged corners in the path; When the garbage picking and sorting vehicle body approaches the blind spot of the corner, the host computer calls the search algorithm to generate an escape path; During the movement of the garbage picking and sorting vehicle, the depth camera updates the local cost map in the ROS in real time, expands the obstacle area, and ensures that the safety distance between the path and the obstacle area is not less than 20 cm; When a new obstacle is detected compared to the global map, the global path is paused, and the DWA algorithm is called to generate an obstacle avoidance trajectory, which is a local path; after the local path planning is completed, the garbage picking and sorting vehicle body moves along the local path, avoids the obstacle, and then moves along the global path again; When the garbage pickup and sorting vehicle detects a suspected garbage target, it stops moving and makes a judgment on the suspected garbage target. In the host computer, YOLOv8 fuses the RGB image and the depth map to output whether the suspected garbage target is garbage. If the suspected garbage target is garbage, the category and three-dimensional coordinates of the garbage are output; wherein the RGB image and the depth map are the RGB-D point cloud output by the depth camera; if the suspected garbage target is not garbage, it is processed according to the newly added obstacle; The upper computer checks the size of the garbage. If the size of the garbage exceeds the upper limit of the gripper of the robotic arm, the upper computer performs obstacle avoidance. After avoiding the obstacle, the upper computer moves again according to the global path. If the size of the garbage does not exceed the upper limit of the gripper of the robotic arm, the upper computer transmits back the three-dimensional coordinates of the garbage, and the robotic arm grabs the garbage and returns it to the warehouse. When the garbage pickup and sorting vehicle body finally reaches the center point of the innermost layer of the area and completes the full coverage of the site, at this time, when the garbage in the site has been picked up and sorted, the task terminates.

8. The intelligent cruise garbage pickup and classification method according to claim 7, characterized in that, The escape path is generated by using the backtracking algorithm in the search algorithm.

9. A computer device, characterized in that, Including: A memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the intelligent cruise garbage pickup and sorting method according to any one of claims 3-8.

10. A computer-readable storage medium, characterized in that Computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the intelligent cruise garbage pickup and sorting method according to any one of claims 3-8.

Images