Intelligent cruise garbage pickup and classification vehicle, method, device and medium

Abstract

The application provides an intelligent cruise garbage pickup and classification vehicle, method, equipment and medium to solve the problem that the existing laser radar can only obtain two-dimensional point cloud data and cannot identify garbage material, volume and other characteristics. The garbage pickup and classification vehicle comprises the following components: an upper computer, a lower computer, a laser radar, a depth camera, an odometer, an IMU, a mechanical arm and a wireless communication port. Through the heterogeneous data cooperation of the laser radar and the depth camera, the limitations of a single sensor are effectively eliminated. The laser radar provides high-precision two-dimensional spatial information, and the depth camera supplements three-dimensional geometric and visual semantic features, significantly enhancing the comprehensive perception ability of the garbage material, shape and volume. This fusion not only solves the problem of garbage material misjudgment in the traditional scheme, but also supports the adaptive grabbing strategy of the mechanical arm, reducing the risk of grabbing failure caused by environmental complexity. A double-bus communication redundancy design is adopted to ensure the stable transmission of key signals.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and in particular relates to an intelligent cruise garbage collection and sorting vehicle, method, equipment and medium. Background Technology

[0002] The intelligent cruise garbage collection and sorting vehicle uses multi-sensor collaborative perception, autonomous navigation and path planning, intelligent robotic arm grasping, and automatic garbage sorting module to achieve fully automatic garbage collection and sorting management in closed areas (such as parks, squares, etc.), providing data support for sanitation management.

[0003] Existing intelligent cruise garbage sorting vehicles using a single LiDAR radar have the following problems: LiDAR can only acquire two-dimensional point cloud data and cannot identify features such as garbage material and volume, resulting in a high failure rate for the robotic arm (e.g., plastic bags and cardboard boxes are easily confused due to their similar reflectivity). This necessitates manual pre-labeling of garbage categories, and the sorting function requires the additional deployment of an image processing unit, increasing hardware costs by more than 30% (e.g., adding a Jetson series host computer). Summary of the Invention

[0004] To address the problem that existing technologies, such as lidar, can only acquire two-dimensional point cloud data and cannot identify features like the material and volume of waste, leading to a high failure rate in robotic arm grasping, this invention provides an intelligent cruise waste picking and sorting vehicle, method, equipment, and medium.

[0005] The technical solution adopted in this invention is:

[0006] A smart cruise garbage collection and sorting vehicle includes:

[0007] The host computer runs ROS on the Jetson Orin Nano embedded platform; the ROS deploys mapping algorithms, the YOLOv8 garbage identification and classification model, the inner spiral cover algorithm, the search algorithm, and the DWA algorithm.

[0008] A lower-level machine, which is connected to the upper-level machine;

[0009] The lidar is connected to the host computer;

[0010] A depth camera, which is connected to the host computer;

[0011] An odometer, connected to the lower-level machine, is used to collect the movement speed of the waste collection and sorting vehicle. The odometer transmits the movement speed of the waste collection and sorting vehicle to the lower-level machine, which processes the speed data to obtain the XY axis coordinates P of the waste collection and sorting vehicle in the three-dimensional coordinate system of the global map. XYand the angle θ of rotation around the Z-axis 里程计 ;

[0012] In the three-dimensional coordinate system, the X-axis extends along the front of the vehicle, the Y-axis is perpendicular to the front of the vehicle, and the Z-axis is perpendicular to the plane formed by the X-axis and the Y-axis; the origin of the three-dimensional coordinate system is the starting point for the construction of the global map.

[0013] An IMU, connected to the lower-level computer, is used to collect the angular acceleration and angular velocity of the waste collection and sorting vehicle body along the XYZ axes in a three-dimensional coordinate system, and to obtain the angle θ of the waste collection and sorting vehicle body rotating around the Z-axis. IMU ;

[0014] A robotic arm, which is mounted on the main body of the waste collection and sorting vehicle, is used for waste collection and sorting;

[0015] A wireless communication port is provided, which is connected to the host computer. The wireless communication port is used to receive remote control signals, allowing the operator to remotely control the garbage collection and sorting vehicle.

[0016] Furthermore, the lower-level machine and the upper-level machine are connected via a serial bus and a CAN bus to achieve communication redundancy. The emergency stop signal is transmitted through dual channels of the serial bus and the CAN bus to reduce the packet loss rate during communication.

[0017] Based on the same inventive concept, the present invention also provides an intelligent cruise garbage collection and sorting method, which is implemented based on the aforementioned intelligent cruise garbage collection and sorting vehicle, and includes the following steps:

[0018] Step S1: The lidar acquires 360° two-dimensional point cloud data;

[0019] RGB-D point clouds are obtained using the depth camera;

[0020] The speed of the waste collection and sorting vehicle is obtained through the odometer and transmitted to the lower-level computer. The lower-level computer processes the speed of the waste collection and sorting vehicle to obtain the XY axis coordinates P of the waste collection and sorting vehicle in the three-dimensional coordinate system. XY and the angle θ of rotation around the Z-axis 里程计 ;

[0021] The angle θ of the rotation of the garbage sorting vehicle body around the Z-axis is obtained through the IMU. IMU ,

[0022] Step S2 involves processing the two-dimensional point cloud data, the RGB-D point cloud, and the XY-axis coordinates P of the waste collection and sorting vehicle body obtained by the lower-level machine from the movement speed of the waste collection and sorting vehicle body transmitted from the odometer. XYθ, the angle of rotation around the Z-axis 里程计 And the angle θ of the rotation of the garbage collection and sorting vehicle body around the Z-axis obtained by the IMU. IMU Perform data processing to generate a filtered 2D raster map and filtered pose data;

[0023] Step S3: When the operator remotely controls the garbage collection and sorting vehicle to move in the field, the movement trajectory covers the field at least twice; during the movement of the garbage collection and sorting vehicle in the field, a mapping algorithm is executed based on the filtered two-dimensional grid map and the filtered pose data to generate the global map.

[0024] In step S4, the garbage collection and sorting vehicle autonomously navigates based on the global map using the inner spiral cover algorithm, the DWA algorithm, and the search algorithm to perform the task of garbage collection and sorting.

[0025] Furthermore, the mapping algorithm in step S3 is the Cartographer algorithm.

[0026] Furthermore, step S2 specifically involves the following steps:

[0027] Step S21: Use the point cloud library PCL in ROS to project the 3D depth data onto the initial two-dimensional raster map to obtain two-dimensional planar data;

[0028] Step S22: The two-dimensional planar data and the initial two-dimensional raster map are subjected to mean filtering to eliminate isolated noise points and obtain the filtered two-dimensional raster map.

[0029] Step S23, the θ 里程计 With the θ IMU Extended Kalman filtering is performed to generate the angle θ of the filtered rotation of the waste sorting vehicle around the Z-axis. 滤波后 The angle θ of the filtered body of the garbage collection and sorting vehicle around the Z-axis 滤波后 With the P XY Combined, the filtered pose data of the garbage collection and sorting vehicle is generated.

[0030] Furthermore, step S3 specifically involves the following steps:

[0031] The pose of the garbage collection and sorting vehicle is used as a node in the Cartographer algorithm;

[0032] The transformation relationships between the nodes are used as edges in the Cartographer algorithm;

[0033] Introducing closed-loop constraints in the backend of the Cartographer algorithm to eliminate accumulated errors solves the problem of mapping large-scale scenes;

[0034] The large-scale scene mapping refers to a large mapping scene;

[0035] In the backend optimization of the Cartographer algorithm, a closed-loop constraint is established by real-time detection of the spatial matching between historical nodes and the current node. When the garbage truck repeatedly passes through the map area, the Cartographer algorithm will generate edges connecting the old and new nodes and use graph optimization technology to globally adjust the pose of all nodes. The closed-loop constraint eliminates the cumulative error of the motion trajectory by fusing the real-time filtered pose data and the real-time filtered two-dimensional grid map.

[0036] The cumulative error refers to the error in the filtered pose data and the filtered two-dimensional grid map caused by the slippage of the garbage collection and sorting vehicle.

[0037] Furthermore, step S4 specifically involves the following steps:

[0038] Step S41: Import the global map;

[0039] Step S42: The garbage collection and sorting vehicle body executes the inner spiral covering algorithm. The garbage collection and sorting vehicle body starts from a corner of the outer perimeter of the site as the starting point of the inner spiral covering algorithm and forms the first zigzag path along the outer boundary.

[0040] After completing each layer of the U-shaped path, the garbage collection and sorting vehicle shrinks inward by a certain distance based on the minimum safe passage distance between obstacles in the site, forming a new nested U-shaped layer.

[0041] Among them, the path of the garbage collection and sorting vehicle body executing the inner spiral covering algorithm is the path composed of the nested back-shaped layers;

[0042] The path formed by the nested back-word layers is a global path;

[0043] Among them, when the garbage collection and sorting vehicle changes its direction of movement, the path smoothness is optimized by B-spline interpolation to eliminate sawtooth corners in the path;

[0044] When the garbage collection and sorting vehicle approaches a blind spot in a corner, the host computer calls the search algorithm to generate an escape path.

[0045] During the movement of the garbage collection and sorting vehicle, the depth camera updates the local cost map in the ROS in real time, expands the obstacle area, and ensures that the safe distance between the path and the obstacle area is not less than 20cm.

[0046] When an obstacle is detected that is newly added compared to the global map, the global path is paused and the DWA algorithm is called to generate an obstacle avoidance trajectory, which is a local path. After the local path is planned, the garbage collection and sorting vehicle moves according to the local path, avoids the obstacle, and then moves according to the global path again.

[0047] When the garbage collection and sorting vehicle detects a suspected garbage target, it stops moving and judges the suspected garbage target. In the host computer, YOLOv8 merges the RGB image and the depth map and outputs whether the suspected garbage target is garbage.

[0048] If the suspected trash target is indeed trash, then the trash category and 3D coordinates are output; wherein, the RGB image and depth map are the RGB-D point cloud output by the depth camera; if the suspected trash target is not trash, then it is handled as a newly added obstacle.

[0049] The host computer verifies the size of the waste. If the size of the waste exceeds the gripping limit of the robotic arm, obstacle avoidance is performed. After avoiding the obstacle, the robot moves along the global path again. If the size of the waste does not exceed the gripping limit of the robotic arm, the three-dimensional coordinates of the waste are returned, and the robotic arm grabs the waste and puts it into storage.

[0050] When the garbage collection and sorting vehicle finally reaches the center of the innermost layer of the area, completing full coverage of the site, the task ends once the garbage in the site has been collected and sorted.

[0051] Furthermore, the backtracking algorithm in the search algorithm is used to generate the escape path.

[0052] Based on the same inventive concept, the present invention also provides a computer device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the aforementioned intelligent cruise garbage collection and sorting method by executing the computer instructions.

[0053] Based on the same inventive concept, the present invention also provides a computer-readable storage medium storing computer instructions, which are used to cause a computer to execute the aforementioned intelligent cruise garbage collection and sorting method.

[0054] The beneficial effects of this invention are:

[0055] By combining heterogeneous data from LiDAR and depth cameras, along with Cartographer and Extended Kalman Filter algorithms, the limitations of single sensors are effectively eliminated. LiDAR provides high-precision two-dimensional spatial information, while the depth camera supplements three-dimensional geometric and visual semantic features, significantly enhancing the comprehensive perception capability of waste material, shape, and volume. This fusion not only solves the problem of misjudging waste material in traditional solutions but also supports adaptive grasping strategies for robotic arms, reducing the risk of grasping failure due to environmental complexity. A dual-bus communication redundancy design ensures stable transmission of critical signals (such as emergency stop commands), avoiding the risk of single-point failure. A hierarchical data processing architecture decouples the underlying motion control (lower-level machine) from the higher-level decision-making and planning (upper-level machine), ensuring both real-time performance and improved computational resource utilization efficiency. This design enables the system to maintain stable operation even in complex dynamic environments. The combination of the inner spiral coverage algorithm, DWA, and backtracking algorithm, through path smoothing and dynamic adjustment of safety spacing, achieves more efficient full-coverage path planning and reduces repeated traversal areas. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a system architecture diagram for an intelligent cruise garbage collection and sorting vehicle.

[0058] Figure 2 A schematic diagram of the movement path of the intelligent cruise garbage collection and sorting vehicle. Detailed Implementation

[0059] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0060] The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, these are merely examples and are not intended to limit the present invention.

[0061] The embodiments of the invention will now be described in detail with reference to the accompanying drawings.

[0062] This embodiment discloses an intelligent cruise garbage collection and sorting vehicle, comprising the following components: a host computer 2, a slave computer 1, a lidar 3, a depth camera 4, an odometer 5, an IMU 6, a robotic arm 7, and a wireless communication port 8; as shown in the attached diagram. Figure 1 As shown. The following text details the function, selection, and connection relationships of each of the above components.

[0063] Host computer 2 runs ROS on the Jetson Orin Nano embedded platform; ROS (Robot Operating System) deploys mapping algorithms, the YOLOv8 garbage identification and classification model, the inner spiral covering algorithm, the search algorithm, and the DWA algorithm;

[0064] The host computer 2 is connected to a PC display screen 9, which is used to display the location information and status information of the garbage collection and sorting vehicle body 10 in real time.

[0065] The lower-level machine 1 uses an STM32F407VET6 microcontroller and is connected to the upper-level machine 2.

[0066] The lidar 3 is the Slamtec A1 lidar 3, and the lidar 3 is connected to the host computer 2.

[0067] The depth camera 4 is an Intel RealSense D435i depth camera 4, which is connected to the host computer 2;

[0068] Odometer 5 is connected to the lower-level computer 1. Odometer 5 is used to collect the movement speed of the garbage collection and sorting vehicle body 10. The odometer transmits the movement speed of the garbage collection and sorting vehicle body 10 to the lower-level computer 1. The lower-level computer 1 processes the movement speed of the garbage collection and sorting vehicle body 10 to obtain the XY axis coordinates P of the garbage collection and sorting vehicle body 10 in the three-dimensional coordinate system of the global map. XY and the angle θ of rotation around the Z-axis 里程计 ;

[0069] In the three-dimensional coordinate system, the X-axis extends along the front of the vehicle, the Y-axis is perpendicular to the front of the vehicle, and the Z-axis is perpendicular to the plane formed by the X-axis and the Y-axis; the origin of the three-dimensional coordinate system is the starting point for the construction of the global map.

[0070] The IMU 6 uses an MPU6050 and connects to the lower-level computer. The IMU 6 is used to collect the angular acceleration and angular velocity of the garbage collection and sorting vehicle body 10 along the XYZ axes in a three-dimensional coordinate system, obtaining the angle θ of rotation of the garbage collection and sorting vehicle body 10 around the Z-axis. IMU ;

[0071] The robotic arm 7 is mounted on the main body 10 of the garbage collection and sorting vehicle and is used for garbage collection and sorting. In this embodiment, the robotic arm 7 has six degrees of freedom, which correspond to three translational degrees of freedom (movement in the X, Y, and Z axes) and three rotational degrees of freedom (pitch, yaw, and roll motion around the X, Y, and Z axes) at the end of the robotic arm 7, respectively, and are realized through the coordinated movement of six independent joints.

[0072] Wireless communication port 8 is a WIFI communication port. Wireless communication port 8 is connected to host computer 2. Wireless communication port 8 is used to receive remote control signals, and the operator remotely controls the garbage collection and sorting vehicle body 10.

[0073] Specifically, the lower-level computer 1 and the upper-level computer 2 are connected via a serial bus and a CAN bus to achieve communication redundancy. The emergency stop signal is transmitted through dual channels of the serial bus and the CAN bus to reduce the packet loss rate during communication.

[0074] Based on the same inventive concept, this embodiment also provides an intelligent cruise garbage collection and sorting method, which is implemented based on the aforementioned intelligent cruise garbage collection and sorting vehicle, and specifically includes the following steps:

[0075] Step S1: Start the garbage collection and sorting vehicle body 10, and the garbage collection and sorting vehicle body 10 begins to move in the site 11; start the host computer 2 and the slave computer 1;

[0076] Activate LiDAR 3 (measurement radius 12m, scanning frequency 8000 times / second), LiDAR 3 generates 360° two-dimensional point cloud data (resolution 5cm×5cm) and transmits it to host computer 2;

[0077] Start Depth Camera 4. Depth Camera 4 outputs RGB-D point cloud (full HD 1080p resolution, 30 frames per second) and transmits it to host computer 2.

[0078] The odometer 5 is activated, and it collects the movement speed of the garbage collection and sorting vehicle body 10. The odometer 5 transmits the movement speed of the garbage collection and sorting vehicle body 10 to the lower-level computer 1. The lower-level computer 1 processes the movement speed of the garbage collection and sorting vehicle body 10 to obtain the XY axis coordinates P of the garbage collection and sorting vehicle body 10 in the three-dimensional coordinate system. XY and the angle θ of rotation around the Z-axis 里程计 The lower-level machine 1 sets the XY axis coordinates P of the garbage collection and sorting vehicle body 10 in the three-dimensional coordinate system.XY and the angle θ of rotation around the Z-axis 里程计 Passed to host computer 2;

[0079] IMU 6 is activated. IMU 6 collects the angular acceleration and angular velocity of the garbage collection and sorting vehicle body 10 along the XYZ axes in the three-dimensional coordinate system, and obtains the angle θ of the garbage collection and sorting vehicle body 10 rotating around the Z-axis. IMU The data is then transmitted to lower-level machine 1, which in turn transmits the angle θ obtained from IMU 6, representing the rotation of the garbage collection and sorting vehicle body 10 around the Z-axis. IMU Passed to host computer 2;

[0080] Step S2: In the host computer 2, the XY axis coordinates P of the garbage collection and sorting vehicle body 10 are obtained by processing the two-dimensional point cloud data, RGB-D point cloud, and the movement speed of the garbage collection and sorting vehicle body 10 transmitted from the odometer 5 in the lower computer 1. XY θ, the angle of rotation around the Z-axis 里程计 And the angle θ of the rotation of the garbage collection and sorting vehicle body 10 around the Z-axis obtained by IMU 6. IMU Perform data processing to generate a filtered 2D raster map and filtered pose data;

[0081] Among them, the 2D point cloud data is the initial 2D raster map; the RGB-D point cloud is 3D depth data;

[0082] Step S3: When the operator remotely controls the garbage collection and sorting vehicle body 10 to move in the site 11, the movement trajectory covers the site 11 at least twice; during the movement of the garbage collection and sorting vehicle body 10 in the site 11, a mapping algorithm is executed based on the filtered two-dimensional grid map and the filtered pose data to generate a global map.

[0083] Step S4: The garbage collection and sorting vehicle autonomously navigates based on the global map using the inner spiral cover algorithm, DWA algorithm, and search algorithm to perform the task of garbage collection and sorting.

[0084] Specifically, the mapping algorithm in step S3 is the Cartographer algorithm.

[0085] Specifically, step S2 involves the following steps:

[0086] Step S21: On the host computer 2, the point cloud library PCL in ROS is used to project the 3D depth data onto the initial two-dimensional raster map to obtain two-dimensional planar data; wherein, the two-dimensional planar data is only converted from the 3D depth data, and the two-dimensional planar data does not include the initial two-dimensional raster map.

[0087] Step S22: The two-dimensional planar data and the initial two-dimensional raster map are subjected to mean filtering to eliminate isolated noise points and obtain the filtered two-dimensional raster map.

[0088] Eliminating isolated noise refers to averaging across neighborhoods (a concept in mean filtering) across data sources. If an isolated outlier exists in one data source, the normal values ​​in the same neighborhood of another data source will dilute the impact of the outlier, ultimately reducing noise interference and generating a smooth (filtered) raster map.

[0089] Step S23: The lower-level machine 1 processes the speed of the garbage collection and sorting vehicle body 10 transmitted from the odometer 5 to obtain the angle θ of the rotation of the garbage collection and sorting vehicle body 10 around the Z-axis. 里程计 And the angle θ of the rotation of the garbage collection and sorting vehicle body 10 around the Z-axis obtained by IMU 6. IMU Extended Kalman filtering is performed to generate the angle θ of the filtered rotation of the waste collection and sorting vehicle body around the Z-axis. 里程计 The angle θ of the filtered rotation of the garbage collection and sorting vehicle body around the Z-axis is... 里程计 The XY coordinates P of the garbage collection and sorting vehicle body 10 are obtained by processing the movement speed of the garbage collection and sorting vehicle body 10 transmitted from the odometer 5 by the lower computer 1. XY Combined, the filtered pose data of the garbage collection and sorting vehicle body is generated.

[0090] Specifically, the method for step S3 is as follows:

[0091] The pose of the garbage collection and sorting vehicle body 10 is used as a node in the Cartographer algorithm, and the transformation relationship between each node is used as an edge in the Cartographer algorithm. Closed-loop constraints are introduced in the back end of the Cartographer algorithm to eliminate accumulated errors in order to solve the problem of large-scale scene mapping.

[0092] Among them, large-scale scene mapping refers to larger mapping scenes; compared with other mapping algorithms such as gmapping, the Cartographer algorithm is suitable for larger application scenarios;

[0093] During steps 2 and 3, when the operator remotely controls the garbage collection and sorting vehicle 10 to move in the site 11, the movement trajectory covers the site 11 at least twice. In the backend optimization of the Cartographer algorithm, a loop constraint is established by real-time detection of the spatial matching between historical nodes and current nodes. When the garbage truck repeatedly passes through the map area, the Cartographer algorithm will generate edges connecting the old and new nodes and use graph optimization techniques (such as SPA or pose graph optimization) to globally adjust the pose of all nodes. The loop constraint eliminates the cumulative error of the movement trajectory by fusing real-time filtered pose data and real-time filtered two-dimensional grid map.

[0094] The cumulative error includes the errors in the filtered pose data and the filtered two-dimensional grid map caused by the slippage of the garbage collection and sorting vehicle body 10.

[0095] Specifically, step S4 involves the following steps:

[0096] Step S41, import the global map;

[0097] Step S42: The garbage collection and sorting vehicle 10 executes the inner spiral covering algorithm. Starting from a corner of the perimeter of the site 11, the garbage collection and sorting vehicle 10 forms the first zigzag path along the outer boundary; as shown in the attached diagram. Figure 2 As shown.

[0098] After completing each layer of the U-shaped path, the garbage collection and sorting vehicle 10 shrinks inward by a certain distance based on the minimum safe passage distance between obstacles in the site 11, forming a new nested U-shaped layer (the distance between adjacent U-shaped layers is 10cm).

[0099] Global path 12 is the path composed of all nested back-to-back layers; as shown in the appendix. Figure 2 As shown, the arrow indicates the direction of movement of the garbage collection and sorting vehicle body 10;

[0100] Among them, when the garbage collection and sorting vehicle body 10 changes its direction of movement, the path smoothness is optimized by B-spline interpolation to eliminate sawtooth corners in the path;

[0101] When the garbage collection and sorting vehicle 10 approaches the blind spot 14 at the corner, the host computer calls a search algorithm to generate an escape path; the escape path is the route the garbage collection and sorting vehicle 10 takes when it encounters a dead end and returns to its original path; as shown in the attached diagram. Figure 2 As shown.

[0102] During the movement of the waste collection and sorting vehicle 10, the depth camera updates the local cost map (costmap_2d) in ROS in real time, expands the obstacle region 15, and ensures that the safe distance between the path and the obstacle region 15 is not less than 20cm; see attached. Figure 2 As shown.

[0103] When a new obstacle is detected compared to the global map, global path 12 is paused, and the DWA algorithm is called to generate obstacle avoidance trajectory 13 (minimum turning radius 15cm). Obstacle avoidance trajectory 13 is a local path. After the local path planning is completed, the garbage collection and sorting vehicle body 10 moves according to the local path, avoids the obstacle, and then moves according to global path 12 again. (See attached...) Figure 2 As shown.

[0104] When the garbage collection and sorting vehicle 10 detects a suspected garbage target, it stops moving and judges the suspected garbage target. In the host computer 2, YOLOv8 fuses the RGB image and the depth map and outputs whether the suspected garbage target is garbage.

[0105] If the target suspected to be trash is indeed trash, output the trash category and its three-dimensional coordinates;

[0106] Among them, the RGB image and depth map are RGB-D point clouds output by the depth camera 4; the waste categories include "recyclable", "kitchen waste", "hazardous", and "other"; the three-dimensional coordinates of the waste are transformed from the TF coordinate system in ROS;

[0107] If the suspected object is not actually trash, it will be treated as a newly added obstacle.

[0108] The host computer 2 verifies the size of the garbage (grip limit ≤ 20cm³). If the size of the garbage exceeds the gripper limit of the robotic arm 7, it will avoid obstacles and move along the global path 12 again after avoiding the obstacles. If the size of the garbage does not exceed the gripper limit of the robotic arm 7, it will transmit the three-dimensional coordinates of the garbage, and the robotic arm 7 will grab the garbage and put it into storage.

[0109] When the garbage collection and sorting vehicle 10 finally reaches the center point of the innermost layer of the area and completes full coverage of the site 11, the task ends when the garbage in the site 11 has been collected and sorted.

[0110] Specifically, a backtracking algorithm, a type of search algorithm, is used to generate an escape path. The backtracking algorithm is a type of search algorithm.

[0111] Based on the same inventive concept, this embodiment also provides a computer device, which includes a memory and a processor, and the memory and the processor are communicatively connected to each other. The memory stores computer instructions, and the processor executes the aforementioned intelligent cruise garbage collection and sorting method by executing the computer instructions.

[0112] Based on the same inventive concept, this embodiment also provides a computer-readable storage medium storing computer instructions for causing a computer to execute the aforementioned intelligent cruise garbage collection and sorting method.

[0113] The intelligent cruise garbage collection and sorting vehicle and method provided in this embodiment have the following advantages compared with the prior art:

[0114] Multimodal perception fusion enhances environmental understanding capabilities

[0115] By leveraging heterogeneous data from LiDAR 3 and depth camera 4, combined with Cartographer and extended Kalman filter algorithms, the limitations of a single sensor are effectively eliminated. LiDAR 3 provides high-precision two-dimensional spatial information, while depth camera 4 supplements with three-dimensional geometric and visual semantic features, significantly enhancing the comprehensive perception of waste material, shape, and volume. This fusion not only solves the problem of misjudging waste material in traditional solutions but also supports the adaptive grasping strategy of robotic arm 7, reducing the risk of grasping failure due to environmental complexity.

[0116] Redundant architecture ensures system reliability

[0117] A dual-bus communication redundancy design ensures stable transmission of critical signals (such as emergency stop commands) and avoids the risk of single-point failure. A hierarchical data processing architecture decouples the lower-level motion control (lower-level machine 1) from the higher-level decision-making and planning (upper-level machine 2), guaranteeing both real-time performance and improved computing resource utilization efficiency. This design enables the system to maintain stable operation even in complex dynamic environments.

[0118] Adaptive algorithms optimize job performance

[0119] The combination of the inner spiral coverage algorithm, DWA, and backtracking algorithm achieves more efficient full-coverage path planning and reduces repeated traversal areas through path smoothing and dynamic adjustment of safety spacing. The mapping method, which integrates closed-loop constraint mechanisms and multi-source data fusion, effectively suppresses accumulated errors during long-term operations and improves the accuracy of large-scene mapping. The coordinated design of local obstacle avoidance and obstacle escape strategies further enhances the system's adaptability to sudden obstacles.

[0120] Integrated design reduces deployment costs

[0121] The complementary functions of the depth camera 4 and the LiDAR 3 avoid the redundant configuration of dedicated image processing units in traditional solutions, enabling multimodal data processing within a single host computer 2 architecture. The six-DOF robotic arm 7, through coordinate system transformation and vision-guided coordinated control, supports the precise grasping of diverse waste shapes, significantly reducing the need for manual intervention, and resulting in a more compact and cost-effective overall hardware architecture.

Claims

1. An intelligent cruise garbage collection and sorting vehicle, characterized in that, include: The host computer runs ROS on the Jetson Orin Nano embedded platform; the ROS deploys mapping algorithms, the YOLOv8 garbage identification and classification model, the inner spiral cover algorithm, the search algorithm, and the DWA algorithm. A lower-level machine, which is connected to the upper-level machine; The lidar is connected to the host computer; A depth camera, which is connected to the host computer; An odometer, connected to the lower-level machine, is used to collect the movement speed of the waste collection and sorting vehicle. The odometer transmits the movement speed of the waste collection and sorting vehicle to the lower-level machine, which processes the speed data to obtain the XY axis coordinates P of the waste collection and sorting vehicle in the three-dimensional coordinate system of the global map. XY and the angle θ of rotation around the Z-axis 里程计 ; In the three-dimensional coordinate system, the X-axis extends along the front of the vehicle, the Y-axis is perpendicular to the front of the vehicle, and the Z-axis is perpendicular to the plane formed by the X-axis and the Y-axis; the origin of the three-dimensional coordinate system is the starting point for the construction of the global map. An IMU is connected with the lower computer, and the IMU is used for collecting XYZ axis angular acceleration and angular velocity of the garbage pickup and classification vehicle body in a three-dimensional coordinate system, and obtaining an angle θ of rotation of the garbage pickup and classification vehicle body around the Z axis IMU ; A robotic arm, which is mounted on the main body of the waste collection and sorting vehicle, is used for waste collection and sorting; A wireless communication port is provided, which is connected to the host computer. The wireless communication port is used to receive remote control signals, allowing the operator to remotely control the garbage collection and sorting vehicle.

2. The intelligent cruise garbage collection and sorting vehicle according to claim 1, characterized in that, The lower-level machine and the upper-level machine are connected via a serial bus and a CAN bus to achieve communication redundancy. The emergency stop signal is transmitted through dual channels of the serial bus and the CAN bus to reduce the packet loss rate during communication.

3. A method for intelligent cruise garbage collection and sorting, implemented based on the intelligent cruise garbage collection and sorting vehicle described in claim 1 or 2, characterized in that, Including the following steps: Step S1: The lidar acquires 360° two-dimensional point cloud data; RGB-D point clouds are obtained using the depth camera; The speed of the waste collection and sorting vehicle is obtained through the odometer and transmitted to the lower-level computer. The lower-level computer processes the speed of the waste collection and sorting vehicle to obtain the XY axis coordinates P of the waste collection and sorting vehicle in the three-dimensional coordinate system. XY and the angle θ of rotation around the Z-axis 里程计 ; The angle θ of the rotation of the garbage sorting vehicle body around the Z-axis is obtained through the IMU. IMU , Step S2 involves processing the two-dimensional point cloud data, the RGB-D point cloud, and the XY-axis coordinates P of the waste collection and sorting vehicle body obtained by the lower-level machine from the movement speed of the waste collection and sorting vehicle body transmitted from the odometer. XY θ, the angle of rotation around the Z-axis 里程计 And the angle θ of the rotation of the garbage collection and sorting vehicle body around the Z-axis obtained by the IMU. IMU Perform data processing to generate a filtered 2D raster map and filtered pose data; Step S3: When the operator remotely controls the garbage collection and sorting vehicle to move in the field, the movement trajectory covers the field at least twice; during the movement of the garbage collection and sorting vehicle in the field, a mapping algorithm is executed based on the filtered two-dimensional grid map and the filtered pose data to generate the global map. In step S4, the garbage collection and sorting vehicle autonomously navigates based on the global map using the inner spiral cover algorithm, the DWA algorithm, and the search algorithm to perform the task of garbage collection and sorting.

4. The intelligent cruise garbage collection and sorting method according to claim 3, characterized in that, The mapping algorithm in step S3 is the Cartographer algorithm.

5. The intelligent cruise garbage collection and sorting method according to claim 4, characterized in that, Step S2 specifically involves the following steps: Step S21: Use the point cloud library PCL in ROS to project the 3D depth data onto the initial two-dimensional raster map to obtain two-dimensional planar data; Step S22: The two-dimensional planar data and the initial two-dimensional raster map are subjected to mean filtering to eliminate isolated noise points and obtain the filtered two-dimensional raster map. Step S23, the θ 里程计 With the θ IMU Extended Kalman filtering is performed to generate the angle θ of the filtered rotation of the waste sorting vehicle around the Z-axis. 滤波后 The angle θ of the filtered body of the garbage collection and sorting vehicle around the Z-axis 滤波后 With the P XY Combined, the filtered pose data of the garbage collection and sorting vehicle is generated.

6. The intelligent cruise garbage collection and sorting method according to claim 5, characterized in that, Step S3 specifically involves the following steps: The pose of the garbage collection and sorting vehicle is used as a node in the Cartographer algorithm; The transformation relationships between the nodes are used as edges in the Cartographer algorithm; Introducing closed-loop constraints in the backend of the Cartographer algorithm to eliminate accumulated errors solves the problem of mapping large-scale scenes; The large-scale scene mapping refers to a large mapping scene; In the backend optimization of the Cartographer algorithm, a closed-loop constraint is established by real-time detection of the spatial matching between historical nodes and the current node. When the garbage truck repeatedly passes through the map area, the Cartographer algorithm will generate edges connecting the old and new nodes and use graph optimization technology to globally adjust the pose of all nodes. The closed-loop constraint eliminates the cumulative error of the motion trajectory by fusing the real-time filtered pose data and the real-time filtered two-dimensional grid map. The cumulative error refers to the error in the filtered pose data and the filtered two-dimensional grid map caused by the slippage of the garbage collection and sorting vehicle.

7. The intelligent cruise garbage collection and sorting method according to claim 6, characterized in that, Step S4 specifically involves the following steps: Step S41: Import the global map; Step S42: The garbage collection and sorting vehicle body executes the inner spiral covering algorithm. The garbage collection and sorting vehicle body starts from a corner of the outer perimeter of the site as the starting point of the inner spiral covering algorithm and forms the first zigzag path along the outer boundary. After completing each layer of the U-shaped path, the garbage collection and sorting vehicle shrinks inward by a certain distance based on the minimum safe passage distance between obstacles in the site, forming a new nested U-shaped layer. Among them, the path of the garbage collection and sorting vehicle body executing the inner spiral covering algorithm is the path composed of the nested back-shaped layers; The path formed by the nested back-word layers is a global path; Among them, when the garbage collection and sorting vehicle changes its direction of movement, the path smoothness is optimized by B-spline interpolation to eliminate sawtooth corners in the path; When the garbage collection and sorting vehicle approaches a blind spot in a corner, the host computer calls the search algorithm to generate an escape path. During the movement of the garbage collection and sorting vehicle, the depth camera updates the local cost map in the ROS in real time, expands the obstacle area, and ensures that the safe distance between the path and the obstacle area is not less than 20cm. When an obstacle is detected that is newly added compared to the global map, the global path is paused and the DWA algorithm is called to generate an obstacle avoidance trajectory, which is a local path. After the local path is planned, the garbage collection and sorting vehicle moves according to the local path, avoids the obstacle, and then moves according to the global path again. When the garbage collection and sorting vehicle detects a suspected garbage target, it stops moving and judges the suspected garbage target. In the host computer, YOLOv8 merges the RGB image and the depth map and outputs whether the suspected garbage target is garbage. If the suspected trash target is indeed trash, then the trash category and 3D coordinates are output; wherein, the RGB image and depth map are the RGB-D point cloud output by the depth camera; if the suspected trash target is not trash, then it is handled as a newly added obstacle. The host computer verifies the size of the waste. If the size of the waste exceeds the gripping limit of the robotic arm, obstacle avoidance is performed. After avoiding the obstacle, the robot moves along the global path again. If the size of the waste does not exceed the gripping limit of the robotic arm, the three-dimensional coordinates of the waste are returned, and the robotic arm grabs the waste and puts it into storage. When the garbage collection and sorting vehicle finally reaches the center point of the innermost layer of the area, completing full coverage of the site, the task ends once the garbage in the site has been collected and sorted.

8. The intelligent cruise garbage collection and sorting method according to claim 7, characterized in that, The escape path is generated using the backtracking algorithm in the search algorithm.

9. A computer device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the intelligent cruise garbage collection and sorting method as described in any one of claims 3-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the intelligent cruise garbage collection and sorting method as described in any one of claims 3-8.

Images