An intelligent path planning and cargo side delivery method based on ROS and an intelligent car

Abstract

The present invention discloses a ROS-based intelligent path planning and cargo side-dropping method and an intelligent trolley. First, a downward suction cup bracket with a camera is printed and installed at the end of the intelligent trolley's robotic arm. A global path based on Dijkstra and a local path based on Transparent Evolutionary Block (TEB) are used to navigate the intelligent trolley to a sorting platform for parallel correction to make it parallel to the sorting platform. If the arm span of the intelligent trolley's robotic arm is less than the required grasping reach, the trolley's posture is corrected by a triangulated path planning method. The robotic arm is controlled to extend at a specific angle and a specific distance for deflection angle grasping. A two-stage recognition of cargo QR codes is performed using a deep neural network-based QR code enhancement and recognition algorithm. Finally, based on all cargo delivery locations, the optimal navigation path is planned and the lateral delivery direction is determined to achieve target cargo grasping and delivery. The method of the present invention optimizes the accuracy and efficiency of the cargo grasping process, effectively improving the overall performance of the intelligent path planning and cargo side-dropping method.

Description

Technical Field

[0001] The present invention relates to the field of path planning, intelligent sorting and identification technology for logistics intelligent trolleys, and in particular to a ROS-based intelligent path planning and cargo side delivery method and an intelligent trolley. Background Art

[0002] In modern logistics and warehousing, the integration of ROS intelligent vehicle navigation, sorting, and recognition technologies is key to achieving automated and efficient management. Traditional robot navigation methods primarily involve global and local path planning. The Dijkstra algorithm is commonly used for global path planning, but it has limitations when dealing with dynamic environments. The Timed Elastic Band (TEB) algorithm excels in local path planning and can effectively handle dynamic obstacles.

[0003] At the same time, sorting and recognition technology realizes automatic classification and recognition through image processing and sensor data, further improving the intelligence level of the ROS smart car system.

[0004] Therefore, the comprehensive application of image processing and recognition technology and path planning technology in the logistics industry for mail sorting and delivery can significantly improve the overall performance and adaptability of the ROS intelligent vehicle system, and further improve the efficiency and accuracy of mail sorting and delivery in the logistics industry. Summary of the Invention

[0005] The problem to be solved by the present invention is to provide an intelligent path planning and cargo side delivery method and an intelligent vehicle based on ROS, which are used to improve the efficiency and accuracy of mail sorting and delivery in the logistics industry.

[0006] The present invention adopts the following technical solution: a ROS-based intelligent path planning and cargo side delivery method, comprising the following steps:

[0007] S1. Use 3D printing technology to print a downward suction cup bracket with a camera. Install it at the end of the smart car's robotic arm. The camera faces horizontally and is used to identify the QR code on the side of the target cargo.

[0008] S2. Build an intelligent map, use the GMAPPING method and the adaptive Monte Carlo positioning method to build and locate the intelligent car, and perform intelligent map cropping to improve the overall processing speed and reduce positioning errors;

[0009] S3. Navigate the smart cart to the target cargo sorting platform using a Dijkstra-based global path planning and TEB-based local path planning algorithm: pre-define the location and orientation of the sorting platform in the global map, obtain the cart's current orientation based on the odometer, calculate the deflection angle between the cart and the sorting platform, and reverse the deflection angle to make the smart cart parallel to the sorting platform, achieving parallel correction and facilitating lateral recognition and grasping by the smart cart.

[0010] S4: Control the smart car to move forward and backward until it can recognize the April tag code of the target item on the sorting table, stop the car, and grab the target item. Based on the location information of the target item, use trigonometric functions and geometric principles to calculate the extension length and deflection angle required for the smart car's robotic arm to grab the current target item. If the extension length or deflection angle exceeds the maximum range that the current robotic arm can reach, the robotic arm is unreachable. Then, perform at least one trigonometric path planning and parallel correction on the smart car's posture until the grabbing requirements are met.

[0011] S5. Calculate the coordinates of the cargo center in the three-dimensional coordinate system of the robot arm chassis based on the deflection angle and distance from the center of the suction cup of the robot arm to the cargo center. Control the robot arm to grab the cargo at this coordinate to achieve deflection angle grabbing.

[0012] S6. A two-stage recognition strategy is used to identify the target cargo QR code. In each stage, the recognition algorithm first enhances the QR code image using a super-resolution method based on residual and convolutional projection. Then, a method based on a deep convolutional network is used to recognize the QR code to obtain the address information of the target cargo to be delivered, including the province, city, and district information.

[0013] S7. Plan the delivery route of the goods based on the address information corresponding to the grabbed goods. Preset the lateral delivery direction of each delivery point based on different delivery plans, thereby controlling the smart car to deliver the goods sideways one by one.

[0014] Preferably, in step S1, the downward suction cup bracket with a camera is installed at the front end of the intelligent car's robotic arm, and a male head of a corresponding shape is drawn according to the interface on the intelligent car. It includes a main bracket, a suction cup connection module, and a camera module; the main bracket and the suction cup connection module are connected by bamboo fiber material, and the suction cup connection module is fixed with a top screw through the screw hole reserved on the main bracket, and is used to suck the target cargo. The suction cup air outlet in the suction cup connection module is placed on the side to reduce the height of the suction cup module.

[0015] Preferably, in step S2, the GMAPPING method uses the odometer data of the smart car in combination with the distance information of the laser radar scan to construct an environmental map in real time; the adaptive Monte Carlo positioning method matches the odometer and laser radar scan information with the environmental information in the map to determine the optimal particles to represent the current position and posture of the smart car; at the same time, the intelligent map clipping method is used to clip irrelevant areas, and the size of the map is reduced by identifying and eliminating the map edges and areas that the smart car cannot reach or are irrelevant.

[0016] Preferably, in step S3, global path planning based on Dijkstra and local path planning based on TEB are used to navigate the smart car to the target cargo sorting platform; the optimal time weight of the TEB algorithm parameter is set so that the smart car speeds up on the straight road and turns tangently close to the edge of the path to adapt to the cargo sorting requirements; the optimization weight parameter for shrinking the trajectory according to the conversion / execution time is reduced, and known obstacles and unknown obstacles are identified by adjusting the forward planning maximum distance parameter; the robot arm parameter is increased to make the smart car move at the fastest speed and improve the grasping and delivery execution speed; parallel correction is performed before the smart car arrives at the sorting platform to ensure that the smart car and the sorting platform remain parallel.

[0017] Preferably, in step S4, the posture of the smart car is adjusted using triangular path planning, which includes the following sub-steps:

[0018] S4.1. Control the smart car to rotate α degrees clockwise. Assume that the distance the smart car needs to adjust is ybias, and the horizontal deviation between the car and the center point of the target cargo is xbias.

[0019] S4.2. Control the smart car to follow a triangular path, correct the deviation between the car and the target cargo, and control the smart car to move forward a distance u, rotate counterclockwise α degrees, and then move back a distance v:

[0020]

[0021] S4.3. When the required extension distance and deflection angle of the robotic arm are still not achievable, repeat steps S4.1 to S4.2 until the required extension distance and deflection angle of the intelligent trolley robotic arm are within the achievable range. Perform parallel correction again to ensure that the intelligent trolley and the sorting platform remain parallel, thereby improving the success rate of grasping.

[0022] Preferably, in step S5, the deflection angle grasping specifically refers to calculating the position of the center point of the grasped goods by the position of the center point of the Apriltag code when the grasping extension distance required by the smart car is less than or equal to the maximum extension distance of the robot arm; performing the deflection angle grasping specifically refers to assuming that all goods have the same size, and the x-axis offset between the center point of the camera at the current position of the smart car and the center point of the Apriltag code is d x , the y-axis offset is d y The measured x-axis offset of the Apriltag code center point from the target cargo center point is x1, and the y-axis offset is y1. Therefore, the x-coordinate of the current position of the robot arm needs to be increased by x1-d. x , the y coordinate increases by y1-d y , that is, to move the end of the robot arm to the three-dimensional coordinate system with the base of the robot arm as the origin of the coordinate system (x0+x1-d x , y0+y1-d y , z0), where x0, y0, and z0 are the three-dimensional coordinates of the center point of the suction cup of the robotic arm before grasping in the current coordinate system.

[0023] Preferably, in step S6, a two-stage recognition of the target goods QR code is performed:

[0024] In the first stage, the distance between the smart cart and the sorting platform is determined by identifying the Apriltag code. The robot arm is then controlled to move to the preset recognition position to ensure that the QR code captured by the fixed-focus camera is in clear focus. As the robot arm moves, the camera repeatedly captures and recognizes the QR code image on the goods. After reaching the recognition position, the camera continues to recognize the QR code until the email address information contained in the QR code is recognized or the recognition time exceeds the preset threshold. The robot arm then extends to the grasping position, and the first stage of recognition ends.

[0025] If the email address information is not recognized in the first stage, QR code recognition will be required again in the second stage. In the second stage, after grabbing the goods, the robotic arm moves the robotic arm and places the goods on the loading platform of the smart car. During the movement, the camera will continue to recognize the QR code multiple times until the delivery address information is successfully recognized; if the email address information has been recognized in the first stage, the second stage recognition step will be skipped for quick grabbing.

[0026] Preferably, in step S7, different delivery path plans are selected according to different distribution situations of cargo locations, and cargo delivery is performed by a lateral delivery method, which specifically includes the following sub-steps:

[0027] S7.1. Delivery Path Planning: Based on the address information identified by the QR code, the shortest path is planned from the current smart car location to each target delivery location until the next pickup location is reached, and the goods are delivered in sequence;

[0028] S7.2. Delivery Sequence and Side-Delivery Direction Planning: Assuming that the target delivery points can be connected to form an arbitrary polygon, regardless of straight lines, two delivery plans can be planned: clockwise and counterclockwise. Deliver to each target delivery point in the polygonal area in sequence according to the planned path. A left-side delivery plan is used for clockwise delivery, and a right-side delivery plan is used for counterclockwise delivery.

[0029] S7.3. During the delivery process, assuming that multiple target locations that need to be delivered are located in a straight line, the intelligent vehicle can be directly controlled to make parallel corrections. For all delivery points on the straight line, the intelligent vehicle can be directly controlled to move forward or backward a specific distance and then rotate the robotic arm to achieve delivery, replacing the original navigation solution and speeding up the delivery.

[0030] The technical solution of the present invention also provides: a ROS-based intelligent path planning and cargo side-dropping intelligent vehicle, which is equipped with a downward suction cup bracket with a camera on the vehicle's robotic arm and uses the above method to perform path planning and cargo side-dropping, including: a mapping navigation module, a posture adjustment module, a robotic arm control and grasping module, and a ROS intelligent vehicle vision module;

[0031] The mapping and navigation module uses the GMAPPING method to build maps and the AMCL method to locate the intelligent vehicle;

[0032] The posture adjustment module is used to obtain the current posture of the intelligent car and perform fine posture adjustment through the triangulation path planning method;

[0033] The robotic arm control and grasping module, through hardware modification, installs a downward suction cup bracket at the front end of the intelligent robot arm to perform optimal path planning for the robotic arm, and controls the rotation angle of the robotic arm to grasp the goods and drop the goods to the designated area;

[0034] The ROS smart car vision module is used to obtain target cargo images, perform image preprocessing, and use a deep neural network-based QR code enhancement and recognition method to identify the target cargo QR code.

[0035] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0036] 1. Local path optimization: Based on the global path, the method of the present invention applies the TEB algorithm to perform local path optimization. The method focuses on adjusting the TEB algorithm parameters to adapt to the needs of cargo sorting, reducing the optimization weight parameter for shrinking the trajectory based on the conversion / execution time, increasing the forward planning maximum distance parameter, and identifying known obstacles and unknown dynamic or static obstacles.

[0037] 2. Multiple error adjustment methods: The method of the present invention determines the position through positioning code recognition and selects different correction methods when the error size is different, thereby ensuring that the goods can be grasped stably while increasing the speed as much as possible.

[0038] 3. Multi-stage intelligent recognition: The method of the present invention combines CNN image recognition technology and two-stage QR code recognition. The ROS smart car can quickly and accurately identify the QR code of the goods to be grabbed.

[0039] 4. Autonomous navigation path optimization: The method of the present invention plans the optimal navigation path before delivery by capturing the location distribution information of the goods, avoiding repeated navigation and improving the working efficiency of the ROS smart car. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a block diagram of the ROS-based intelligent path planning and cargo side-drop ROS smart car module of the present invention;

[0041] Figure 2 This is a block diagram of the GMAPPING method mapping process module of the mapping navigation module of the present invention;

[0042] Figure 3 This is a structural diagram of the 3D-printed downward suction cup bracket with a camera of the present invention;

[0043] Figure 4 This is a structural diagram of the suction cup connection module in the downward suction cup bracket of the present invention;

[0044] Figure 5 This is a structural diagram of the top end of the main bracket in the downward suction type suction cup bracket of the present invention;

[0045] Figure 6 This is a flow chart of the autonomous navigation path of the present invention;

[0046] Figure 7 This is a schematic diagram of the robot arm's deflection angle grasping when the ROS smart car of the present invention is not aligned with the mailbox;

[0047] Figure 8 Schematic diagram of the distance and deflection angle grasping between the ROS smart car and the counter and mailbox of the present invention;

[0048] Figure 9 This is a schematic diagram of the ROS intelligent vehicle triangulation correction of the present invention;

[0049] Figure 10 This is a network structure diagram of the QR code enhancement model of the present invention;

[0050] Figure 11 This is a schematic diagram of the QR code recognition principle based on convolutional neural network in the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the application are further elaborated in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in this field on this embodiment fall within the scope of protection of the present invention. At the same time, the step numbers in the embodiments of the present invention are only set for the convenience of explanation and description, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0052] In one embodiment of the present invention, a smart car is provided that is controlled by an intelligent path planning and cargo side-dropping method based on ROS. By installing a downward suction cup bracket with a camera on the robot arm of the car, path planning and cargo side-dropping are performed. Figure 1 As shown, the ROS smart car includes: mapping and navigation module, posture adjustment module, robotic arm control and grasping module, and ROS smart car vision module;

[0053] The mapping and navigation module uses the GMAPPING method to build maps, uses the Adaptive Monte Carlo Localization (AMCL) method to locate the intelligent vehicle, and performs intelligent map cropping.

[0054] Specifically, the GMAPPING method uses data from lidar and odometry, such as Figure 2 As shown in the figure, the odometer data is first used to calculate the posture of the ROS smart car, and then the distance information scanned by the lidar is combined to locate the relative position of the obstacle to the smart car, completing the map construction;

[0055] The posture adjustment module is used to obtain the current posture of the intelligent car and perform fine posture adjustment through the triangulation path planning method;

[0056] The robotic arm control and grasping module, through hardware modification, installs a downward suction cup bracket at the front end of the intelligent robot arm to plan the optimal path of the robotic arm and control the rotation angle of the robotic arm to grasp the goods. If there are goods placed in the same horizontal position, they can be directly moved horizontally for quick grasping.

[0057] The ROS smart car vision module is used to obtain target cargo images, perform image preprocessing, and use the convolutional neural network-based QR code recognition and enhancement method to recognize the QR code.

[0058] The method for controlling the ROS smart car to perform path planning and cargo side delivery includes the following steps:

[0059] Step 1: 3D print a downward suction cup bracket with a camera, with the camera facing horizontally so that it can be aligned with the QR code on the side of the target product for recognition;

[0060] like Figure 3 As shown, a male connector of corresponding shape is drawn according to the interface on the robot smart car. The position of the camera is offset according to the position of the QR code on the mail box to improve the vertical positioning tolerance. The suction cup air outlet is placed on the side to reduce the height of the suction cup module.

[0061] Downward suction cup bracket, such as Figure 4 As shown, it includes: a main bracket and a suction cup connection module. The connection between the main bracket and the suction cup connection module is made of lightweight, high-toughness, low-cost, and ubiquitous bamboo fiber material.

[0062] Further, if Figure 5 As shown, in this embodiment, the suction cup connection module is fixed using screws through screw holes reserved on the main bracket, which reduces the damage rate and facilitates replacement and maintenance at any time.

[0063] Step 2: Build a map using the mapping and navigation module, build the map using the GMAPPING method, localize the intelligent vehicle using the AMCL method, and perform intelligent map cropping to improve overall processing speed and reduce positioning errors.

[0064] The GMAPPING method uses the smart car's odometer data combined with distance information from LiDAR scans to construct an environmental map in real time. The Adaptive Monte Carlo Localization (AMCL) method matches the odometer and LiDAR scan information with the environmental information in the map to determine the optimal particle to represent the smart car's current position and posture. Simultaneously, intelligent map clipping technology is used to accurately clip irrelevant areas. By identifying and removing map edges and areas that are inaccessible or irrelevant to the smart car, the map size is significantly reduced. This technology not only effectively reduces the system's computational workload during map construction and localization, improving overall processing speed, but also prevents positioning errors in boundary areas. By clipping out invalid areas, the system can focus more on environmental information within the core path and working range, thereby enhancing positioning accuracy, reducing the risk of error propagation, and ultimately improving the efficiency and reliability of the entire path planning system, especially in complex or dynamic environments.

[0065] Step 3: Use Dijkstra-based global and TEB (Timed Elastic Band)-based local path planning to enable the ROS smart car to automatically navigate to the target point. In order to facilitate the grabbing of goods on the sorting table, the position and orientation of the sorting table are first pre-defined in the global map. Then, the current orientation of the car is obtained according to the odometer. Then, the deflection angle between the car and the sorting table is calculated. By rotating the car in the opposite direction, the smart car can be made parallel to the sorting table.

[0066] In particular, the optimal time weight of the TEB algorithm parameters is set, so that the smart car speeds up on straight roads and makes tangential turns close to the edge of the path to adapt to the needs of cargo sorting. The optimization weight parameter for shrinking the trajectory according to the conversion / execution time is reduced, and the maximum forward planning distance parameter is adjusted to identify known and unknown obstacles.

[0067] In this embodiment, weight_optimaltime is set to the optimal time weight. When weight_optimaltime is increased to 5, the vehicle's linear speed increases significantly on the straight road. The larger the value, the closer it is to max_vel_x, and the vehicle turns along the tangent line close to the edge of the path.

[0068] We further set the maximum lookahead distance (max_global_plan_lookahead_dist) to 1.5, specifying the maximum length (cumulative Euclidean distance) of the global plan subset to be optimized. The actual length is then determined by a logical combination of the local costmap size and this maximum bound. Setting it to zero or a negative number disables this limit. This distance should 1. increase with the vehicle's maximum speed 2. not exceed the reliable measurement range of sensors such as lidar 3. and should not exceed the size of the local costmap, meaning the TEB cannot be required to plan beyond the local costmap.

[0069] like Figure 6 As shown, the smart car starts navigating. It first searches for a temporary target cargo point. If so, it uses the temporary target cargo coordinates as the navigation target. Otherwise, it obtains the sorting platform coordinates from the coordinate file as the navigation target, adjusts the smart car coordinates to align with the target mail box, moves to the target point, and saves the current coordinates as the temporary coordinate point.

[0070] Before the smart car reaches the sorting platform, it will perform parallel correction to ensure that it remains parallel to the sorting platform. This includes the following sub-steps:

[0071] S3.1. Read the data from the smart car's attitude sensor. The data format is Euler quaternion (w, x, y, z).

[0072] S3.2, convert the attitude data expressed in the form of Euler quaternion into angle data (yaw x ,yaw y ,yaw z ), yaw x ,yaw y ,yaw z Respectively represent the angles between the current posture of the smart car and the positive directions of the x, y, and z axes;

[0073] S3.3. Target attitude angle when the smart car is parallel to the sorting platform x Subtract the current attitude angle yaw of the smart car obtained in step S3.2 x , find the deviation angle yawbias = target x -yaw x , issue the command rot(yawbias,speed,torlance) to the smart car, which means the smart car rotates yawbias degrees at speed with an allowable error of torlance;

[0074] S3.4. Due to the rotation accuracy error of the robot, parallel correction may not be achieved in one go. In this case, steps S3.1-S3.3 will be repeatedly executed. The speed and torlance values ​​will be reduced each time to gradually converge the yawbias value. When the yawbias value is less than 1, parallel correction is completed.

[0075] During the navigation process, first, Dijkstra global path planning is used to generate a fixed optimal global path;

[0076] Preferably, this embodiment uses the Dijkstra algorithm to initialize the set (representing the set of vertices for which the shortest path has been determined), and a priority queue Q (representing the set of vertices that can be used to optimize the path), by continuously taking a vertex u from Q, and using the distance w from u to the adjacent vertex to update the shortest path dist(v) of all vertices adjacent to u in S until the queue Q is empty:

[0077] dist(v)=min(dist(v),dist(u)+w(u,v))

[0078] Where w(u,v) represents the distance between vertices u and v, and dist(·) represents the shortest path planned;

[0079] The classic Dijkstra algorithm does not consider the existence of obstacles around the path. The Dijkstra algorithm of this embodiment improves this by introducing a medium cost neutral_cost to make the planned shortest path away from obstacles:

[0080] dist(v)=min(dist(v),dist(u)+w(u,v)+neutrl_cost*obstacle(u,v))

[0081] Among them, obstacle(u,v) represents the sum of the distances from all obstacles adjacent to the path between u and v to the path.

[0082] Specifically, for a path from u to v, let the path be e(u,v), dist(e(u,v),x) represents the distance from obstacle x to path e, then:

[0083] obstacle(u,v)=∑dist(e(u,v),x i )

[0084] This embodiment modifies the neutral_cost of the Dijkstra algorithm in ROS. Compared with the original algorithm, the modified algorithm plans a path with a larger turning radius, which can effectively improve the turning efficiency of the ROS smart car.

[0085] Then, during the navigation process, the local path planning is continuously modified through decision factors such as obstacle avoidance, and finally the target point is successfully navigated.

[0086] The Transitional Electric Boundary (TEB) algorithm is used to subsequently correct the initial global trajectory generated by Dijkstra's global path planner, optimizing the ROS smart car's trajectory. Specifically, the TEB algorithm divides the ROS smart car's motion into two dimensions: linear velocity v and angular velocity ω. It then assigns a score to each velocity combination to search for the optimal motion strategy. These scores comprehensively consider factors such as target direction, obstacle avoidance capability, and obstacle distance, ensuring that the ROS smart car can decelerate in time to avoid collisions. Furthermore, the TEB algorithm parameters were specifically adjusted to accommodate cargo sorting needs. The optimization weight parameter for shrinking the trajectory based on conversion / execution time was reduced, the maximum forward planning distance parameter was increased, and the identification of known obstacles and unknown dynamic or static obstacles was achieved.

[0087] Step 4: Use Apriltag image recognition to assist in locating the target mail box and perform pose and grasping position correction.

[0088] This example uses the ROS_Apriltag software package and the apriltags2_ros node to implement Apriltag image recognition. The apriltags2_ros node subscribes to two topic messages: the original image topic and the camera information topic containing calibration matrix information. After completing Apriltag image recognition, the apriltags2_ros node publishes three topic messages: a standard TF topic containing TF information, a topic containing position and quaternion attitude information, and an image topic containing the highlighted image after detection. After obtaining the location information of the Apriltag image, the position of the mailbox can be accurately located.

[0089] When the error between the ROS smart car and the mail box is within 3 cm, this embodiment uses the distance and angle between the ROS smart car and the position calibration code to calculate the location of the mail, and uses the adjustment of the robotic arm's grasping angle to compensate for the horizontal offset error of the ROS smart car, avoiding further adjustment and saving time.

[0090] When the ROS smart car is not aligned with the mail box and the error is within 3cm, Figure 7 As shown in (a) in the figure, the deflection angle is designed to make the robot arm deflect and grasp, as shown in Figure 7 The specific operation is as follows: Assume that the offset between the camera and the Apriltag code when the ROS smart car navigates to the counter is d x ,like Figure 8 As shown in (a), the standard error of the Apriltag code from the mailbox is x, and the distance from the mailbox to the counter is d. y , the deflection angle of the robotic arm of the ROS smart car is controlled as: Perform horizontal offset grabbing, such as Figure 8 As shown in (b) in .

[0091] Step 5: Use triangulated path planning to correct deviations to correct larger errors. When the horizontal deviation between the robot and the mail box is large, the robot arm cannot grasp the goods by deflecting the angle. At this time, based on the distance information from the ROS smart car to the sorting platform, triangulated path adjustment is used to further refine the posture of the ROS smart car.

[0092] First, let the car rotate α degrees clockwise, as shown in Figure 9 As shown in the figure, the distance that needs to be adjusted is ybias (cm). Assuming that the horizontal deviation between the car and the center point of the mailbox is xbias (cm), the deviation between the car and the mailbox can be corrected by controlling the ROS smart car to follow a triangular path.

[0093] The ROS smart car moves forward a distance u (cm), rotates counterclockwise a degrees, and then moves back a distance v (cm). After the above adjustments, the ROS smart car will be adjusted to a posture that is more conducive to grabbing the mail.

[0094]

[0095] In particular, in this embodiment, α is set to approximately 30 degrees, which is a fixed value with the best effect obtained through multiple experiments.

[0096] Step 6: Use the deep neural network-based two-stage QR code recognition algorithm in the ROS smart car vision module to enhance and recognize the email QR code.

[0097] Furthermore, QR code enhancement technology is first used to make small codes clearer. In restricted environments, such as low-resolution cameras, long distances, and dim lighting, the resolution of QR code images drops sharply, and edges become blurred. This poses a significant challenge to QR code recognition. This embodiment overcomes this challenge through the QR code enhancement method.

[0098] "Large images with small codes" is a common difficulty encountered during long-distance scanning and long-press code recognition. Image quality is limited by the original camera hardware, resulting in blurred edges, which poses a significant challenge to the ROS smart car scanning engine. This embodiment of the QR code enhancement method can make small codes clearer.

[0099] Specifically, in the objective function, Figure 10 As shown in the figure, specifically in the model structure design, the first input is a low-resolution QR code image (LR). Through a 3×3 convolution (Conv) operation combined with a Leaky ReLU (LReLU) activation function, preliminary low-resolution features are extracted. Next, the model introduces multiple residual blocks to capture deeper features. Each residual block contains a combination of convolution and activation functions. These residual blocks use skip connections (C, representing connections) to ensure the effective transfer of gradients, avoiding the gradient vanishing problem that may occur in deep networks. The introduction of residual blocks is a core improvement of the model, which can enhance the ability of feature learning and improve the image detail recovery effect.

[0100] The model then compresses the feature dimension through 1×1 convolution (Conv) to reduce computational complexity. Next, 3×3 deconvolution (Deconv) and Leaky ReLU are used for upsampling, gradually reconstructing the low-resolution feature map into a high-resolution image. In this process, the Nearest Upsampling (Nearest Neighbor Interpolation Upsampling) technique is used to maintain the integrity of the image information during the upsampling process. Finally, another 1×1 convolution is used to convert the feature map into a high-resolution output (HR) of the QR code. The entire network has been highly tuned to minimize the size of the model. By combining the L2 / L1 loss function, the model effectively balances the reconstruction error of the QR code, so that the final super-resolution model reaches a volume of only 23KB, which fully meets the application requirements of smart cars for model miniaturization.

[0101] Furthermore, we used a convolutional neural network-based QR code recognition algorithm to improve the accuracy of province identification, and used WeChat's open-source QR code recognition module for decoding. It should be noted that WeChat's open-source QR code recognition project, wechat_qrcode, is included in the OpenCVcontrib project and can be used directly starting with OpenCV version 4.5.2.

[0102] Specifically, the QR code recognition model is called to recognize the QR code on the goods. Figure 11 As shown in the figure, a neural network is used based on the SSD framework to build an algorithm model, and targeted optimization is carried out using technologies such as residual connection (ResidualConcat), depthwise convolution (DepthwiseConvolution), dilatedconvolution (DilatedConvolution) and convolution projection (ConvolutionProjection).

[0103] Specifically, first, deep convolution is applied to the feature extraction stage, significantly reducing the computational complexity of the model by decomposing the standard convolution operation into channel-by-channel convolution. This method effectively reduces the number of parameters, allowing the network to significantly improve computational efficiency while maintaining a high feature extraction capability. Secondly, dilated convolution is introduced into the middle layer of the network to expand the receptive field of the convolution kernel. Without increasing the number of parameters and computational complexity, dilated convolution can capture more contextual information, thereby enhancing the ability to detect targets of different scales, and is particularly suitable for multi-scale object recognition and detection tasks. To further improve the stability and training depth of the model, residual connections are also applied in the network. By introducing jump connections between multiple layers, the network can learn the residual between input and output, thereby alleviating the gradient vanishing problem caused by the deepening of the network. This not only improves the training speed of the model, but also effectively avoids performance degradation during the deep learning process.

[0104] Finally, convolutional projection technology is used to reduce the network's dimensionality. Through 1×1 convolution operations, high-dimensional features are mapped to a low-dimensional space. This dimensionality reduction operation reduces the number of channels while retaining key feature information, achieving efficient feature representation and optimizing computational resources. This method further compresses the data volume, effectively improving inference speed and detection efficiency. Through these optimization steps, the detection accuracy and computational speed of object detection tasks have been significantly improved, demonstrating strong applicability and potential for widespread adoption.

[0105] After optimization, the size of the neural network algorithm model in this embodiment is only 943KB, which well meets the business requirements of low latency, small size and high recall.

[0106] Furthermore, in recognizing the QR code, this embodiment uses multi-stage segmented QR code recognition to ensure its success rate.

[0107] In the first stage, the distance is determined by identifying the positioning code, and the camera performs the first stage recognition in less than or equal to 3 seconds at the preset optimal recognition position. If the recognition is successful, the goods are immediately grabbed.

[0108] If the delivery information contained in the QR code is not recognized in the first stage, keep the QR code recognition turned on before grabbing the goods and placing them on the smart car loading platform, and try to perform the second stage recognition of the QR code to avoid recognition risks caused by light and other factors, and improve the success rate of grabbing.

[0109] Step 7: Based on the delivery location corresponding to the grabbed goods, plan the delivery navigation and determine the lateral delivery direction.

[0110] Depending on the distribution of the cargo locations, clockwise and counterclockwise delivery path planning schemes can be used, which include the following sub-steps:

[0111] 1. Delivery route planning: Based on the address information recognized by the QR code, the shortest route is planned from the current smart car location to each target delivery location until the next cargo pickup location is reached, and the cargo is delivered in sequence;

[0112] 2. Delivery sequence and side delivery direction planning: Assuming that the target delivery points can be connected to form a polygon of any shape (not considering the simplest straight line case), two delivery plans can be planned: clockwise and counterclockwise. Delivery is carried out to each target delivery point in the polygon area according to the planned path. The left delivery plan is used for clockwise delivery and the right delivery plan is used for counterclockwise delivery.

[0113] 3. During the delivery process, assuming that multiple target locations that need to be delivered are located in a straight line, by directly controlling the intelligent vehicle for parallel correction, for all delivery points on the straight line, the intelligent vehicle is directly controlled to move forward or backward a specific distance and then rotate the robotic arm to achieve delivery, replacing the original navigation solution and speeding up the delivery.

[0114] To save time, this embodiment utilizes side delivery technology. Side delivery requires a smaller angle of deflection than rear delivery, improving cargo sorting efficiency. When the delivery path is planned in a clockwise direction, considering that the collection boxes are located at the edge of the site, it is most efficient to deliver all goods on the left side. Similarly, when the delivery path is planned in a counterclockwise direction, it is more efficient to deliver all goods on the right side.

[0115] It should be noted that if the positions of the two goods to be delivered are parallel to the horizontal and vertical axes of the site, the ROS smart car can be directly controlled to make a parallel correction and then move the distance between the two goods in a straight line, so as to achieve rapid movement of the smart car along the horizontal and vertical axes to the target delivery position, thereby saving delivery time and avoiding errors caused by re-navigation.

[0116] In summary, the present invention is based on the ROS-based intelligent path planning and cargo side delivery method. First, a downward suction cup bracket with a camera is 3D printed, and the deflection angle of the intelligent car and the sorting platform is judged to perform parallel correction when grabbing the goods; then, the position of the intelligent car is corrected by triangulation-like path planning, and the robotic arm is controlled to extend toward a specific angle and a specific distance to perform deflection angle grabbing; then, the QR code enhancement and recognition technology based on convolutional neural networks is used to perform two-stage recognition of the cargo QR code; finally, according to the delivery position corresponding to the grabbed cargo, the cargo delivery navigation is planned and the lateral delivery direction is determined.

[0117] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A ROS-based intelligent path planning and cargo side delivery method, characterized in that: The steps include: S1. Use 3D printing technology to print a downward suction cup bracket with a camera. Install it at the end of the smart car's robotic arm. The camera faces horizontally and is used to identify the QR code on the side of the target cargo. S2. Build an intelligent map, use the GMAPPING method and the adaptive Monte Carlo positioning method to build and locate the intelligent car, and perform intelligent map cropping to improve the overall processing speed and reduce positioning errors; S3. Navigate the smart cart to the target cargo sorting platform using a Dijkstra-based global path planning and TEB local path planning algorithm: pre-define the location and orientation of the sorting platform in the global map, obtain the cart's current orientation based on the odometer, calculate the deflection angle between the cart and the sorting platform, and reverse the deflection angle to make the smart cart parallel to the sorting platform, thus achieving parallel correction. S4: Control the smart car to move forward and backward until it can recognize the April tag code of the target item on the sorting table, stop the car, and grab the target item. Based on the location information of the target item, use trigonometric functions and geometric principles to calculate the extension length and deflection angle required for the smart car's robotic arm to grab the current target item. If the extension length or deflection angle exceeds the maximum range that the current robotic arm can reach, the robotic arm is unreachable. Perform at least one trigonometric path planning and parallel correction on the smart car's posture until the grabbing requirements are met. S5. Calculate the coordinates of the cargo center in the three-dimensional coordinate system of the robot arm chassis based on the deflection angle and distance from the center of the suction cup of the robot arm to the cargo center. Control the robot arm to grab the cargo at this coordinate to achieve deflection angle grabbing. S6. A two-stage recognition strategy is used to identify the target cargo QR code. In each stage, the recognition algorithm first enhances the QR code image using a super-resolution method based on residual and convolutional projection. Then, a method based on a deep convolutional network is used to recognize the QR code to obtain the address information of the target cargo to be delivered, including the province, city, and district information. S7. Plan the delivery route of the goods based on the address information corresponding to the grabbed goods. Preset the lateral delivery direction of each delivery point based on different delivery plans, thereby controlling the smart car to deliver the goods sideways one by one.

2. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S1, the downward suction cup bracket with a camera is installed at the front end of the smart car's robotic arm, with a male head of a corresponding shape drawn according to the interface on the smart car. It includes a main bracket, a suction cup connection module, and a camera module. The main bracket and the suction cup connection module are connected by bamboo fiber material. The suction cup connection module is fixed with a top screw through the screw hole reserved on the main bracket and is used to suck the target cargo. The suction cup air outlet in the suction cup connection module is placed on the side to reduce the height of the suction cup module.

3. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S2, the GMAPPING method uses the odometer data of the smart car combined with the distance information scanned by the lidar to construct an environmental map in real time. The adaptive Monte Carlo positioning method matches the odometer and lidar scanning information with the environmental information in the map to determine the optimal particle to represent the current position and posture of the smart car. At the same time, the intelligent map clipping method is used to clip irrelevant areas. By identifying and eliminating the map edges and areas that the smart car cannot reach or are irrelevant, the size of the map is reduced.

4. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S3, the global path planning based on Dijkstra and the local path planning based on TEB are used to navigate the smart car to the target cargo sorting platform; the optimal time weight of the TEB algorithm parameter is set so that the smart car can speed up on the straight road and make tangent turns close to the edge of the path to adapt to the cargo sorting requirements; the optimization weight parameter for shrinking the trajectory according to the conversion / execution time is reduced, and the known and unknown obstacles are identified by adjusting the forward planning maximum distance parameter; and the robot arm's max_velocity parameter is set to maximize its movement speed.

5. The ROS-based intelligent path planning and cargo side delivery method according to claim 4 is characterized in that: In step S3, assuming that the goods are separated one by one and placed in a row on a rectangular sorting platform, after the smart car navigates to the sorting platform, if the parallel deviation angle with the sorting platform is greater than a preset threshold, parallel correction is performed to ensure that the smart car and the sorting platform remain parallel. The process includes the following sub-steps: S3.

1. Read the data from the smart car's attitude sensor. The data format is Euler quaternion (w, x, y, z). Among them, w is the real part of the quaternion, which represents the relationship between the rotation angle and the rotation axis, and determines the "size" or "amplitude" of the rotation. x, y, z are the imaginary parts of the quaternion, which are associated with the direction of the rotation axis and represent the direction of the rotation axis, forming the spatial direction of the rotation. S3.2, convert the attitude data expressed in the form of Euler quaternion into angle data (yaw x ,yawy,yaw z ), yaw x ,yawy,yaw z Respectively represent the angles between the current posture of the smart car and the positive directions of the x, y, and z axes; S3.

3. Target attitude angle when the smart car is parallel to the sorting platform x Subtract the current attitude angle yaw of the smart car obtained in step S3.2 x , calculate the deviation angle yawbias: <h2 style=";text-align:left;direction:ltr">yawbias=target<h2 style=";text-align:left;direction:ltr"> x <h2 style=";text-align:left;direction:ltr"> -yaw<h2 style=";text-align:left;direction:ltr"> x Among them, target x Indicates the target attitude angle when the smart car is parallel to the sorting platform, yaw x represents the current attitude angle obtained from the quaternion conversion in step S3.2, and yawbias represents the rotation angle difference between the current attitude of the smart car and the target attitude; Issue the command rot(yawbias,speed,torlance) to the smart car, which means the smart car rotates at the speed of speed y awbias degree, the allowable error is torlance; S3.

4. Due to the car's rotation accuracy error, after one correction, if the yawbias value is greater than 1, steps S3.1-S3.3 will be repeated continuously. Each time the speed and torlance values ​​are reduced to gradually converge the yawbias value. When the yawbias value is less than 1, the parallel correction is completed.

6. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S4, the posture of the smart car is adjusted using triangular path planning, which includes the following sub-steps: S4.

1. Control the smart car to rotate α degrees clockwise. Assume that the distance the smart car needs to adjust is ybias, and the horizontal deviation between the car and the center point of the target cargo is xbias. S4.

2. Control the smart car to follow a triangular path, correct the deviation between the car and the target cargo, and control the smart car to move forward a distance u, rotate counterclockwise α degrees, and then move back a distance v: S4.

3. When the required extension distance and deflection angle of the robotic arm are still not achievable, repeat steps S4.1 to S4.2 until the required extension distance and deflection angle of the intelligent robot arm are within the achievable range. Perform parallel correction again to ensure that the intelligent robot and the sorting platform remain parallel, thereby improving the grasping success rate.

7. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S5, the deflection angle grasping is specifically as follows: when the grasping extension distance required by the smart car is less than or equal to the maximum extension distance of the robot arm, the position of the center point of the grasped cargo is calculated according to the position of the center point of the Apriltag code; Perform deflection angle grasping. Specifically, assuming that all goods have the same size, the x-axis offset between the center point of the camera at the current position of the smart car and the center point of the Apriltag code is d x , the y-axis offset is d y , the measured x-axis offset of the Apriltag code center from the target cargo center is x1, and the y-axis offset is y1; increase the x-coordinate of the current position of the robot arm by x1-d x , the y coordinate increases by y1-d y , move the end of the robot arm to the three-dimensional coordinate system with the base of the robot arm as the origin of the coordinate system (x0+x1-d x , y0+y1-d y , z0), where x0, y0, and z0 are the three-dimensional coordinates of the center point of the suction cup of the robotic arm before grasping in the current coordinate system.

8. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S6, a two-stage recognition strategy is used to identify the target goods QR code: In the first stage, the distance between the smart cart and the sorting platform is determined by identifying the Apriltag code. The robot arm is then controlled to move to the preset recognition position to ensure that the QR code captured by the fixed-focus camera is in clear focus. As the robot arm moves, the camera repeatedly captures and recognizes the QR code image on the goods. After reaching the recognition position, the camera continues to recognize the QR code until the email address information contained in the QR code is recognized or the recognition time exceeds the preset threshold. The robot arm then extends to the grasping position, and the first stage of recognition ends. If the email address information is not recognized in the first stage, QR code recognition will be required again in the second stage. In the second stage, after grabbing the goods, the robotic arm moves the robotic arm and places the goods on the loading platform of the smart car. During the movement, the camera will continue to recognize the QR code multiple times until the delivery address information is successfully recognized; if the email address information has been recognized in the first stage, the second stage recognition step will be skipped for quick grabbing.

9. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S7, different delivery path plans are selected according to the different distribution of cargo locations, and cargo delivery is performed using a lateral delivery method, which specifically includes the following sub-steps: S7.

1. Delivery Path Planning: Based on the address information identified by the QR code, the shortest path is planned from the current smart car location to each target delivery location until the next pickup location is reached, and the goods are delivered in sequence; S7.

2. Delivery Sequence and Side-Delivery Direction Planning: Assuming that the target delivery points can be connected to form an arbitrary polygon, regardless of straight lines, two delivery plans can be planned: clockwise and counterclockwise. Deliver to each target delivery point in the polygonal area in sequence according to the planned path. A left-side delivery plan is used for clockwise delivery, and a right-side delivery plan is used for counterclockwise delivery. S7.

3. During the delivery process, assuming that multiple target locations that need to be delivered are located in a straight line, the intelligent vehicle can be directly controlled to make parallel corrections. For all delivery points on the straight line, the intelligent vehicle can be directly controlled to move forward or backward a specific distance and then rotate the robotic arm to achieve delivery, replacing the original navigation solution and speeding up the delivery.

10. A ROS-based intelligent path planning and cargo side-dropping intelligent vehicle, which performs path planning and cargo side-dropping by the method according to any one of claims 1 to 9, characterized in that: include: Mapping and navigation module, posture adjustment module, robotic arm control and grasping module, and intelligent car vision module; The robot arm control and grasping module, through hardware modification, installs a downward suction cup bracket with a camera on the intelligent robot arm to perform optimal path planning for the intelligent robot arm, and controls the rotation angle of the robot arm to grasp the goods and drop the goods to the designated area; The mapping and navigation module uses the GMAPPING method to combine odometer data with the distance information scanned by the LiDAR to locate the relative position of obstacles to the smart car, completing the map construction. The AMCL method is used to match the map environment of each particle's position with the environmental information scanned by the LiDAR and assign weights to perform smart car positioning. The posture adjustment module is used to obtain the current posture of the intelligent car and perform fine posture adjustment through the triangulation path planning method; The smart car's vision module is used to obtain a picture of the target cargo, perform image preprocessing to crop out the QR code portion, and then use a two-stage QR code recognition method based on a deep neural network to identify the target cargo's QR code.

Images