ROS-based intelligent path planning and cargo side throwing method and intelligent trolley

Abstract

The invention discloses an intelligent path planning and cargo side throwing method based on an ROS and an intelligent trolley. The method comprises the steps that firstly, a downsuction type suction cup support with a camera is printed and installed at the tail end of a mechanical arm of the intelligent trolley; using a Dijkstra-based global path and a TEB-based local path to navigate the intelligent trolley to a sorting table for parallel correction so as to enable the intelligent trolley to be parallel to the sorting table; if the arm extension length of the mechanical arm of the intelligent trolley is smaller than the needed grabbing extension distance, similar triangular path planning correction is conducted on the posture of the trolley; a mechanical arm is controlled to stretch out by a specific distance towards a specific angle to conduct deflection angle grabbing, and goods two-dimensional code two-stage recognition is conducted through a two-dimensional code enhancement and recognition algorithm based on a deep neural network; and finally, according to the delivery positions of all the goods, an optimal navigation path is planned, the lateral delivery direction is judged, and target goods are grabbed and delivered. According to the method, the precision and efficiency of the cargo grabbing process are optimized, and the overall performance of the intelligent path planning and cargo side throwing method is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of path planning, intelligent sorting and identification technology for logistics intelligent trolleys, and in particular to an intelligent path planning and cargo side-dropping method based on ROS and an intelligent trolley. Background Art

[0002] In modern logistics and warehouse management, the combination of ROS intelligent vehicle navigation, sorting and recognition technology is the key to achieving automation and efficient management. Traditional robot navigation methods mainly include global path planning and local path planning. Among them, the Dijkstra algorithm is often used for global path planning, but it has limitations when dealing with dynamic environments. The TEB (Timed Elastic Band) algorithm performs well in local path planning and can effectively deal with dynamic obstacles.

[0003] At the same time, sorting and recognition technology realizes automatic classification and recognition through image processing and sensor data, further improving the intelligence level of the ROS smart car system.

[0004] Therefore, the comprehensive application of image processing recognition technology and path planning technology in the logistics industry for mail sorting and delivery can significantly improve the overall performance and adaptability of the ROS intelligent vehicle system, and further improve the efficiency and accuracy of mail sorting and delivery in the logistics industry. Summary of the invention

[0005] The problem to be solved by the present invention is to provide an intelligent path planning and cargo side-dropping method and an intelligent vehicle based on ROS, which are used to improve the efficiency and accuracy of mail sorting and delivery in the logistics industry.

[0006] The present invention adopts the following technical solution: a ROS-based intelligent path planning and cargo side delivery method, comprising the following steps:

[0007] S1. Use 3D printing technology to print a downward suction cup bracket with a camera, which is installed at the end of the robot arm of the smart car. The camera faces the horizontal direction and is used to identify the QR code on the side of the target cargo.

[0008] S2. Build an intelligent map, use the GMAPPING method and the adaptive Monte Carlo positioning method to build and locate the intelligent car, and perform intelligent map cropping to improve the overall processing speed and reduce positioning errors;

[0009] S3. Use the global path planning based on Dijkstra and the local path planning algorithm of TEB to navigate the smart car to the target cargo sorting platform: pre-define the position and orientation information of the sorting platform in the global map, obtain the current orientation of the car according to the odometer, calculate the deflection angle between the car and the sorting platform, and reversely rotate the car by the deflection angle to make the smart car parallel to the sorting platform to achieve parallel correction, so as to facilitate the smart car to perform lateral recognition and grasping;

[0010] S4, control the smart car to move forward and backward until the car can identify the April tag code of the target goods on the sorting table, stop the car, and grab the target goods; according to the location information of the target goods, use trigonometric functions and geometric principles to calculate the extension length and deflection angle required for the smart car's mechanical arm to grab the current target goods. If the extension length or deflection angle exceeds the maximum range that the current mechanical arm can reach, the mechanical arm is unreachable, and the posture of the smart car is subjected to at least one trigonometric path planning and parallel correction until the grabbing requirements are met;

[0011] S5. According to the deflection angle and distance from the center point of the suction position of the robot arm suction cup to the center point of the cargo, the coordinates of the center point of the cargo in the three-dimensional coordinate system of the robot arm chassis are calculated, and the robot arm is controlled to grab the cargo at this coordinate to achieve deflection angle grabbing;

[0012] S6. A two-stage recognition strategy is used to identify the target goods QR code. The recognition algorithm in each stage first uses a super-resolution method based on residual and convolutional projection to enhance the QR code image, and then uses a method based on a deep convolutional network to identify the target goods contained in the QR code. The address information to be delivered, including province, city and district information, is obtained;

[0013] S7. According to the address information corresponding to the grabbed goods, the goods delivery route is planned. According to different delivery plans, the lateral delivery direction of each delivery point is preset, so as to control the smart car to deliver the goods one by one.

[0014] Preferably, in step S1, the downward suction cup bracket with a camera is installed at the front end of the smart car mechanical arm by drawing a male head of a corresponding shape according to the interface on the smart car, and includes a main bracket, a suction cup connection module, and a camera module; the main bracket and the suction cup connection module are connected by bamboo fiber material, and the suction cup connection module is fixed by a screw hole reserved on the main bracket using a top screw to suck the target cargo, and the suction cup air outlet in the suction cup connection module is placed on the side to reduce the height of the suction cup module.

[0015] Preferably, in step S2, the GMAPPING method uses the odometer data of the smart car combined with the distance information of the laser radar scan to construct an environmental map in real time; the adaptive Monte Carlo positioning method matches the odometer and laser radar scan information with the environmental information in the map to determine the optimal particles to represent the current position and posture of the smart car; at the same time, an intelligent map clipping method is used to clip irrelevant areas, and the size of the map is reduced by identifying and eliminating the map edges and areas that the smart car cannot reach or are irrelevant.

[0016] Preferably, in step S3, the global path planning based on Dijkstra and the local path planning based on TEB are used to navigate the smart car to the target cargo sorting platform; the optimal time weight of the TEB algorithm parameter is set so that the smart car can speed up on the straight road and turn tangent to the edge of the path to meet the cargo sorting needs; the optimization weight parameter for shrinking the trajectory according to the conversion / execution time is reduced, and the known obstacles and unknown obstacles are identified by adjusting the forward planning longest distance parameter; the robot arm parameters are increased to make the smart car move at the fastest speed and improve the grasping and delivery execution speed; parallel correction is performed before the smart car arrives at the sorting platform to ensure that the smart car and the sorting platform remain parallel.

[0017] Preferably, in step S4, the posture of the smart car is adjusted using triangular path planning, including the following sub-steps:

[0018] S4.1. Control the smart car to rotate α degrees clockwise. Assume that the distance that the smart car needs to adjust is ybias, and the horizontal deviation between the car and the center point of the target cargo is xbias.

[0019] S4.2. Control the smart car to follow a triangular path, correct the deviation between the car and the target cargo, control the smart car to move forward a distance u, rotate counterclockwise a degrees, and then move back a distance v:

[0020]

[0021] S4.3. When the required extension distance and deflection angle of the robot arm are still unreachable, repeat steps S4.1 to S4.2 until the required extension distance and deflection angle of the robot arm of the smart car are within the reachable range, and perform parallel correction again to ensure that the smart car and the sorting platform remain parallel, thereby improving the success rate of grasping.

[0022] Preferably, in step S5, the deflection angle grasping specifically refers to that when the grasping extension distance required by the smart car is less than or equal to the maximum extension distance of the robot arm, the position of the center point of the grasped cargo is calculated by the position of the center point of the Apriltag code; the deflection angle grasping specifically refers to that, assuming that all cargoes have the same size, the x-axis direction offset between the center point of the camera at the current position of the smart car and the center point of the Apriltag code is d x , the y-axis offset is d y The measured x-axis offset of the Apriltag code center point from the target cargo center point is x 1 , the y-axis offset is y 1 . Then the x coordinate of the current position of the robot arm needs to be increased by x 1 -d x , the y coordinate increases by y 1 -d y , that is, to move the end of the robot arm to a three-dimensional coordinate system with the robot arm base as the origin of the coordinate system (x 0 +x 1 -d x ,y 0 +y 1 -d y , z 0 ) where x 0 ,y 0 ,z 0 They are respectively the three-dimensional coordinates of the center point of the suction cup of the robotic arm before grasping in the current coordinate system.

[0023] Preferably, in step S6, a two-stage recognition of the target goods QR code is performed:

[0024] In the first stage, the distance between the smart car and the sorting platform is determined by identifying the Apriltag code, and the end of the robotic arm is controlled to move to the preset identification position to ensure that the QR code captured by the fixed-focus camera is in clear focus. During the movement of the robotic arm, the camera collects and identifies the QR code images on the goods multiple times. After reaching the identification position, the camera continues to identify the QR code until the email address information contained in the QR code is identified or the identification time exceeds the preset threshold. Then the robotic arm continues to extend to the grasping position, and the first stage of identification ends.

[0025] If the email address information is not recognized in the first stage, the QR code recognition will be required again in the second stage. In the second stage, after grabbing the goods, the robotic arm moves the robotic arm to place the goods on the loading platform of the smart car. During the movement, the camera will continue to recognize the QR code multiple times until the delivery address information is successfully recognized; if the email address information has been recognized in the first stage, the recognition step of the second stage will be skipped for quick grabbing.

[0026] Preferably, in step S7, different delivery path plans are selected according to different distribution conditions of cargo locations, and cargo delivery is performed by a lateral delivery method, which specifically includes the following sub-steps:

[0027] S7.1, delivery path planning: Based on the address information recognized by the QR code, a shortest path is planned from the current smart car position to each target delivery position until the next cargo grabbing position is reached, and the cargo is delivered in sequence;

[0028] S7.2, delivery sequence and side delivery direction planning: Assuming that the target delivery points can be connected into a polygon of any shape, without considering the straight line situation, two delivery plans can be planned: clockwise and counterclockwise. Deliver to each target delivery point on the polygon area in turn according to the planned path. The left delivery plan is used for clockwise delivery and the right delivery plan is used for counterclockwise delivery.

[0029] S7.3. During the delivery process, assuming that multiple target locations that need to be delivered are on a straight line, after directly controlling the intelligent car to make parallel corrections, for all delivery points on the straight line, directly control the intelligent car to move forward or backward a specific distance and then rotate the robotic arm to achieve delivery, replacing the original navigation solution and speeding up the delivery.

[0030] The technical solution of the present invention also provides: a ROS-based intelligent path planning and cargo side-dropping intelligent trolley, a downward suction cup bracket with a camera is installed on the trolley mechanical arm, and the above method is used for path planning and cargo side-dropping, including: a mapping navigation module, a posture adjustment module, a mechanical arm control and grasping module, and a ROS intelligent vehicle vision module;

[0031] The mapping and navigation module uses the GMAPPING method to build maps and the AMCL method to locate the smart car;

[0032] The posture adjustment module is used to obtain the current posture of the intelligent car and perform fine posture adjustment through a triangular path planning method;

[0033] The robot arm control and grasping module is modified by hardware to install a downward suction cup bracket at the front end of the robot arm of the smart car, perform optimal path planning for the robot arm, and control the rotation angle of the robot arm to grasp the goods and drop the goods to the designated area;

[0034] The ROS smart car vision module is used to obtain target cargo images, perform image preprocessing, and use a deep neural network-based QR code enhancement and recognition method to identify the target cargo QR code.

[0035] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0036] 1. Local path optimization: Based on the global path, the method of the present invention applies the TEB algorithm to optimize the local path, focusing on adjusting the TEB algorithm parameters to adapt to the needs of cargo sorting, reducing the optimization weight parameters for shrinking the trajectory according to the conversion / execution time, increasing the forward planning maximum distance parameters, and identifying known obstacles and unknown dynamic or static obstacles.

[0037] 2. Multiple error adjustment methods: The method of the present invention determines the position through positioning code recognition, and selects different correction methods when the error size is different, thereby ensuring that the goods can be stably grasped while speeding up the speed as much as possible.

[0038] 3. Multi-stage intelligent recognition: The method of the present invention combines CNN image recognition technology and two-stage QR code recognition. The ROS smart car can quickly and accurately recognize the QR code of the goods to be grabbed.

[0039] 4. Autonomous navigation path optimization: The method of the present invention plans the optimal navigation path before delivery by capturing the location distribution information of the goods, avoiding repeated navigation and improving the working efficiency of the ROS smart car. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a block diagram of the ROS intelligent vehicle module for intelligent path planning and cargo side delivery based on ROS of the present invention;

[0041] Figure 2 It is a block diagram of the mapping process module of the GMAPPING method of the mapping navigation module of the present invention;

[0042] Figure 3 This is a structural diagram of a 3D printed downward suction cup bracket with a camera according to the present invention;

[0043] Figure 4 This is a structural diagram of a suction cup connection module in a downward suction cup bracket of the present invention;

[0044] Figure 5 This is a structural diagram of the top end of the main bracket in the downward suction cup bracket of the present invention;

[0045] Figure 6 This is a flow chart of the autonomous navigation path of the present invention;

[0046] Figure 7 A schematic diagram of the robot arm's deflection angle grasping when the ROS smart car of the present invention is not aligned with the mail box;

[0047] Figure 8 A schematic diagram of the distance and deflection angle grasping between the ROS smart car and the counter and the mail box of the present invention;

[0048] Fig. 9 This is a schematic diagram of the ROS smart car triangulation correction of the present invention;

[0049] Fig.10 This is a network structure diagram of the two-dimensional code enhancement model of the present invention;

[0050] Fig.11 This is a schematic diagram of the two-dimensional code recognition principle based on convolutional neural network of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the application is further elaborated in detail below in conjunction with the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in the field on this embodiment belong to the protection scope of the present invention. At the same time, for the step numbering in the embodiment of the present invention, it is only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0052] In one embodiment of the present invention, a smart car controlled by an intelligent path planning and cargo side-dropping method based on ROS is provided, wherein a downward suction cup bracket with a camera is installed on the robot arm of the car to perform path planning and cargo side-dropping, such as Figure 1 As shown, the ROS smart car includes: a map building and navigation module, a posture adjustment module, a robotic arm control and grasping module, and a ROS smart car vision module;

[0053] The map building and navigation module builds maps through the GMAPPING method, uses the Adaptive Monte Carlo Localization (AMCL) method to locate the smart car, and performs intelligent map cropping;

[0054] Specifically, the GMAPPING method uses data from lidar and odometer, such as Figure 2 As shown in the figure, the odometer data is first used to calculate the posture of the ROS smart car, and then the distance information scanned by the laser radar is combined to locate the relative position of the obstacle to the smart car, completing the construction of the map;

[0055] The posture adjustment module is used to obtain the current posture of the intelligent car and make fine adjustments to the posture through the triangular path planning method;

[0056] The robot arm control and grasping module, through hardware modification, installs a downward suction cup bracket at the front end of the smart car robot arm, performs optimal path planning for the robot arm, and controls the rotation angle of the robot arm to grasp the goods. If there are goods placed at the same horizontal position, they can be directly moved horizontally for quick grasping;

[0057] The ROS smart car vision module is used to obtain target cargo images, perform image preprocessing, and use the convolutional neural network-based QR code recognition and enhancement method to recognize the QR code.

[0058] The method for controlling the ROS smart car to perform path planning and cargo side delivery includes the following steps:

[0059] Step 1: 3D print a downward suction cup bracket with a camera, with the camera facing horizontally so as to align with the QR code on the side of the target cargo for identification;

[0060] like Figure 3 As shown, a male head of corresponding shape is drawn according to the interface on the robot smart car, and the position of the camera is offset according to the position of the QR code on the mail box to improve the vertical positioning tolerance, and the suction cup air outlet is placed on the side to reduce the height of the suction cup module.

[0061] Downward suction cup bracket, such as Figure 4 As shown, it includes: a main bracket and a suction cup connecting module. The connection between the main bracket and the suction cup connecting module is made of lightweight, high-toughness, low-cost, and ubiquitous bamboo fiber material.

[0062] Furthermore, if Figure 5 As shown, in this embodiment, the suction cup connection module is fixed by using screws through screw holes reserved on the main bracket, which reduces the damage rate and facilitates replacement and maintenance at any time.

[0063] Step 2: Build a map through the mapping and navigation module, build a map through the GMAPPING method, locate the smart car through the AMCL method, and perform intelligent map cropping to improve the overall processing speed and reduce positioning errors.

[0064] The GMAPPING method uses the odometer data of the smart car combined with the distance information scanned by the LiDAR to build an environmental map in real time; the Adaptive Monte Carlo Localization (AMCL) method matches the odometer and LiDAR scanning information with the environmental information in the map to determine the optimal particles to represent the current position and posture of the smart car. At the same time, the intelligent map clipping technology is used to accurately clip irrelevant areas, and the map size is significantly reduced by identifying and eliminating the map edge and the areas that the smart car cannot reach or are irrelevant. This technology not only effectively reduces the amount of calculation of the system in the process of map construction and positioning, improves the overall processing speed, but also prevents the car from having positioning deviation problems in the boundary area. By clipping out invalid areas, the system can focus more on the core path and environmental information within the working range, thereby enhancing the accuracy of positioning, reducing the risk of error propagation, and ultimately improving the efficiency and reliability of the entire path planning system, especially in complex or dynamic environments.

[0065] Step 3: Use Dijkstra-based global and TEB (Timed Elastic Band)-based local path planning to realize automatic navigation of the ROS smart car to the target point. In order to facilitate the grabbing of goods on the sorting table, the position and orientation of the sorting table are first pre-defined in the global map, and then the current orientation of the car is obtained according to the odometer. Then, the deflection angle between the car and the sorting table is calculated, and the deflection angle of the car is reversed to rotate the car in the opposite direction to make it parallel to the sorting table.

[0066] In particular, the optimal time weight of the TEB algorithm parameters is set so that the smart car can speed up on the straight road and make tangent turns close to the edge of the path to meet the needs of cargo sorting; the optimization weight parameter for shrinking the trajectory according to the conversion / execution time is reduced, and the known and unknown obstacles are identified by adjusting the forward planning maximum distance parameter.

[0067] In this embodiment, weight_optimaltime is set as the optimal time weight. When weight_optimaltime is increased to 5, the vehicle's linear speed on the straight road is significantly accelerated, and the larger it is, the closer it is to max_vel_x, and it turns along the tangent line close to the edge of the path.

[0068] Further set the maximum forward look distance max_global_plan_lookahead_dist to 1.5, specifying the maximum length (cumulative Euclidean distance) of the global plan subset to be optimized. The actual length is then determined by the logical combination of the local cost map size and this maximum limit, set to zero or a negative number to disable this limit. This distance 1. should increase with the increase of the maximum speed of the vehicle 2. should not exceed the reliable measurement range of sensors such as lidar 3. should not exceed the size of the local cost map, that is, TEB cannot be required to plan outside the local cost map.

[0069] like Figure 6 As shown, the smart car starts to navigate. First, it searches for a temporary target cargo point. If so, it uses the temporary target cargo coordinates as the navigation target. Otherwise, it obtains the sorting platform coordinate point from the coordinate file as the navigation target, adjusts the smart car coordinates to align with the target mail box, moves to the target point, and saves the current coordinates as the temporary coordinate point.

[0070] Before the smart car reaches the sorting platform, it will perform parallel correction to ensure that it remains parallel to the sorting platform, including the following sub-steps:

[0071] S3.1. Read the data from the smart car's attitude sensor. The data format is Euler quaternion (w, x, y, z).

[0072] S3.2. Convert the posture data expressed in the form of Euler quaternion into data expressed in angle (yaw x ,yaw y ,yaw z ), yaw x ,yaw y ,yaw z Respectively represent the angles between the current posture of the smart car and the positive directions of the x, y, and z axes;

[0073] S3.3. Target attitude angle when the smart car is parallel to the sorting platform x Subtract the current attitude angle yaw of the smart car obtained in step S3.2 x , and obtain the deviation angle yawbias = target x -yaw x , issue the command rot(yawbias,speed,torlance) to the smart car, which means the smart car rotates yawbias degrees at a speed of speed, with an allowable error of torlance;

[0074] S3.4. Due to the error of the car's rotation accuracy, parallel correction may not be achieved in one go. In this case, steps S3.1-S3.3 will be repeated continuously. Each time the speed and torlance values ​​are reduced to gradually converge the yawbias value. When the yawbias value is less than 1, parallel correction is completed.

[0075] During the navigation process, first, Dijkstra global path planning is used to generate a fixed optimal global path;

[0076] Preferably, this embodiment uses the Dijkstra algorithm to initialize the set (representing the set of vertices with the shortest path determined), and a priority queue Q (representing the set of vertices that can be used to optimize the path), by continuously taking vertex u from Q, and using the distance w from u to the adjacent vertex to update the shortest path dist(v) of all vertices adjacent to u in S until the queue Q is empty:

[0077] dist(v)=min(dist(v),dist(u)+w(u,v))

[0078] Where w(u,v) represents the distance between vertices u and v, and dist(·) represents the shortest path planned;

[0079] The classic Dijkstra algorithm does not consider the existence of obstacles around the path. The Dijkstra algorithm of this embodiment improves this by introducing a medium cost neutral_cost to make the planned shortest path away from obstacles:

[0080] dist(v)=min(dist(v),dist(u)+w(u,v)+neutrl_cost*obstacle(u,v))

[0081] Among them, obstacle(u,v) represents the sum of the distances from all obstacles adjacent to the path between u and v to the path.

[0082] Specifically, for a path from u to v, let the path be e(u,v), dist(e(u,v),x) represents the distance from obstacle x to path e, then:

[0083] obstacle(u,v)=∑dist(e(u,v),x i )

[0084] This embodiment modifies the neutral_cost of the Dijkstra algorithm in ROS. Compared with the original algorithm, the modified algorithm plans a path with a larger turning radius, which can effectively improve the turning efficiency of the ROS smart car.

[0085] Then, during the navigation process, the local path planning is continuously modified through decision factors such as obstacle avoidance, and finally the target point is successfully navigated.

[0086] Among them, the TEB algorithm is used to perform subsequent corrections to the initial global trajectory generated by Dijkstra's global path planner to optimize the motion trajectory of the ROS smart car. Specifically, the TEB algorithm divides the motion of the ROS smart car into two dimensions: linear velocity v and angular velocity ω, and scores each speed combination to search for the optimal motion strategy. These scores take into account factors such as target direction, obstacle avoidance ability, and obstacle distance to ensure that the ROS smart car can slow down in time to avoid collisions. On this basis, the TEB algorithm parameters are adjusted to meet the needs of cargo sorting, reduce the optimization weight parameters for shrinking the trajectory according to the conversion / execution time, increase the forward planning maximum distance parameters, and identify known obstacles and unknown dynamic or static obstacles.

[0087] Step 4: Use Apriltag image recognition to assist in locating the target mail box and perform posture and grasping position correction.

[0088] This embodiment is based on the ROS_Apriltag software package and uses the apriltags2_ros node to implement Apriltag image recognition. The apriltags2_ros node subscribes to two topic messages, namely: the original image topic and the camera information topic containing the calibration matrix information. After completing the Apriltag image recognition, the apriltags2_ros node publishes three topic messages: the standard tf topic containing tf information, the topic containing the position and quaternion attitude information, and the image topic containing the highlighted image after detection. After obtaining the location information of the Apriltag image, the location of the mail box can be accurately located.

[0089] When there is an error of less than 3 cm between the ROS smart car and the mail box, this embodiment uses the distance and angle between the ROS smart car and the position calibration code to calculate the location of the mail, and uses the adjustment of the robotic arm's grasping angle to compensate for the horizontal offset error of the ROS smart car, avoiding re-adjustment to save time.

[0090] When the ROS smart car is not aligned with the mail box and the error is within 3cm, Figure 7 As shown in (a) in the figure, the deflection angle is designed to make the robot arm deflect and grasp, as shown in Figure 7 The specific operation is as follows: Assume that the offset between the camera and the Apriltag code when the ROS smart car navigates to the counter is d x ,like Figure 8 As shown in (a), the standard error of the Apriltag code from the mail box is x, and the distance from the mail box to the counter is d. y , the deflection angle of the robotic arm of the ROS smart car is controlled as: Perform horizontal offset grabbing, such as Figure 8 As shown in (b) in .

[0091] Step 5: Correct the larger error by correcting the deviation through triangular path planning. When the horizontal deviation between the car and the mail box is large, the robot arm cannot grasp the goods by deflecting the angle. At this time, according to the distance information from the ROS smart car to the sorting platform, the triangular path adjustment is used to further refine the posture of the ROS smart car.

[0092] First, let the car rotate α degrees clockwise, such as Fig. 9 As shown in the figure, the distance that needs to be adjusted is ybias (cm). Assuming that the horizontal deviation between the car and the center point of the mailbox is xbias (cm), the deviation between the car and the mailbox can be corrected by controlling the ROS smart car to take a triangular path.

[0093] The ROS smart car moves forward a distance u (cm), rotates counterclockwise a degrees, and then moves back a distance v (cm). After the above adjustments, the ROS smart car will be adjusted to a posture that is more conducive to grabbing mail.

[0094]

[0095] In particular, in this embodiment, α is set to about 30 degrees, which is a fixed value with the best effect obtained through multiple experiments.

[0096] Step 6: Use the two-stage QR code recognition algorithm based on deep neural network in the ROS smart car vision module to enhance and recognize the email QR code.

[0097] Furthermore, the QR code enhancement technology is first used to make the small code clearer. In a restricted environment, such as low camera pixel, long distance, dim light, the QR code image resolution drops sharply and the edges become blurred, which brings great challenges to the recognition of the QR code. This embodiment overcomes this challenge through the QR code enhancement method.

[0098] "Large image and small code" is a common difficulty faced by long-distance code scanning and long-press code recognition. The image quality is affected by the original camera hardware and the edges are blurred, which brings great challenges to the recognition of the ROS smart car code scanning engine. The QR code enhancement method in this embodiment can make the small code clearer.

[0099] Specifically, in the objective function, Fig.10 As shown in the figure, specifically in the model structure design, the first input is a low-resolution QR code image (LR). Through a 3×3 convolution (Conv) operation combined with a Leaky ReLU (LReLU) activation function, preliminary low-resolution features are extracted. Next, the model introduces multiple residual blocks to capture deeper features. Each residual block contains a combination of convolution and activation functions. These residual blocks use jump connections (C, representing connections) to ensure the effective transmission of gradients, avoiding the gradient vanishing problem that may occur in deep networks. The introduction of residual blocks is the core improvement of the model, which can enhance the ability of feature learning and improve the detail restoration effect of images.

[0100] Then, the model compresses the feature dimension through 1×1 convolution (Conv) to reduce the computational complexity. Next, 3×3 deconvolution (Deconv) and Leaky ReLU are used for upsampling to gradually reconstruct the low-resolution feature map into a high-resolution image. In this process, the Nearest Upsampling (Nearest Neighbor Interpolation Upsampling) technology is used to maintain the integrity of the image information during the upsampling process. Finally, through another 1×1 convolution, the feature map is converted into a high-resolution output (HR) of the QR code. The entire network has been highly tuned to minimize the size of the model. By combining the L2 / L1 loss function, the model effectively balances the reconstruction error of the QR code, so that the final super-resolution model reaches a volume of only 23KB, which fully meets the application requirements of smart cars for model miniaturization.

[0101] Furthermore, a convolutional neural network-based QR code recognition algorithm is used to improve the accuracy of province recognition, and the WeChat open source QR code recognition module is used for decoding. It should be noted that the WeChat open source QR code recognition wechat_qrcode project is included in the OpenCVcontrib project and can be used directly starting from OpenCV4.5.2.

[0102] Specifically, the QR code recognition model is called to recognize the QR code on the goods. Fig.11 As shown in the figure, the algorithm model is constructed based on the SSD framework using a neural network, and targeted optimization is carried out using technologies such as residual connection (ResidualConcat), depthwise convolution (DepthwiseConvolution), dilatedconvolution (DilatedConvolution) and convolution projection (ConvolutionProjection).

[0103] Specifically, first, deep convolution is applied to the feature extraction stage, which significantly reduces the computational complexity of the model by decomposing the standard convolution operation into channel-by-channel convolution. This method effectively reduces the number of parameters, allowing the network to significantly improve computational efficiency while maintaining a high feature extraction capability; secondly, dilated convolution is introduced into the middle layer of the network to expand the receptive field of the convolution kernel. Without increasing the number of parameters and the amount of computation, dilated convolution can capture more contextual information, thereby enhancing the ability to detect targets of different scales, and is particularly suitable for multi-scale object recognition and detection tasks; in order to further improve the stability and training depth of the model, residual connections are also applied in the network. By introducing jump connections between multiple layers, the network can learn the residual between input and output, thereby alleviating the problem of gradient vanishing caused by the deepening of the network. This not only improves the training speed of the model, but also effectively avoids performance degradation during deep learning.

[0104] Finally, convolution projection technology is used for dimensionality reduction processing of the network. Through 1×1 convolution operation, high-dimensional features are mapped to low-dimensional space. This dimensionality reduction operation can retain key feature information while reducing the number of channels, thereby achieving efficient feature expression and computing resource optimization. In this way, the model further compresses the amount of data and effectively improves the reasoning speed and detection efficiency. Through the above optimization steps, the detection accuracy and operation speed in the target detection task are significantly improved, and it has strong applicability and promotion value.

[0105] After optimization, the size of the neural network algorithm model in this embodiment is only 943KB, which well meets the business requirements of low latency, small size and high recall.

[0106] In addition, in recognizing the two-dimensional code, the present embodiment uses multi-stage segmented two-dimensional code recognition to ensure its success rate.

[0107] In the first stage, the distance is determined by identifying the positioning code, so that the camera performs the first stage recognition for less than or equal to 3 seconds at the preset optimal recognition position. If the recognition is successful, the goods are grabbed immediately.

[0108] If the delivery information contained in the QR code is not recognized in the first stage, keep the QR code recognition turned on and try to perform the second stage recognition of the QR code before grabbing the goods and placing them on the smart car loading platform to avoid recognition risks caused by light and other factors and improve the success rate of grabbing.

[0109] Step 7: According to the delivery location corresponding to the grabbed goods, plan the goods delivery navigation and determine the lateral delivery direction.

[0110] According to the different distribution of cargo locations, clockwise and counterclockwise delivery path planning schemes can be adopted respectively, which include the following sub-steps:

[0111] 1. Delivery route planning: Based on the address information identified by the QR code, the shortest route is planned from the current smart car position to each target delivery position until the next cargo grabbing position is reached, and the cargo is delivered in sequence;

[0112] 2. Delivery order and side delivery direction planning: Assuming that the target delivery points can be connected into a polygon of any shape, ignoring the simplest straight line, two delivery plans can be planned: clockwise and counterclockwise. Deliver to each target delivery point on the polygonal area in turn according to the planned path. The left delivery plan is used for clockwise delivery and the right delivery plan is used for counterclockwise delivery.

[0113] 3. During the delivery process, assuming that multiple target locations that need to be delivered are exactly in a straight line, by directly controlling the intelligent car for parallel correction, for all delivery points on the straight line, directly control the intelligent car to move forward or backward a specific distance and then rotate the robotic arm to achieve delivery, replacing the original navigation solution and speeding up the delivery.

[0114] In order to save time, this embodiment adopts the technology of side delivery. Side delivery requires a smaller deflection angle during delivery than the rear delivery solution, which improves the efficiency of cargo sorting. When the delivery path is planned to be clockwise, considering that the cargo collection boxes are placed at the edge of the site, it is most efficient to deliver all goods on the left side. Similarly, when the delivery path is planned to be counterclockwise, it is more efficient to deliver all goods on the right side.

[0115] It should be noted that if the positions of the two goods to be delivered are parallel to the horizontal and vertical axes of the site, the ROS smart car can be directly controlled to make a parallel correction and then move the distance between the two goods in a straight line, so as to achieve rapid movement of the smart car along the horizontal and vertical axes to the target delivery position, thereby saving delivery time and avoiding errors caused by re-navigation.

[0116] In summary, the intelligent path planning and cargo side delivery method based on ROS of the present invention, firstly, through 3D printing of a downward suction cup bracket with a camera, and when grabbing the goods, the deflection angle of the smart car and the sorting platform is judged for parallel correction; then, the posture of the smart car is corrected by triangular path planning, and the robotic arm is controlled to extend a specific distance at a specific angle to perform deflection angle grabbing; then, the two-dimensional code enhancement and recognition technology based on convolutional neural network is used to perform two-stage recognition of the cargo QR code; finally, according to the delivery position corresponding to the grabbed cargo, the cargo delivery navigation is planned and the lateral delivery direction is judged.

[0117] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent path planning and cargo side delivery method based on ROS, characterized in that: The steps include: S1. Use 3D printing technology to print a downward suction cup bracket with a camera, which is installed at the end of the robot arm of the smart car. The camera faces the horizontal direction and is used to identify the QR code on the side of the target cargo. S2. Build an intelligent map, use the GMAPPING method and the adaptive Monte Carlo positioning method to build and locate the intelligent car, and perform intelligent map cropping to improve the overall processing speed and reduce positioning errors; S3. Use the global path planning based on Dijkstra and the local path planning algorithm of TEB to navigate the smart car to the target cargo sorting platform: pre-define the position and orientation information of the sorting platform in the global map, obtain the current orientation of the car according to the odometer, calculate the deflection angle between the car and the sorting platform, and reversely rotate the car by the deflection angle to make the smart car parallel to the sorting platform to achieve parallel correction; S4, control the smart car to move forward and backward until the car can identify the April tag code of the target goods on the sorting table, stop the car, and grab the target goods; according to the location information of the target goods, use trigonometric functions and geometric principles to calculate the extension length and deflection angle required for the smart car's mechanical arm to grab the current target goods. If the extension length or deflection angle exceeds the maximum range that the current mechanical arm can reach, the mechanical arm is unreachable, and the posture of the smart car is subjected to at least one trigonometric path planning and parallel correction until the grabbing requirements are met; S5. According to the deflection angle and distance from the center point of the suction position of the robot arm suction cup to the center point of the cargo, the coordinates of the center point of the cargo in the three-dimensional coordinate system of the robot arm chassis are calculated, and the robot arm is controlled to grab the cargo at this coordinate to achieve deflection angle grabbing; S6. A two-stage recognition strategy is used to identify the target goods QR code. The recognition algorithm in each stage first uses a super-resolution method based on residual and convolutional projection to enhance the QR code image, and then uses a method based on a deep convolutional network to identify the target goods contained in the QR code. The address information to be delivered, including province, city and district information, is obtained; S7. According to the address information corresponding to the grabbed goods, the goods delivery route is planned. According to different delivery plans, the lateral delivery direction of each delivery point is preset, so as to control the smart car to deliver the goods one by one.

2. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S1, the downward suction cup bracket with a camera is installed at the front end of the smart car mechanical arm by drawing a male head of a corresponding shape according to the interface on the smart car, and includes a main bracket, a suction cup connection module, and a camera module; the main bracket and the suction cup connection module are connected by bamboo fiber material, and the suction cup connection module is fixed by a screw hole reserved on the main bracket using a top screw to suck the target cargo, and the suction cup air outlet in the suction cup connection module is placed on the side to reduce the height of the suction cup module.

3. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S2, the GMAPPING method uses the odometer data of the smart car combined with the distance information of the laser radar scan to build an environmental map in real time; the adaptive Monte Carlo positioning method matches the odometer and laser radar scan information with the environmental information in the map to determine the optimal particles to represent the current position and posture of the smart car; at the same time, the intelligent map clipping method is used to clip irrelevant areas, and the size of the map is reduced by identifying and eliminating the map edges and areas that the smart car cannot reach or are irrelevant.

4. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S3, the global path planning based on Dijkstra and the local path planning based on TEB are used to navigate the smart car to the target cargo sorting platform; the optimal time weight of the TEB algorithm parameter is set so that the smart car can speed up on the straight road and make tangent turns close to the edge of the path to meet the cargo sorting needs; the optimization weight parameter for shrinking the trajectory according to the conversion / execution time is reduced, and the known obstacles and unknown obstacles are identified by adjusting the forward planning maximum distance parameter; the robot arm's max_velocity parameter is set to make its movement speed reach the fastest.

5. The ROS-based intelligent path planning and cargo side delivery method according to claim 4 is characterized in that: In step S3, assuming that the goods are separated one by one and placed in a row on a rectangular sorting table, after the smart car navigates to the sorting table, if the parallel deviation angle with the sorting table is greater than a preset threshold, parallel correction is performed to ensure that the smart car and the sorting table remain parallel, including the following sub-steps: S3.

1. Read the data of the smart car's attitude sensor. The data format is Euler quaternion (w, x, y, z), where w is the real part of the quaternion, which indicates the relationship between the rotation angle and the rotation axis, and determines the "size" or "amplitude" of the rotation. x, y, z are the imaginary parts of the quaternion, which are associated with the direction of the rotation axis, indicating the direction of the rotation axis and forming the spatial direction of the rotation. S3.

2. Convert the posture data expressed in the form of Euler quaternion into data expressed in angle (yaw x ,yawy,yaw z ), yaw x ,yawy,yaw z Respectively represent the angles between the current posture of the smart car and the positive directions of the x, y, and z axes; S3.

3. Target attitude angle when the smart car is parallel to the sorting platform x Subtract the current attitude angle yaw of the smart car obtained in step S3.2 x , find the deviation angle yawbias: <h2 style=";text-align:left;direction:ltr">yawbias=target<h2 style=";text-align:left;direction:ltr"> x <h2 style=";text-align:left;direction:ltr"> -yaw<h2 style=";text-align:left;direction:ltr"> x Among them, target x Indicates the target attitude angle when the smart car is parallel to the sorting platform, yaw x represents the current attitude angle obtained from the quaternion conversion in step S3.2, and yawbias represents the rotation angle difference between the current attitude of the smart car and the target attitude; Issue the command rot(yawbias,speed,torlance) to the smart car, which means the smart car rotates at the speed of speed y awbias degree, the allowable error is torlance; S3.

4. Due to the rotation accuracy error of the car, after correction once, if the yawbias value is greater than 1, the steps S3.1-S3.3 will be repeated continuously; each time the speed and torlance values ​​are reduced to make the yawbias value gradually converge. When the yawbias value is less than 1, the parallel correction is completed.

6. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S4, the posture of the smart car is adjusted using triangular path planning, including the following sub-steps: S4.

1. Control the smart car to rotate α degrees clockwise. Assume that the distance that the smart car needs to adjust is ybias, and the horizontal deviation between the car and the center point of the target cargo is xbias. S4.

2. Control the smart car to follow a triangular path, correct the deviation between the car and the target cargo, control the smart car to move forward a distance u, rotate counterclockwise a degrees, and then move back a distance v: S4.

3. When the required extension distance and deflection angle of the robot arm are still unreachable, repeat steps S4.1 to S4.2 until the required extension distance and deflection angle of the robot arm of the smart car are within the reachable range, and perform parallel correction again to ensure that the smart car and the sorting platform remain parallel, thereby improving the grasping success rate.

7. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S5, the deflection angle grasping is specifically as follows: when the grasping extension distance required by the smart car is less than or equal to the maximum extension distance of the robot arm, the position of the center point of the grasped cargo is calculated by the position of the center point of the Apriltag code; Perform deflection angle grabbing, specifically: Assuming that all goods have the same size, the x-axis offset between the center point of the camera at the current position of the smart car and the center point of the Apriltag code is d x , the y-axis offset is d y , the measured x-axis offset of the Apriltag code center point from the target cargo center point is x1, and the y-axis offset is y1; increase the x-coordinate of the current position of the robot arm by x1-d x , the y coordinate increases by y1-d y , move the end of the robot arm to the three-dimensional coordinate system with the robot arm base as the origin of the coordinate system (x0+x1-d x ,y0+y1-d y , z0), where x0, y0, and z0 are the three-dimensional coordinates of the center point of the suction cup of the robotic arm before grasping in the current coordinate system.

8. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S6, a two-stage recognition strategy is used to recognize the target cargo QR code: In the first stage, the distance between the smart car and the sorting platform is determined by identifying the Apriltag code, and the end of the robotic arm is controlled to move to the preset identification position to ensure that the QR code captured by the fixed-focus camera is in clear focus. During the movement of the robotic arm, the camera collects and identifies the QR code images on the goods multiple times. After reaching the identification position, the camera continues to identify the QR code until the email address information contained in the QR code is identified or the identification time exceeds the preset threshold. Then the robotic arm continues to extend to the grasping position, and the first stage of identification ends. If the email address information is not recognized in the first stage, the QR code recognition will be required again in the second stage. In the second stage, after grabbing the goods, the robotic arm moves the robotic arm to place the goods on the loading platform of the smart car. During the movement, the camera will continue to recognize the QR code multiple times until the delivery address information is successfully recognized; if the email address information has been recognized in the first stage, the recognition step of the second stage will be skipped for quick grabbing.

9. The ROS-based intelligent path planning and cargo side delivery method according to claim 1 is characterized in that: In step S7, different delivery path plans are selected according to different distribution conditions of the cargo locations, and cargo delivery is performed by a lateral delivery method, which specifically includes the following sub-steps: S7.1, delivery path planning: Based on the address information recognized by the QR code, a shortest path is planned from the current smart car position to each target delivery position until the next cargo grabbing position is reached, and the cargo is delivered in sequence; S7.2, delivery sequence and side delivery direction planning: Assuming that the target delivery points can be connected into a polygon of any shape, without considering the straight line situation, two delivery plans can be planned: clockwise and counterclockwise. Deliver to each target delivery point on the polygon area in turn according to the planned path. The left delivery plan is used for clockwise delivery and the right delivery plan is used for counterclockwise delivery. S7.

3. During the delivery process, assuming that multiple target locations that need to be delivered are on a straight line, after directly controlling the intelligent car to make parallel corrections, for all delivery points on the straight line, directly control the intelligent car to move forward or backward a specific distance and then rotate the robotic arm to achieve delivery, replacing the original navigation solution and speeding up the delivery.

10. An intelligent path planning and cargo side-dropping intelligent vehicle based on ROS, which performs path planning and cargo side-dropping by the method according to any one of claims 1 to 9, characterized in that: include: Mapping and navigation module, posture adjustment module, robotic arm control and grasping module, and smart car vision module; The robot arm control and grasping module, through hardware modification, installs a downward suction cup bracket with a camera on the smart car robot arm, performs optimal path planning for the smart car robot arm, and controls the rotation angle of the robot arm to grasp the goods and drop the goods to the designated area; The mapping and navigation module uses the GMAPPING method to locate the relative position of obstacles to the smart car by combining the odometer data with the distance information scanned by the laser radar, and completes the construction of the map; the map environment of each particle's position is matched with the environment information scanned by the laser radar through the AMCL method and weights are assigned to locate the smart car; The posture adjustment module is used to obtain the current posture of the intelligent car and perform fine posture adjustment through a triangular path planning method; The smart car vision module is used to obtain a picture of the target goods, perform picture preprocessing to crop out the QR code portion, and then use a two-stage QR code recognition method based on a deep neural network to recognize the QR code of the target goods.

Images