A vision-based sorting robot planning system and method

Abstract

This invention discloses a sorting robot planning system and method based on visual recognition, relating to the fields of machine vision and sorting robot control technology. The system includes the following steps: establishing a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt; predicting package arrival time and robot return path, generating an optimal capture time window and path planning strategy; adjusting camera parameters in real time based on parameters output by a dynamic acquisition strategy generator to ensure image clarity under high-speed motion; performing image acquisition and improving image quality through multi-frame fusion and super-resolution reconstruction, evaluating the effectiveness of the capture results; and controlling the movement of the sorting robot to perform sorting tasks based on the planning results and image processing results. This invention is characterized by strong practicality and high efficiency / intelligence.

Description

Technical Field

[0001] This invention relates to the field of machine vision and sorting robot control technology, specifically to a sorting robot planning system and method based on visual recognition. Background Technology

[0002] With the rapid development of e-commerce and intelligent manufacturing, logistics sorting systems face the severe challenge of efficiently handling massive volumes of parcels. Traditional sorting robots typically employ a serial "sort-pause-collect" work mode, requiring a significant reduction in conveyor belt speed or even a complete halt during parcel information collection, making it difficult to find new breakthroughs in sorting efficiency bottlenecks. Especially when dealing with peak business periods such as "Double Eleven," this interrupted operation mode severely restricts overall throughput and causes congestion at logistics nodes.

[0003] Current technologies for improving dynamic capture capabilities mainly rely on hardware upgrades: one is to use industrial cameras with a frame rate of over 1000 frames per second to capture moving targets, but such equipment costs over $20,000 per unit, and the accompanying dedicated image processing system leads to a surge in upgrade costs; the other is to add auxiliary positioning devices such as LiDAR and ToF sensors, but these are susceptible to interference from reflections and dust in complex environments. Therefore, it is essential to design a practical, efficient, and intelligent vision-based sorting robot planning system and method. Summary of the Invention

[0004] The purpose of this invention is to provide a sorting robot planning system and method based on visual recognition to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a sorting robot planning method based on visual recognition, comprising the following steps:

[0006] Establish a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt;

[0007] Predict the package arrival time and the robot's return path, and generate the optimal capture time window and path planning strategy;

[0008] Based on the parameters output by the dynamic acquisition strategy generator, the camera parameters are adjusted in real time to ensure image clarity under high-speed motion.

[0009] Perform image acquisition, improve image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture results;

[0010] Based on the planning results and image processing results, the movement of the sorting robot is controlled to perform the sorting task.

[0011] Based on the above technical solution, the specific method for establishing a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt is as follows:

[0012] First, establish a conveyor belt coordinate system with the starting point of the conveyor belt as the origin, the X-axis along the conveying direction as the X-axis, and the vertical direction as the Y-axis. Then, calculate the package position using the encoder pulse count n. Its conversion expression is: ,in This indicates the position of the package on the conveyor belt, where n is the number of encoder pulses. This is the conversion factor from encoder pulses to distance;

[0013] Then, a robot coordinate system is established, based on the inverse kinematics model of the robotic arm, and through joint angles... The end effector position of the robotic arm is calculated using the following expression: The angle of joint 1 is... The angle of joint 2 is determined by its horizontal position. Calculated based on the geometric relationship between the length of the robotic arm and the target position. These are the lengths of the first and second segments of the robotic arm, respectively.

[0014] Finally, through the preset transformation matrix The position of the conveyor belt is mapped to the robot coordinate system to achieve position synchronization between the two.

[0015] According to the above technical solution, the specific method for establishing a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt also includes:

[0016] The deviation compensation formula for using the clock of the PTP synchronous robot controller and the conveyor belt encoder is as follows: ,in This is the amount of compensation for clock synchronization error. Let i be the encoder timestamp for the i-th measurement. Let N be the timestamp of the robot controller during the i-th measurement, and N be the size of the sliding window, which will be automatically compensated for in subsequent calculations.

[0017] Based on the above technical solution, in predicting the package arrival time and the robot's return path, and generating the optimal capture time window and path planning strategy, the specific method for generating the optimal capture time window is as follows:

[0018] Get the current conveyor belt speed Package current location Current angle of the robotic arm and maximum joint angle Then, based on the conveyor belt speed and remaining distance Through formula Calculate the time it takes for the package to arrive at the sorting point, where The remaining distance is obtained by subtracting the current position of the package from the total length of the conveyor belt;

[0019] The time required for the robotic arm to return is calculated based on the difference between the current joint angle of the robotic arm and the joint angle at the starting position of the target sorting, as well as the maximum joint angular velocity. The calculation expression is as follows: In the formula The difference in the angle of rotation required for joint 1 and joint 2, This represents the maximum angle of the joint.

[0020] Set safety margin value The capture time window is obtained by subtracting the robotic arm's return time and safety margin from the package's arrival time. If the capture time window If the value is greater than 0, a capture task is generated and path optimization is initiated; if the capture time window is less than or equal to 0 for 3 consecutive times, the conveyor belt speed is reduced.

[0021] Based on the above technical solution, in predicting the package arrival time and the robot's return path, and generating the optimal capture time window and path planning strategy, the specific method for generating the path planning strategy is as follows:

[0022] Environmental modeling is performed, dividing the reachable space of the robotic arm into... × × The grid is used to mark obstacle areas and define joint angle limits and end-effector maximum acceleration.

[0023] The cost function is calculated by taking into account time cost, energy cost and jitter cost. The comprehensive cost is calculated by weighted summation. The time cost is related to the path length and the speed of the robotic arm, the energy cost is related to the joint angle change and angular velocity, and the jitter cost is related to the square integral of the second derivative of the joint angle.

[0024] The A* algorithm is used for path search. The node with the smallest F value (G value + H value) is selected as the current node. Neighboring nodes are continuously generated and their costs are updated until the target point is found. Then, B-spline curves are used to fit the discrete path points.

[0025] According to the above technical solution, the specific method for adjusting camera parameters in real time based on the parameters output by the dynamic acquisition strategy generator is as follows:

[0026] Adjusting shutter speed: First, calculate relative speed. , The relative speed refers to the speed of the conveyor belt. With robotic arm speed The angle between the direction of the robotic arm's movement and the conveyor belt The resultant velocity is calculated based on the relative velocity; then, the shutter speed is set according to the relative velocity. The principle for setting the shutter speed is to minimize motion blur while ensuring that it does not exceed the camera's maximum shutter speed limit. The calculation expression is as follows: ,in This indicates the actual size represented by each pixel. This is the camera's maximum shutter speed;

[0027] Autofocus optimization: Define a 200-degree angle at the center of the image. A 200-pixel area is designated as the pre-focus area. Within this area, the formula is applied: The grayscale gradient is calculated pixel by pixel within the focus area, and then the position with the highest contrast is searched.

[0028] The focus step size is adjusted by the contrast change rate, which is the rate at which the contrast changes as the focus position changes. The specific calculation formula is as follows: Where k is an adjustment factor. This represents the gradient of contrast as a function of position.

[0029] According to the above technical solution, the specific methods for performing image acquisition, improving image quality through multi-frame fusion and super-resolution reconstruction, and evaluating the effectiveness of the capture results include:

[0030] Through formula Calculate the displacement of each frame relative to the previous frame, where It is a constant. The displacement between adjacent frames is expressed in pixels. Then, the multiple frames are aligned based on the displacement information. Finally, the aligned multiple frames are fused together.

[0031] Based on multi-frame synthesis and noise reduction, short and long frame fusion processing is performed: according to the movement speed of the package. The weight ratio of short frames and long frames is dynamically adjusted.

[0032] According to the above technical solution, the specific method for performing image acquisition, improving image quality through multi-frame fusion and super-resolution reconstruction, and evaluating the effectiveness of the capture results further includes:

[0033] The quality of the processed images is evaluated, with the main evaluation metrics including sharpness score. The score is determined by the Laplacian operator and the SSIM index; the Laplacian operator is used to measure the edge sharpness of the image, while the SSIM index is used to measure the similarity between the image and the reference image.

[0034] Through formula Calculate the image sharpness score, where... Rate the image sharpness. This is the Laplace gradient value. The structural similarity index is then used, and the score is compared with a preset threshold. When comparing, If the image quality is satisfactory, the image quality is considered acceptable; otherwise, the safety margin is increased. If the conveyor belt fails three times in a row, reduce its speed.

[0035] A vision-based sorting robot planning system includes:

[0036] The spatiotemporal joint modeling module is used to establish a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt, so as to achieve a precise correlation between their motion states.

[0037] The dynamic acquisition strategy generator is used to predict the arrival time of the package and the return path of the robot based on the spatiotemporal model, and generate the best capture time window and path planning strategy.

[0038] An adaptive parameter controller is used to adjust camera parameters in real time based on the parameters output by the dynamic acquisition strategy generator to ensure image clarity under high-speed motion.

[0039] The image acquisition and processing module is used to perform image acquisition, improve image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture results.

[0040] The control and execution module is used to control the movement of the sorting robot and execute sorting tasks based on the planning results and image processing results.

[0041] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention transforms the return time of the robotic arm into a data acquisition window through spatiotemporal collaborative planning, and overcomes hardware limitations by combining multi-frame fusion and super-resolution reconstruction technologies in image acquisition and processing planning, as well as innovative path planning strategies. This not only significantly improves sorting efficiency and reduces hardware costs and dependence, but also enhances adaptability to complex environments and non-standard packages, improves sorting reliability, optimizes the overall system performance, and achieves high-speed dynamic sorting without slowing down or stopping. Attached Figure Description

[0042] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0043] Figure 1 This is a schematic diagram of a sorting robot planning method based on vision recognition according to the present invention;

[0044] Figure 2This is a schematic diagram of a sorting robot planning system based on visual recognition according to the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] Please see Figure 1 This invention provides a technical solution: a sorting robot planning method based on visual recognition, comprising the following steps:

[0048] S101: Establish a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt to achieve precise correlation between their motion states and provide a spatiotemporal reference for dynamic capture.

[0049] In this embodiment of the invention, a conveyor belt coordinate system is first established with the starting point of the conveyor belt as the origin, the conveying direction as the X-axis, and the vertical direction as the Y-axis. The package position is then calculated using the encoder pulse count n. Its conversion expression is: ,in This indicates the position of the package on the conveyor belt, where n is the number of encoder pulses. The encoder pulse to distance conversion coefficients are used; then, a robot coordinate system is established, based on the inverse kinematics model of the robotic arm, through joint angles. The end effector position of the robotic arm is calculated using the following expression: The angle of joint 1 is... The angle of joint 2 is determined by its horizontal position. Calculated based on the geometric relationship between the length of the robotic arm and the target position. These represent the lengths of the first and second segments of the robotic arm, respectively; finally, a preset transformation matrix is ​​used. Mapping the conveyor belt position to the robot coordinate system synchronizes their positions. An example is: .

[0050] Furthermore, PTP (Precision Time Protocol) is used to synchronize the clocks of the robot controller and the conveyor encoder. The deviation compensation formula is as follows: ,in This is the amount of compensation for clock synchronization error. Let i be the encoder timestamp for the i-th measurement. Let N be the robot controller timestamp for the i-th measurement, and N be the sliding window size. This ensures the clock deviation between the conveyor encoder and the robot controller is less than 1ms. For example, if the sliding window size N=10, the deviation after 10 measurements is... Then the synchronization error This deviation will be automatically compensated for in subsequent calculations.

[0051] S102: Based on the spatiotemporal model, predict the arrival time of the package and the return path of the robot, and generate the best capture time window and path planning strategy.

[0052] In this embodiment of the invention, the current conveyor belt speed is first obtained. Package current location Current angle of the robotic arm and maximum joint angle Then, based on the conveyor belt speed and remaining distance Through formula Calculate the time it takes for the package to arrive at the sorting point, where The remaining distance is obtained by subtracting the current position of the package from the total length of the conveyor belt.

[0053] Furthermore, based on the difference between the current joint angle of the robotic arm and the joint angle at the target sorting starting position, as well as the maximum joint angular velocity, the time required for the robotic arm to return is calculated. The calculation expression is as follows: In the formula The difference in the angle of rotation required for joint 1 and joint 2, For example, the maximum joint angle, if the maximum joint angular velocity... When rotation is required ,but .

[0054] Finally, set the safety margin value. The capture time window is obtained by subtracting the robotic arm's return time and safety margin from the package's arrival time. If the capture time window If the value is greater than 0, a capture task is generated and path optimization is initiated; if the capture time window is less than or equal to 0 for 3 consecutive times, the conveyor belt speed is reduced.

[0055] In this embodiment of the invention, the path planning strategy specifically includes:

[0056] Environmental modeling is performed, dividing the reachable space of the robotic arm into... × × The grid is used to mark obstacle areas and define motion constraints such as joint angle limits and end-effector maximum acceleration.

[0057] The cost function is calculated by taking into account time cost, energy cost and jitter cost. The comprehensive cost is calculated by weighted summation. The time cost is related to the path length and the speed of the robotic arm, the energy cost is related to the joint angle change and angular velocity, and the jitter cost is related to the square integral of the second derivative of the joint angle.

[0058] The A* algorithm is used for path search, selecting the node with the smallest F-value (G-value + H-value) as the current node. Neighboring nodes are continuously generated and their costs are updated until the target point is found. Then, B-spline curves are used to fit the discrete path points, ensuring smooth motion and limiting the radius of curvature to prevent robot arm jitter, thus achieving path search and optimization.

[0059] Finally, the robot's kinematics model is called to verify whether the path intersects with obstacles. If a collision occurs, the path search is restarted. The final result, including the final path point sequence, total time, and total energy consumption, is then output.

[0060] Step 103: Adjust camera parameters in real time according to the parameters output by the dynamic acquisition strategy generator to ensure image clarity under high-speed motion.

[0061] For example, in this embodiment of the invention, when dynamically adjusting the shutter speed, the relative speed is first calculated. , The relative speed refers to the speed of the conveyor belt. With robotic arm speed The angle between the direction of the robotic arm's movement and the conveyor belt The resultant velocity is calculated based on the relative velocity; then, the shutter speed is set according to the relative velocity. The principle for setting the shutter speed is to minimize motion blur while ensuring that it does not exceed the camera's maximum shutter speed limit. The calculation expression is as follows: ,in This indicates the actual size represented by each pixel. For example, the maximum shutter speed of the camera, if the relative speed... =3.17m / s, the actual size represented by each pixel =0.1mm / px, the camera's maximum shutter speed At that time, = However, the actual value will be the closest value supported by the camera, which is 0.1ms, to ensure image clarity.

[0062] Furthermore, during autofocus optimization, a 200-degree angle is first defined at the center of the image. A 200-pixel area is designated as the pre-focus area. Within this area, the formula is applied: The process involves calculating the grayscale gradient pixel-by-pixel within the focus area; then, it searches for the location with the highest contrast, as high-contrast areas typically indicate sharper image details and more accurate focus. To optimize the focusing process, the focus step size is adjusted based on the contrast change rate, i.e., the rate at which the contrast changes as the focus position changes. The specific calculation formula is as follows: Where k is an adjustment factor. This represents the contrast gradient as a function of position. Adjusting the focus step size using the contrast change rate allows for faster finding of the optimal focus position, improving focusing efficiency and accuracy. Finally, the adaptive parameter controller effectively reduces image blur caused by rapid object movement by calculating relative speed in real time and optimizing shutter speed. Simultaneously, through contrast detection and focus step size optimization, fast and accurate autofocus is achieved, ensuring sharp images are captured under various motion conditions.

[0063] Step 104: Perform image acquisition, improve image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture results.

[0064] Specifically, images of the target area are captured using a high-precision camera. After acquiring multiple frames, multi-frame synthesis and noise reduction processing is performed. First, an improved optical flow method is used to calculate the inter-frame displacement. This method can accurately estimate the motion between adjacent frames. (Using the formula...) ,in It is a constant. This represents the displacement between adjacent frames, expressed in pixels; the displacement of each frame relative to the previous frame can be calculated. Then, multiple frames are aligned based on this displacement information to eliminate blur and ghosting caused by motion. Finally, by fusing the aligned multiple frames, noise can be effectively reduced, and image sharpness and signal-to-noise ratio can be improved.

[0065] In addition to multi-frame synthesis and noise reduction, the system also performs short-long frame fusion processing. This is based on the package's movement speed. Dynamically adjust the weight ratio between short and long frames. At that time, short frame weight Set to 0.8, the weight for longer frames is 1- =0.2. In this way, more information from short frames can be preserved when an object is moving quickly, thus avoiding motion blur caused by long frames; while when the object is moving slowly, more information from long frames can be utilized to improve the detail of the image.

[0066] Next, the module utilizes the ESPCN network for super-resolution reconstruction. In this embodiment, the ESPCN network is involved; it's a highly efficient convolutional neural network capable of converting low-resolution images into high-resolution ones. In this module, the input image is 640×480, and after processing by the ESPCN network, the output is a high-resolution image of 1280×960. The network structure includes two convolutional layers and a pixel shaving layer. The first convolutional layer increases the number of channels in the input image from 3 to 64, the second convolutional layer reduces the number of channels from 64 to 32, and finally, the pixel shaving layer achieves a 2x upsampling. This network structure effectively improves image resolution while preserving image details.

[0067] Finally, the module performs a quality assessment on the processed image. The main assessment metrics include sharpness score. The score is determined by both the Laplacian operator and the SSIM (Structural Similarity) index. The Laplacian operator measures the edge sharpness of the image, while the SSIM index measures the similarity between the image and a reference image. This is achieved through the formula... In the formula Rate the image sharpness. This is the Laplace gradient value. Using a structural similarity index, an image sharpness score can be calculated. Then, the score is compared with a preset threshold. When comparing, If the image quality is satisfactory, the image quality is considered acceptable; otherwise, the safety margin is increased. If the conveyor belt fails three times in a row, its speed will be reduced.

[0068] Step S105: Based on the planning results and image processing results, control the movement of the sorting robot to perform the sorting task.

[0069] Example 2

[0070] Embodiment 2 of the present invention provides a sorting robot planning system based on vision recognition. Figure 2 This is a schematic diagram of the module composition of the vision recognition-based sorting robot planning system provided in Embodiment 2 of the present invention, as shown below. Figure 2 As shown, the system includes:

[0071] The spatiotemporal joint modeling module is used to establish a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt, so as to achieve a precise correlation between their motion states.

[0072] The dynamic acquisition strategy generator is used to predict the arrival time of the package and the return path of the robot based on the spatiotemporal model, and generate the best capture time window and path planning strategy.

[0073] An adaptive parameter controller is used to adjust camera shutter speed, focus, ISO and other parameters in real time based on the parameters output by the dynamic acquisition strategy generator to ensure image clarity under high-speed motion.

[0074] The image acquisition and processing module is used to perform image acquisition, improve image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture results.

[0075] The control and execution module is used to control the movement of the sorting robot and execute sorting tasks based on the planning results and image processing results.

[0076] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A sorting robot planning method based on vision recognition, characterized in that: Includes the following steps: Establish a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt; Predict the package arrival time and the robot's return path, and generate the optimal capture time window and path planning strategy; Based on the parameters output by the dynamic acquisition strategy generator, the camera parameters are adjusted in real time to ensure image clarity under high-speed motion. Perform image acquisition, improve image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture results; Based on the planning and image processing results, the movement of the sorting robot is controlled to perform the sorting task; In predicting package arrival time and robot return path, and generating the optimal capture time window and path planning strategy, the specific method for generating the optimal capture time window is as follows: Get the current conveyor belt speed Package current location Current angle of the robotic arm and maximum joint angle Then, based on the conveyor belt speed and remaining distance Through formula Calculate the time it takes for the package to arrive at the sorting point, where The remaining distance is obtained by subtracting the current position of the package from the total length of the conveyor belt; The time required for the robotic arm to return is calculated based on the difference between the current joint angle of the robotic arm and the joint angle at the starting position of the target sorting, as well as the maximum joint angular velocity. The calculation expression is as follows: In the formula The difference in the angle of rotation required for joint 1 and joint 2, This represents the maximum angle of the joint. Set safety margin value The capture time window is obtained by subtracting the robotic arm's return time and safety margin from the package's arrival time. If the capture time window If the value is greater than 0, a capture task is generated and path optimization is initiated; if the capture time window is less than or equal to 0 for 3 consecutive times, the conveyor belt speed is reduced.

2. The sorting robot planning method based on vision recognition according to claim 1, characterized in that: The specific method for establishing a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt is as follows: First, establish a conveyor belt coordinate system with the starting point of the conveyor belt as the origin, the X-axis along the conveying direction as the X-axis, and the vertical direction as the Y-axis. Then, calculate the package position using the encoder pulse count n. Its conversion expression is: ,in This indicates the position of the package on the conveyor belt, where n is the number of encoder pulses. This is the conversion factor from encoder pulses to distance; Then, a robot coordinate system is established, based on the inverse kinematics model of the robotic arm, and through joint angles... The end effector position of the robotic arm is calculated using the following expression: The angle of joint 1 is... The angle of joint 2 is determined by its horizontal position. Calculated based on the geometric relationship between the length of the robotic arm and the target position. These are the lengths of the first and second segments of the robotic arm, respectively. Finally, through the preset transformation matrix The position of the conveyor belt is mapped to the robot coordinate system to achieve position synchronization between the two.

3. The sorting robot planning method based on vision recognition according to claim 2, characterized in that: Specific methods for establishing a unified spatiotemporal coordinate system between sorting robots and conveyor belts also include: The deviation compensation formula for using the clock of the PTP synchronous robot controller and the conveyor belt encoder is as follows: ,in This is the amount of compensation for clock synchronization error. Let i be the encoder timestamp for the i-th measurement. Let N be the timestamp of the robot controller during the i-th measurement, and N be the size of the sliding window, which will be automatically compensated for in subsequent calculations.

4. The sorting robot planning method based on vision recognition according to claim 1, characterized in that: In predicting package arrival time and robot return path, and generating the optimal capture time window and path planning strategy, the specific method for generating the path planning strategy is as follows: Environmental modeling is performed, dividing the reachable space of the robotic arm into... × × The grid is used to mark obstacle areas and define joint angle limits and end-effector maximum acceleration. The cost function is calculated by taking into account time cost, energy cost and jitter cost. The comprehensive cost is calculated by weighted summation. The time cost is related to the path length and the speed of the robotic arm, the energy cost is related to the joint angle change and angular velocity, and the jitter cost is related to the square integral of the second derivative of the joint angle. The A* algorithm is used for path search. The node with the smallest F value is selected as the current node. Neighboring nodes are continuously generated and their costs are updated until the target point is found. Then, B-spline curves are used to fit the discrete path points.

5. The sorting robot planning method based on vision recognition according to claim 1, characterized in that: The specific method for adjusting camera parameters in real time based on the parameters output by the dynamic acquisition strategy generator is as follows: Adjusting shutter speed: First, calculate relative speed. , The relative speed refers to the speed of the conveyor belt. With robotic arm speed The angle between the direction of the robotic arm's movement and the conveyor belt The resultant velocity is calculated based on the relative velocity; then, the shutter speed is set according to the relative velocity. The principle for setting the shutter speed is to minimize motion blur while ensuring that it does not exceed the camera's maximum shutter speed limit. The calculation expression is as follows: ,in This indicates the actual size represented by each pixel. This is the camera's maximum shutter speed; Autofocus optimization: Define a 200-degree angle at the center of the image. A 200-pixel area is designated as the pre-focus area. Within this area, the formula is applied: The grayscale gradient is calculated pixel by pixel within the focus area, and then the position with the highest contrast is searched. The focus step size is adjusted by the contrast change rate, which is the rate at which the contrast changes as the focus position changes. The specific calculation formula is as follows: Where k is an adjustment factor. This represents the gradient of contrast as a function of position.

6. The sorting robot planning method based on vision recognition according to claim 1, characterized in that: Specific methods for performing image acquisition, improving image quality through multi-frame fusion and super-resolution reconstruction, and evaluating the effectiveness of capture results include: Through formula Calculate the displacement of each frame relative to the previous frame, where It is a constant. The displacement between adjacent frames is expressed in pixels. Then, the multiple frames are aligned based on the displacement information. Finally, the aligned multiple frames are fused together. Based on multi-frame synthesis and noise reduction, short and long frame fusion processing is performed: according to the movement speed of the package. The weight ratio of short frames and long frames is dynamically adjusted.

7. The sorting robot planning method based on vision recognition according to claim 6, characterized in that: Specific methods for performing image acquisition, improving image quality through multi-frame fusion and super-resolution reconstruction, and evaluating the effectiveness of capture results also include: The quality of the processed images is evaluated, with the main evaluation metrics including sharpness score. The score is determined by the Laplacian operator and the SSIM index; the Laplacian operator is used to measure the edge sharpness of the image, while the SSIM index is used to measure the similarity between the image and the reference image. Through formula Calculate the image sharpness score, where... Rate the image sharpness. This is the Laplace gradient value. The structural similarity index is then used, and the score is compared with a preset threshold. When comparing, If the image quality is satisfactory, the image quality is considered acceptable; otherwise, the safety margin is increased. If the conveyor belt fails three times in a row, reduce its speed.

8. A vision-based sorting robot planning system for implementing the method of claim 1, characterized in that: include: The spatiotemporal joint modeling module is used to establish a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt, so as to achieve a precise correlation between their motion states. The dynamic acquisition strategy generator is used to predict the arrival time of the package and the return path of the robot based on the spatiotemporal model, and generate the best capture time window and path planning strategy. An adaptive parameter controller is used to adjust camera parameters in real time based on the parameters output by the dynamic acquisition strategy generator to ensure image clarity under high-speed motion. The image acquisition and processing module is used to perform image acquisition, improve image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture results. The control and execution module is used to control the movement of the sorting robot and execute sorting tasks based on the planning results and image processing results.

Images