Sorting robot planning system and method based on visual identification

Abstract

The invention discloses a sorting robot planning system and method based on visual identification, and relates to the technical field of machine vision and sorting robot control, and the method comprises the following steps: building a unified space-time coordinate system between a sorting robot and a conveyor belt; predicting parcel arrival time and a robot return path, and generating an optimal snapshot time window and a path planning strategy; adjusting camera parameters in real time according to parameters output by the dynamic acquisition strategy generator, and ensuring image definition under high-speed motion; image acquisition is executed, the image quality is improved through multi-frame fusion and super-resolution reconstruction, and the effectiveness of a snapshot result is evaluated; according to the planning result and the image processing result, the sorting robot is controlled to move and execute the sorting task, and the method has the advantages of being high in practicability, efficient and intelligent.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision and sorting robot control, and specifically provides a sorting robot planning system and method based on visual recognition. Background Art

[0002] With the rapid development of e-commerce and intelligent manufacturing, the logistics sorting system is facing the severe challenge of efficiently processing a huge number of packages. Traditional sorting robots usually adopt a serial working mode of "sorting - pausing - collecting". During the package information collection link, it is necessary to significantly reduce the conveyor belt speed or even suspend operation, resulting in a bottleneck in sorting efficiency and making it difficult to seek new breakthroughs. Especially when dealing with business peaks such as "Double Eleven", this interruptive operation mode seriously restricts the overall throughput and causes congestion at logistics nodes.

[0003] In order to improve the dynamic capture ability, the existing technology mainly relies on hardware upgrade solutions: one is to use industrial cameras with a thousand-frame level to capture moving targets, but the unit price of such devices exceeds $20,000, and the supporting dedicated image processing system leads to a sharp increase in transformation costs; the other is to install auxiliary positioning devices such as lidar and ToF sensors, but they are vulnerable to reflection and dust interference in complex environments. Therefore, it is necessary to design a sorting robot planning system and method based on visual recognition with strong practicability and high efficiency and intelligence. Summary of the Invention

[0004] The purpose of the present invention is to provide a sorting robot planning system and method based on visual recognition to solve the problems raised in the above background art.

[0005] To solve the above technical problems, the present invention provides the following technical solution: A sorting robot planning method based on visual recognition, including the following steps:

[0006] Establish a unified spatio-temporal coordinate system between the sorting robot and the conveyor belt;

[0007] Predict the package arrival time and the robot return path, and generate the optimal capture time window and path planning strategy;

[0008] According to the parameters output by the dynamic acquisition strategy generator, adjust the camera parameters in real time to ensure the image clarity under high-speed movement;

[0009] Execute image acquisition, and improve the image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture result;

[0010] According to the planning result and the image processing result, control the movement of the sorting robot and execute the sorting task.

[0011] According to the above technical solution, the specific method for establishing a unified spatio-temporal coordinate system between the sorting robot and the conveyor belt is:

[0012] First, taking the starting point of the conveyor belt as the origin, the conveying direction as the X-axis, and the vertical direction as the Y-axis, establish a conveyor belt coordinate system, and convert the package position x through the number of encoder pulses n b , and its conversion expression is: x b = n·k p , where x b represents the package position on the conveyor belt, n is the number of encoder pulses, and k p is the conversion coefficient from encoder pulses to distance;

[0013] Then establish a robot coordinate system. Based on the inverse kinematics model of the robotic arm, calculate the end position of the robotic arm through the joint angles θ 1 , θ 2 , z. The calculation expression is: where the angle of joint 1, i.e., θ 1 is determined by the horizontal position, and the angle of joint 2, i.e., θ 2 is calculated from the geometric relationship between the robotic arm length and the target position. L 1 , L 2 are the lengths of the first and second segments of the robotic arm respectively;

[0014] Finally, through the preset conversion matrix map the conveyor belt position to the robot coordinate system to achieve position synchronization between the two.

[0015] According to the above technical solution, the specific method for establishing a unified spatio-temporal coordinate system between the sorting robot and the conveyor belt further includes:

[0016] Adopt PTP to synchronize the clocks of the robot controller and the conveyor belt encoder. The deviation compensation formula is: where t s is the clock synchronization error compensation amount, is the encoder timestamp of the i-th measurement, is the robot controller timestamp of the i-th measurement, and N is the sliding window size. This deviation is automatically compensated in subsequent calculations.

[0017] According to the above technical solution, in predicting the package arrival time and the robot return path and generating the optimal capture time window and path planning strategy, the specific method for generating the optimal capture time window is:

[0018] Obtain the current conveyor belt speed v b , the current package position x b , the current robotic arm angles θ 1 , θ 2 and the maximum joint angle Then, according to the conveyor belt speed v b and the remaining distance Lr , the time when the package reaches the sorting port is calculated through the formula , where L r is the remaining distance, which is obtained by subtracting the current position of the package from the total length of the conveyor belt;

[0019] According to the difference in joint angles between the current joint angle of the robotic arm and the joint angle of the target sorting starting position, as well as the maximum joint angular velocity, calculate the time required for the robotic arm to return. The calculation expression is: In the formula, Δθ 1 , Δθ 2 is the difference in angles that joints 1 and 2 need to rotate, is the maximum joint angle;

[0020] Set the safety margin value ΔT s , and obtain the capture time window T by subtracting the return time of the robotic arm and the safety margin from the arrival time of the package c . If the capture time window T c > 0, generate a capture task and enter path optimization; if the capture time window is less than or equal to 0 for 3 consecutive times, trigger the conveyor belt to decelerate.

[0021] According to the above technical solution, in predicting the package arrival time and the robot return path, and generating the optimal capture time window and path planning strategy, the specific method for generating the path planning strategy is:

[0022] Model the environment, divide the reachable space of the robotic arm into a grid of m 0 × m 0 × m 0 , mark the obstacle areas, and define the joint angle limits and the maximum end acceleration;

[0023] Perform cost function operations, comprehensively consider the time cost, energy consumption cost, and jitter cost, and calculate the comprehensive cost using the weighted sum method. The time cost is related to the path length and the robotic arm speed, the energy consumption cost is related to the joint angle change and the angular velocity, and the jitter cost is related to the integral of the square of the second derivative of the joint angle;

[0024] Use the A* algorithm for path search, select the node with the smallest F value (G value + H value) as the current node, continuously generate adjacent nodes and update their costs until the target point is found. Then, use the B-spline curve to fit the discrete path points.

[0025] According to the above technical solution, the specific method for adjusting the camera parameters in real time according to the parameters output by the dynamic acquisition strategy generator is:

[0026] Adjust the shutter time: First, calculate the relative velocity v rel , v rel = v b + vr · cosθ, where the relative speed refers to the conveyor belt speed v b and the robotic arm speed v r at the resultant velocity in the angle θ between the direction of robotic arm movement and the conveyor belt; then set the shutter time according to the relative speed. The principle for setting the shutter time is to minimize motion blur while ensuring not exceeding the maximum shutter time limit of the camera. Its calculation expression is where S p represents the actual size represented by each pixel, and t max is the maximum shutter time of the camera;

[0027] Auto - focus optimization: Designate a 200×200 pixel area at the center of the image as the pre - focus area. Within the pre - focus area, use the formula: Contrast = ∑ x,y |I(x + 1,y)-I(x,y)|+|I(x,y + 1)-I(x,y)| to calculate the gray - level gradient pixel - by - pixel within the focus area, and then search for the position with the maximum contrast;

[0028] Adjust the focus step based on the contrast change rate. The adjustment of the focus step is based on the contrast change rate, that is, the change rate of contrast when the focus position changes. The specific calculation formula is where k is an adjustment coefficient, is the gradient of the contrast changing with the position.

[0029] According to the above technical solution, the specific method for performing image acquisition, improving image quality through multi - frame fusion and super - resolution reconstruction, and evaluating the effectiveness of the captured results includes:

[0030] Calculate the displacement of each frame relative to the previous frame through the formula where λ is a constant, Δx is the displacement between adjacent frames, with the unit of pixel. Then align the multi - frame images according to the displacement information. Finally, fuse the aligned multi - frame images;

[0031] On the basis of multi - frame synthesis noise reduction, perform short - long frame fusion processing: According to the moving speed v b of the package, dynamically adjust the weight ratio of short frames and long frames.

[0032] According to the above technical solution, the specific method for performing image acquisition, improving image quality through multi - frame fusion and super - resolution reconstruction, and evaluating the effectiveness of the captured results also includes:

[0033] Perform quality assessment on the processed image. The evaluation indicators mainly include the clarity score Q s, this score is jointly determined by the Laplacian operator and the SSIM index; among them, the Laplacian operator is used to measure the edge sharpness of the image, while the SSIM index is used to measure the similarity between the image and the reference image;

[0034] Through the formula calculate the sharpness score of the image, where Q s is the image sharpness score, Laplacian(I) is the Laplacian gradient value, SSIM is the structural similarity index, and then, compare the score with the preset threshold Q min for comparison. When Q s >Q min , it is determined that the image quality is qualified, otherwise, increase the safety time margin ΔT s ; if it is unqualified for 3 consecutive times, reduce the conveyor belt speed.

[0035] A sorting robot planning system based on visual recognition, including:

[0036] A spatio-temporal joint modeling module, which is used to establish a unified spatio-temporal coordinate system between the sorting robot and the conveyor belt to accurately correlate the motion states of the two;

[0037] A dynamic acquisition strategy generator, which is used to predict the package arrival time and the robot return path according to the spatio-temporal model, and generate the best capture time window and path planning strategy;

[0038] An adaptive parameter controller, which is used to adjust the camera parameters in real time according to the parameters output by the dynamic acquisition strategy generator to ensure the image sharpness under high-speed motion;

[0039] An image acquisition and processing module, which is used to perform image acquisition, and improve the image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture results;

[0040] A control and execution module, which is used to control the motion of the sorting robot and execute the sorting task according to the planning results and the image processing results.

[0041] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: In the present invention, by converting the manipulator return time into an acquisition window through spatio-temporal collaborative planning, combining technologies such as multi-frame fusion and super-resolution reconstruction in image acquisition and processing to break through hardware limitations, and innovative path planning strategies, not only significantly improve the sorting efficiency, reduce hardware costs and dependencies, but also enhance the adaptability to complex environments and non-standard packages, improve the sorting reliability, optimize the overall performance of the system, and achieve high-speed dynamic sorting of "no speed reduction and no pause". Description of the Drawings

[0042] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, and do not constitute a limitation to the present invention.

[0043] In the accompanying drawings:

[0044] Figure 1 It is a schematic diagram of a sorting robot planning method based on visual recognition according to the present invention;

[0045] Figure 2 It is a schematic diagram of the composition of a sorting robot planning system based on visual recognition according to the present invention. Specific embodiments

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Embodiment 1

[0048] Please refer to Figure 1 , the present invention provides a technical solution: a sorting robot planning method based on visual recognition, including the following steps:

[0049] S101: Establish a unified spatio-temporal coordinate system between the sorting robot and the conveyor belt to accurately associate the motion states of the two, and provide a spatio-temporal reference for dynamic capture.

[0050] In the embodiment of the present invention, first, taking the starting point of the conveyor belt as the origin, the conveying direction as the X-axis, and the vertical direction as the Y-axis, a conveyor belt coordinate system is established. The position x of the package is converted through the encoder pulse number n b , and its conversion expression is: x b = n·k p , where x b represents the position of the package on the conveyor belt, n is the encoder pulse number, and k p is the conversion coefficient from encoder pulse to distance; then a robot coordinate system is established. Based on the inverse kinematics model of the robotic arm, the end position of the robotic arm is calculated through the joint angles θ 1 , θ 2 , z, and the calculation expression is: where the joint 1 angle, i.e., θ 1 is determined by the horizontal position, and the joint 2 angle, i.e., θ 2 is calculated from the geometric relationship between the robotic arm length and the target position, L 1 , L 2are the lengths of the first and second segments of the robotic arm respectively; finally, through a preset transformation matrix map the position of the conveyor belt to the robot coordinate system to achieve position synchronization between the two. Exemplarily:

[0051] Furthermore, the Precision Time Protocol (PTP) is used to synchronize the clocks of the robot controller and the conveyor belt encoder. The deviation compensation formula is: where t s is the clock synchronization error compensation amount, is the encoder timestamp of the i-th measurement, is the robot controller timestamp of the i-th measurement, N is the size of the sliding window, ensuring that the clock deviation between the conveyor belt encoder and the robot controller is less than 1 ms. For example, if the sliding window size N = 10 and the measurement deviation amounts for 10 times are [1.2, 0.8, -0.5, …, 1.1] ms, then the synchronization error This deviation is automatically compensated in subsequent calculations.

[0052] S102: Predict the package arrival time and the robot return path according to the spatio-temporal model, and generate the optimal capture time window and path planning strategy.

[0053] In the embodiment of the present invention, first obtain the current conveyor belt speed v b , the current position x of the package b , the current angle θ of the robotic arm 1 , θ 2 and the maximum joint angle Then, according to the conveyor belt speed v b and the remaining distance L r , through the formula calculate the time for the package to reach the sorting port, where L r is the remaining distance, which is obtained by subtracting the current position of the package from the total length of the conveyor belt.

[0054] Furthermore, according to the joint angle difference between the current joint angle of the robotic arm and the joint angle of the target sorting starting position, and the maximum joint angular velocity, calculate the time required for the robotic arm to return. Its calculation expression is: In the formula, Δθ 1 , Δθ 2 is the angle difference that joints 1 and 2 need to rotate, is the maximum joint angle. Exemplarily, if the maximum joint angular velocity when it is necessary to rotate Δθ 1 = 45°, then T r = 0.5 s.

[0055] Finally, set the safety margin value ΔT s, the capture time window T is obtained by subtracting the manipulator return time and the safety margin from the package arrival time. c . If the capture time window T c > 0, a capture task is generated and path optimization is entered; if the capture time window is less than or equal to 0 for three consecutive times, the conveyor belt speed is reduced.

[0056] In the embodiment of the present invention, the path planning strategy specifically includes:

[0057] Model the environment, divide the reachable space of the manipulator into a grid of m 0 ×m 0 ×m 0 , mark the obstacle area, and define motion constraints such as joint angle limits and maximum end acceleration;

[0058] Perform cost function operations, comprehensively consider time cost, energy consumption cost and jitter cost, and calculate the comprehensive cost using the weighted sum method, where the time cost is related to the path length and the manipulator speed, the energy consumption cost is related to the joint angle change and the angular velocity, and the jitter cost is related to the integral of the square of the second derivative of the joint angle;

[0059] Use the A* algorithm to perform path search, select the node with the smallest F value (G value + H value) as the current node, continuously generate adjacent nodes and update their costs until the target point is found. Then, use the B-spline curve to fit the discrete path points to ensure smooth motion and limit the curvature radius to avoid manipulator jitter, thereby realizing path search and optimization;

[0060] Finally, call the robot kinematic model to verify whether the path intersects with obstacles. If there is a collision, re-search the path. Finally, output the results such as the final path point sequence, total time and total energy consumption.

[0061] Step 103: According to the parameters output by the dynamic acquisition strategy generator, adjust the camera parameters in real time to ensure the image clarity under high-speed motion.

[0062] Exemplarily, in the embodiment of the present invention, when dynamically adjusting the shutter time, first calculate the relative velocity v rel , v rel = v b + v r ·cosθ, where the relative velocity refers to the resultant velocity of the conveyor belt speed v b and the manipulator speed v r at the included angle θ between the manipulator motion direction and the conveyor belt; then set the shutter time according to the relative velocity. The principle of setting the shutter time is to minimize motion blur while ensuring that it does not exceed the maximum shutter time limit of the camera. Its calculation expression is where S pRepresents the actual size represented by each pixel, t max Is the maximum shutter time of the camera. Exemplarily, if the relative speed v rel = 3.17 m / s, the actual size S represented by each pixel p = 0.1 mm / px, and the maximum shutter time t of the camera max = 1 / 1000 s, then t k = However, the actual value will be the nearest value supported by the camera, which is 0.1 ms, to ensure image clarity.

[0063] Furthermore, during the autofocus optimization, first, a 200×200 pixel area is demarcated at the center of the image as the pre-focus area. Within the pre-focus area, through the formula: Contrast = Σ x,y |I(x + 1, y) - I(x, y)| + |I(x, y + 1) - I(x, y)|, the gray-scale gradient is calculated pixel by pixel within the focus area; then the position with the maximum contrast is searched for, because an area with high contrast usually means clearer image details and more accurate focusing. To optimize the focusing process, the focusing step size is adjusted. The adjustment of the focusing step size is based on the contrast change rate, that is, the change rate of the contrast when the focusing position changes. The specific calculation formula is where k is an adjustment coefficient, is the gradient of the contrast changing with the position. Adjusting the focusing step size through the contrast change rate can find the optimal focusing position faster, improving the focusing efficiency and accuracy. Finally, the adaptive parameter controller effectively reduces the image blur caused by the rapid movement of the object by calculating the relative speed and optimizing the shutter time in real time. At the same time, through the contrast detection method and the optimization of the focusing step size, fast and accurate autofocus is achieved, ensuring clear images can be captured under different motion states.

[0064] Step 104: Perform image acquisition, and improve the image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of the capture result.

[0065] Specifically, images of the target area are captured by a high-precision camera. After multiple frames of images are captured, multi-frame synthesis and noise reduction processing are performed. First, the improved optical flow method is used to calculate the inter-frame displacement Δx. This method can accurately estimate the motion situation between adjacent frames. Through the formula where λ is a constant, Δx is the displacement amount between adjacent frames, in pixels; the displacement of each frame relative to the previous frame can be calculated. Then, based on this displacement information, the multiple frames of images are aligned to eliminate the blur and ghosting caused by motion. Finally, by fusing the aligned multiple frames of images, the noise can be effectively reduced, and the clarity and signal-to-noise ratio of the image can be improved.

[0066] On the basis of multi-frame synthesis noise reduction, the system will also perform short-long frame fusion processing. According to the moving speed v of the package b , dynamically adjust the weight ratio of short frames and long frames. When v b ≥2 m / s, the weight of the short frame α is set to 0.8, and the weight of the long frame is 1-α = 0.2. In this way, more information of the short frame can be retained when the object is moving fast to avoid motion blur caused by the long frame; while when the object is moving slowly, more information of the long frame is utilized to improve the detail performance of the image.

[0067] Next, the module will perform super-resolution reconstruction using the ESPCN network. In this embodiment, the ESPCN network is involved. The ESPCN network is an efficient convolutional neural network that can convert a low-resolution image into a high-resolution image. In this module, the input image is 640×480, and after being processed by the ESPCN network, the output is a high-resolution image of 1280×960. The network structure includes two convolutional layers and a pixel rearrangement layer (PixelShuffle). The first convolutional layer increases the number of channels of the input image from 3 to 64, the second convolutional layer reduces the number of channels from 64 to 32, and finally 2-fold upsampling is achieved through the pixel rearrangement layer. This network structure can effectively improve the resolution of the image while maintaining the image details.

[0068] Finally, the module will perform quality assessment on the processed image. The evaluation metrics mainly include the clarity score Q s , which is jointly determined by the Laplacian operator and the SSIM (structural similarity) index. Among them, the Laplacian operator is used to measure the edge clarity of the image, and the SSIM index is used to measure the similarity between the image and the reference image. Through the formula In the formula, Q s is the image clarity score, Laplacian(I) is the Laplacian gradient value, and SSIM is the structural similarity index. The clarity score of the image can be calculated. Then, the score is compared with the preset threshold Q min . When Q s >Q min , it is determined that the image quality is qualified; otherwise, the safety time margin ΔT s is increased. If it is unqualified for 3 consecutive times, the conveyor belt speed is reduced.

[0069] Step S105: Control the movement of the sorting robot according to the planning result and the image processing result to perform the sorting task.

[0070] Embodiment 2

[0071] Embodiment 2 of the present invention provides a sorting robot planning system based on visual recognition. Figure 2 It is a schematic diagram of the module composition for implementing the sorting robot planning system based on visual recognition provided in Embodiment 2 of the present invention. As Figure 2 shown, the system includes:

[0072] A spatio-temporal joint modeling module for establishing a unified spatio-temporal coordinate system between the sorting robot and the conveyor belt to accurately correlate their motion states;

[0073] A dynamic acquisition strategy generator for predicting the package arrival time and the robot return path according to the spatio-temporal model, and generating an optimal capture time window and a path planning strategy;

[0074] An adaptive parameter controller for adjusting parameters such as the camera shutter, focus, and ISO in real time according to the parameters output by the dynamic acquisition strategy generator to ensure the image clarity under high-speed movement;

[0075] An image acquisition and processing module for performing image acquisition, and improving the image quality through multi-frame fusion and super-resolution reconstruction, and evaluating the effectiveness of the capture results;

[0076] A control and execution module for controlling the movement of the sorting robot and performing sorting tasks according to the planning results and the image processing results.

[0077] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0078] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so as to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing steps for implementing the functions specified in one flow Figure 1 one process or a plurality of processes and / or boxes Figure 1 steps for implementing the functions specified in one box or a plurality of boxes.

[0080] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit of the present invention and the scope protected by the claims. All of these fall within the protection scope of the present invention.

Claims

1. A sorting robot planning method based on visual recognition, characterized in that: The following steps are involved: Establish a unified space-time coordinate system between the sorting robot and the conveyor belt; Predict the package arrival time and the robot's return path, and generate the best capture time window and path planning strategy; According to the parameters output by the dynamic acquisition strategy generator, the camera parameters are adjusted in real time to ensure image clarity under high-speed motion; Perform image acquisition, improve image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of capture results; According to the planning results and image processing results, the movement of the sorting robot is controlled to perform the sorting task.

2. A sorting robot planning method based on visual recognition according to claim 1, characterized in that: The specific method of establishing a unified space-time coordinate system between the sorting robot and the conveyor belt is: First, take the starting point of the conveyor belt as the origin, the conveying direction as the X axis, and the vertical direction as the Y axis to establish the conveyor belt coordinate system, and convert the package position x by the encoder pulse number n b , its conversion expression is: b =n·k p , where x b Indicates the position of the package on the conveyor belt, n is the number of encoder pulses, k p is the conversion factor from encoder pulse to distance; Then the robot coordinate system is established. Based on the inverse kinematics model of the robot arm, the end position of the robot arm is calculated through the joint angles θ1, θ2, and z. The calculation expression is: The angle of joint 1, i.e., θ1, is determined by the horizontal position, and the angle of joint 2, i.e., θ2, is calculated by the geometric relationship between the length of the robot arm and the target position. L1 and L2 are the lengths of the first and second segments of the robot arm, respectively. Finally, through the preset transformation matrix Map the conveyor belt position to the robot coordinate system to synchronize the positions of the two.

3. The method for planning a sorting robot based on visual recognition according to claim 2, characterized in that: The specific method of establishing a unified space-time coordinate system between the sorting robot and the conveyor belt also includes: PTP is used to synchronize the clocks of the robot controller and the conveyor encoder. The deviation compensation formula is: where t s is the clock synchronization error compensation, is the encoder timestamp of the ith measurement, is the timestamp of the robot controller at the i-th measurement, N is the sliding window size, and this deviation is automatically compensated in subsequent calculations.

4. The method for planning a sorting robot based on visual recognition according to claim 1, characterized in that: Predict the arrival time of the package and the return path of the robot, generate the optimal capture time window and path planning strategy, and the specific method of generating the optimal capture time window is as follows: Get the current conveyor belt speed v b , wrap the current position x b , the current angle of the robot arm θ1, θ2 and the maximum joint angle Then according to the conveyor belt speed v b and the remaining distance L r , through the formula Calculate the time it takes for the package to arrive at the sorting port, where L r is the remaining distance, obtained by subtracting the current position of the package from the total length of the conveyor belt; The time required for the robot to return is calculated based on the difference between the current joint angle of the robot and the joint angle of the target sorting starting position, as well as the maximum joint angular velocity. The calculation expression is: Where Δθ1, Δθ2 are the angle differences between joint 1 and joint 2. is the maximum angle of the joint; Set the safety margin value ΔT s , the capture time window T is obtained by subtracting the robot arm return time and safety margin from the package arrival time c ; If the capture time window T c >0, a snapshot task is generated and path optimization is started; if the snapshot time window is less than or equal to 0 for three consecutive times, the conveyor belt is triggered to slow down.

5. The method for planning a sorting robot based on visual recognition according to claim 1, characterized in that: Predict the arrival time of the package and the return path of the robot, generate the best capture time window and path planning strategy, and the specific method of generating the path planning strategy is as follows: Model the environment, divide the reachable space of the robot into a grid of m0×m0×m0, mark the obstacle area, and define the joint angle limit and the maximum acceleration of the end; Perform cost function calculation, comprehensively consider time cost, energy cost and jitter cost, and use weighted sum method to calculate the comprehensive cost, where time cost is related to path length and robot arm speed, energy cost is related to joint angle change and angular velocity, and jitter cost is related to the square integral of the second-order derivative of joint angle; Use the A* algorithm to search for paths, select the node with the smallest F value (G value + H value) as the current node, continuously generate adjacent nodes and update their costs until the target point is found, and then use the B-spline curve to fit the discrete path points.

6. The method for planning a sorting robot based on visual recognition according to claim 1, characterized in that: According to the parameters output by the dynamic acquisition strategy generator, the specific method of adjusting the camera parameters in real time is: Adjust shutter time: First calculate relative speed v rel , v rel =v b +v r ·cosθ, where the relative speed is the conveyor belt speed v b and the robot arm speed v r The total speed at the angle θ between the robot arm's movement direction and the conveyor belt; then the shutter time is set according to the relative speed. The principle of setting the shutter time is to minimize motion blur while ensuring that the maximum shutter time limit of the camera is not exceeded. The calculation expression is: Where S p Indicates the actual size represented by each pixel, t max The maximum shutter time of the camera; Autofocus optimization: A 200×200 pixel area is defined in the center of the image as the pre-focus area. Within the pre-focus area, the formula: Contrast = ∑ x,y |I(x+1,y)-I(x,y)|+|I(x,y+1)-I(x,y|, calculate the grayscale gradient pixel by pixel in the focus area, and then search for the position with the largest contrast; The focus step length is adjusted by the contrast change rate. The focus step length is adjusted based on the contrast change rate, that is, the rate of change of contrast when the focus position changes. The specific calculation formula is: Where k is an adjustment factor, is the gradient of contrast variation with position.

7. The method for planning a sorting robot based on visual recognition according to claim 1, characterized in that: Perform image acquisition and improve image quality through multi-frame fusion and super-resolution reconstruction. Specific methods for evaluating the effectiveness of snapshot results include: By formula Calculate the displacement of each frame relative to the previous frame, where λ is a constant and Δx is the displacement between adjacent frames in pixels. Then align multiple frames according to the displacement information. Finally, fuse the aligned multiple frames. Based on multi-frame synthesis noise reduction, short and long frame fusion processing is performed: according to the movement speed v of the package b , dynamically adjust the weight ratio of short frames and long frames.

8. The method for planning a sorting robot based on visual recognition according to claim 7, characterized in that: Perform image acquisition and improve image quality through multi-frame fusion and super-resolution reconstruction. Specific methods for evaluating the effectiveness of snapshot results also include: The quality of the processed image is evaluated. The evaluation indicators mainly include the clarity score Q s , the score is determined by the Laplacian operator and the SSIM index; the Laplacian operator is used to measure the edge clarity of the image, while the SSIM index is used to measure the similarity between the image and the reference image; By formula Calculate the image clarity score, where Q s is the image clarity score, Laplacian (I) is the Laplace gradient value, SSIM is the structural similarity index, and then the score is compared with the preset threshold Q min For comparison, when Q s >Q min If the image quality is qualified, the safety time margin ΔT is increased. s ; If it fails for 3 consecutive times, the conveyor belt speed will be reduced.

9. A sorting robot planning system based on visual recognition, characterized in that: include: The spatiotemporal joint modeling module is used to establish a unified spatiotemporal coordinate system between the sorting robot and the conveyor belt, and realize the precise correlation of the motion states of the two. Dynamic collection strategy generator, which is used to predict the package arrival time and the robot return path based on the spatiotemporal model, and generate the optimal capture time window and path planning strategy; Adaptive parameter controller, used to adjust camera parameters in real time according to the parameters output by the dynamic acquisition strategy generator to ensure image clarity under high-speed motion; Image acquisition and processing module, used to perform image acquisition, improve image quality through multi-frame fusion and super-resolution reconstruction, and evaluate the effectiveness of capture results; The control and execution module is used to control the movement of the sorting robot and perform the sorting task according to the planning results and image processing results.

Images