Robot sorting track self-optimization system based on reinforcement learning
By using a reinforcement learning-based robot sorting trajectory self-optimization system, the problem of insufficient trajectory optimization capability of traditional robot sorting systems in dynamic environments is solved. It realizes online adaptive optimization and multi-objective collaborative optimization, improves the accuracy and safety of sorting operations, and reduces the difficulty of system deployment and maintenance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LIAONING UNIVERSITY OF TECHNOLOGY
- Filing Date
- 2026-01-30
- Publication Date
- 2026-04-28
AI Technical Summary
Traditional robotic sorting systems suffer from poor environmental adaptability, limited trajectory optimization capabilities, a disconnect between perception and decision-making, and insufficient dynamic object processing capabilities when faced with dynamically changing working environments, diverse object shapes, and unstructured layouts. As a result, they struggle to achieve efficient and safe trajectory planning in complex and dynamic sorting tasks.
A robot sorting trajectory self-optimization system based on reinforcement learning is adopted. The system acquires state information through the perception module, constructs an instruction generation model through the data processing module, and drives the robot to run through the execution management module. This enables online adaptation and optimization of the trajectory, integrates global scene analysis and local real-time perception, and supports multi-objective collaborative optimization.
It enables online adaptive optimization of robot trajectories, improving robustness and intelligence in complex dynamic environments, enhancing the accuracy and flexibility of sorting operations, reducing the difficulty of system deployment and maintenance, supporting multi-objective collaborative optimization, extending equipment lifespan, and reducing operating costs.
Smart Images
Figure CN121928554A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of industrial robot control and artificial intelligence technology, specifically relating to a robot sorting trajectory self-optimization system based on reinforcement learning. Background Technology
[0002] Currently, robotic sorting systems are widely used in industrial production, particularly in warehousing and logistics, and flexible manufacturing. Traditional robotic sorting systems often employ pre-programmed fixed trajectories or closed-loop control methods based on simple vision guidance to grasp and sort objects at specific locations and in fixed postures. These systems typically suffer from poor adaptability, insufficient trajectory flexibility, and difficulty in handling sudden disturbances when facing dynamically changing working environments, diverse object shapes, and unstructured layouts. Specifically: Poor environmental adaptability: Traditional systems often rely on structured environmental assumptions, such as fixed lighting, known object positions, and static backgrounds. Once dynamic obstacles, changes in lighting, or random distribution of object positions appear in the environment, the system's recognition accuracy and trajectory execution reliability will significantly decrease.
[0003] Limited trajectory optimization capabilities: Existing systems typically employ offline programming or simple online path planning algorithms (such as A*, RRT, etc.), lacking self-learning and optimization mechanisms based on real-time environmental feedback. Path planning often only considers geometric obstacle avoidance, failing to fully integrate multi-objective optimization such as work efficiency, energy consumption, and mechanical impact, and is difficult to adjust in real time in dynamic environments.
[0004] The separation of perception and decision-making: In traditional architectures, the environmental perception module, path planning module, and control execution module are often designed independently, with information flow being unidirectional and lacking a collaborative optimization mechanism based on global scene understanding and local real-time feedback. This results in slow overall response speed and low level of decision-making intelligence when the system faces complex and dynamic sorting tasks.
[0005] Insufficient dynamic object handling capability: In collaborative scenarios involving conveyor belts, mobile AGVs, or other robots, traditional systems have a weak ability to predict and avoid the trajectory of dynamic obstacles, and usually adopt simple emergency stop or avoidance strategies, which seriously affects the continuity and efficiency of sorting operations.
[0006] With the development of artificial intelligence technology, reinforcement learning has shown great potential in the field of robot control due to its ability to learn optimal strategies through trial and error with the environment. However, existing reinforcement learning-based robot control solutions are mostly concentrated in simulation environments or single tasks. In complex, high-real-time industrial sorting scenarios, they still face challenges such as low sample efficiency, difficulty in designing reward functions, and the difficulty in balancing safety and real-time performance. Therefore, there is an urgent need for a robot sorting trajectory control system that can deeply integrate real-time perception, intelligent decision-making, and precise control, and has online self-optimization capabilities. Summary of the Invention
[0007] The present invention aims to propose a robot sorting trajectory self-optimization system based on reinforcement learning, and to solve the problems existing in the prior art.
[0008] To achieve the above objectives, the present invention provides the following solution: a robot sorting trajectory self-optimization system based on reinforcement learning, comprising: The sensing module is used to acquire status information of the sorting work scenario and the robot's operating information; The data processing module is used to construct an instruction generation model and obtain operating instructions based on the instruction generation model, the state information, and the operating information; the operating instructions include: robot movement instructions and robot action instructions; The execution management module is used to store the status information, the running information, and the running instructions, and to drive the robot to run based on the running instructions to complete the sorting action.
[0009] More preferably, the sensing module includes: a first data acquisition unit, a running data acquisition unit, and a second data acquisition unit; The first data acquisition unit is integrated on the robot body and is used to acquire the status information in the robot's operating scenario in real time; The operation data acquisition unit is used to acquire the operation information; The second data acquisition unit is used to acquire the status information of the complete sorting work scenario.
[0010] More preferably, the data processing module includes: a global data analysis unit, a local data analysis unit, a first instruction generation unit, and a second instruction generation unit; The global data analysis unit is connected to the second data acquisition unit and is used to analyze the status information of the complete sorting work scenario to obtain global analysis results. The local data analysis unit is connected to the first data acquisition unit and the operation data acquisition unit, and is used to analyze the state information and operation information in the robot operation scenario to obtain local analysis results; The first instruction generation unit is connected to the global data analysis unit and the local data analysis unit, and is used to generate the robot movement instruction based on the global analysis result and the local analysis result; The second instruction generation unit is connected to the local data analysis unit and is used to generate the robot action instructions based on the local analysis results.
[0011] More preferably, the global data analysis unit includes: an information splicing subunit and an object recognition subunit; the information splicing subunit is used to splice the state information of the complete sorting work scene obtained by several second data acquisition units to obtain a complete sorting work scene image; the object recognition subunit is used to perform object recognition and state recognition on the complete sorting work scene image to obtain global analysis results.
[0012] More preferably, the local data analysis unit is provided with the object recognition subunit, which is used to identify and classify the state information in the running scene collected on the robot's running path; it is also provided with an early warning subunit, which is used to judge the distance of dynamic objects, and when the distance exceeds the safe distance threshold, to re-plan the path or stop and wait until the distance is greater than the safe distance threshold, and then execute the aborted path.
[0013] More preferably, the execution management module includes: a storage unit and a controller unit; The storage unit is used to store the status information, the running information, and the running instructions; The controller unit is used to drive the robot to run based on the running instructions.
[0014] More preferably, both the first instruction generation unit and the second instruction generation unit generate corresponding instructions based on an instruction generation model, wherein the instruction generation model includes: a local instruction generation part and a global instruction generation part; the local instruction generation part is used to generate robot action instructions; and the global instruction generation part is used to generate robot movement instructions. The local instruction generation section includes: a classification block, a contour extraction block, a local path planning block, and a second actuator; the classification block is used to classify the sorted objects based on image recognition technology, and determine the endpoint of the local path planning based on the classification results; the contour extraction block identifies the object contour using an object contour recognition algorithm; the robot's end-effector sorting mechanism grasps the object based on the object contour recognition results; the second actuator converts the local planned path into action instructions, generating robot action instructions; The global instruction generation section includes: a spatial model building block, a trajectory prediction block, a global trajectory planning block, and a first actuator; the spatial model building block is used to build models of objects in the global analysis results to obtain a virtual original map with coordinate information; the trajectory prediction block is used to predict the trajectory of dynamic objects to obtain prediction results; the global trajectory planning block plans the robot's travel path based on the virtual original map and the prediction results to obtain a planned path; the first actuator converts the global planned path into action instructions to generate robot movement instructions.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Achieved online adaptive and continuous optimization of trajectory: This invention continuously interacts with the environment through a reinforcement learning framework, enabling automatic optimization of the robot's movement trajectory and action strategy based on real-time feedback. It not only generates feasible paths in the initial task but also accumulates experience through continuous learning, autonomously improving the efficiency, smoothness, and safety of the trajectory in subsequent tasks, thus evolving from "able to work" to "excellent work."
[0016] 2. Enhanced robustness and intelligence in complex dynamic environments: Through a two-layer architecture integrating global scene analysis and local real-time perception, the system can simultaneously grasp the overall layout and dynamic changes of the workspace. Combined with advanced trajectory prediction algorithms and dynamic path replanning mechanisms, the system can predict the trajectory of dynamic objects in advance and adjust its own path smoothly in real time, significantly improving the success rate and safety of operations in unstructured and dynamic environments.
[0017] 3. Achieving an efficient closed loop of perception-decision-execution: This invention deeply integrates the perception module, data processing module, and execution management module, constructing a complete closed loop from environmental information acquisition, state vector construction, multi-target instruction generation to precise motion control. Perceived information is not only used for initial decision-making but also continuously fed back to the reinforcement learning model for policy evaluation and updates, forming a virtuous self-optimizing cycle of "perception driving decision-making, execution verifying perception, and results optimizing the model."
[0018] 4. Supports multi-objective collaborative optimization: In path planning and motion generation, this invention comprehensively considers multiple performance indicators such as running time, energy consumption, and mechanical impact, and can output a series of Pareto optimal solutions. Based on actual needs, it selects the most suitable trajectory scheme through a decision-making method. This enables the system to further optimize comprehensive performance indicators, extend equipment life, and reduce operating costs while meeting basic sorting functions.
[0019] 5. Improve the accuracy and flexibility of sorting operations: Through high-precision object recognition, contour extraction, and local fine path planning, the system can accurately identify and classify various objects and plan precise grasping trajectories that are adapted to their shapes and postures, thereby improving the accuracy of sorting and the ability to handle irregularly shaped objects.
[0020] 6. Modular design enhances system scalability and maintainability: Each module and subunit of the system is clearly designed and has well-defined interfaces, which facilitates the replacement, upgrading or parameter adjustment of functional modules for different sorting scenarios, reduces the difficulty of system deployment and maintenance, and has good engineering application prospects. Attached Figure Description
[0021] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a schematic diagram of the robot sorting trajectory self-optimization system framework based on reinforcement learning, according to an embodiment of the present invention. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0025] Example 1: like Figure 1 As shown, this embodiment provides a robot sorting trajectory self-optimization system based on reinforcement learning, including: a perception module for acquiring state information of the sorting work scenario and robot operation information; a data processing module for constructing an instruction generation model and obtaining operation instructions based on the instruction generation model, state information, and operation information; the operation instructions include: robot movement instructions and robot action instructions; and an execution management module for storing state information, operation information, and operation instructions, and driving the robot to run based on the operation instructions to complete the sorting action.
[0026] A further implementation involves the perception module comprising: a first data acquisition unit, an operational data acquisition unit, and a second data acquisition unit; the first data acquisition unit is integrated on the robot body and is used to acquire real-time status information in the robot's operational scenario; the operational data acquisition unit is used to acquire operational information; and the second data acquisition unit is used to acquire status information of the complete sorting work scenario.
[0027] A further implementation involves a data processing module comprising: a global data analysis unit, a local data analysis unit, a first instruction generation unit, and a second instruction generation unit; the global data analysis unit is connected to the second data acquisition unit and is used to analyze the state information of the complete sorting work scenario to obtain global analysis results; the local data analysis unit is connected to the first data acquisition unit and the running data acquisition unit and is used to analyze the state information and running information in the robot running scenario to obtain local analysis results; the first instruction generation unit is connected to the global data analysis unit and the local data analysis unit and is used to generate robot movement instructions based on the global analysis results and the local analysis results; the second instruction generation unit is connected to the local data analysis unit and is used to generate robot action instructions based on the local analysis results.
[0028] Specifically, the global data analysis unit includes an information splicing subunit and an object recognition subunit. The information splicing subunit is used to splice the state information of the complete sorting work scene obtained by several second data acquisition units to obtain a complete sorting work scene image. Then, the object recognition subunit is used to perform object recognition and state recognition on the complete sorting work scene image to obtain the global analysis results.
[0029] The information stitching subunit uses an integration model to register and reconstruct the status information of several sorting work scenes to obtain a complete sorting work scene image. In this embodiment, the integration model includes a first stitching part and a second stitching part; the first stitching part is used to register the input image. The second stitching part is used to stitch the registered images to obtain a complete sorting work scene image.
[0030] The first stitching process, which involves registering the two input images, includes: First, input the two images. Fr , Ft (This embodiment uses) Fr For reference image, Ft The target image is sequentially input into ResNet and the feature pyramid to obtain first and second feature maps of different sizes; the size of the first feature map is one-eighth of the input image, and the size of the second feature map is one-sixteenth of the input image.
[0031] Subsequently, the first feature map is subjected to cross-attention matching, concatenation, regression, and direct linear transformation in sequence to obtain the first transformed feature map. A spatial transformation network is then used to transform the parameters of the first and second feature maps, performing two masked cross-attention matching, concatenation, regression, and thin-plate spline transformations in sequence to obtain the second transformed feature map.
[0032] In this embodiment, the first feature map is directly matched using cross-attention. In subsequent matching, two masking operations are performed on the non-overlapping regions. After the first masking operation, the similarity between the first transformed feature map and the second feature map is: ; In the formula, X i This represents the first transform feature map after the first masking. i Feature projection and second feature map j The similarity of all feature points among the feature projections; Q i Represents the first transformation feature map i The query vector; K j Representing the second feature map j The key vector; superscript T Indicates transpose; M 1 indicates the first masking operation; O j Represents the first transformation feature map i With the second feature map j The overlapping parts of the images.
[0033] in, ; In the formula, H and H -1 Represents homography transformation and inverse homography; C Indicates a cropping operation; E This represents a matrix of all ones with the same dimension as the current feature map.
[0034] At this point, only the second feature map is considered. j A masking operation was performed, and the second masking operation was applied to the first transformed feature map. i After the second masking operation, the similarity between the first transformed feature map and the second feature map is: ; In the formula, This indicates the second masking operation; V jRepresenting the second feature map j The value vector; This represents the multiplication of the elements of a matrix.
[0035] in, .
[0036] Finally, geometric transformations are performed on the target image and the second transformation feature map to obtain the registered image.
[0037] The loss function of the first stitching part mainly consists of two parts: the photometric consistency loss and the geometric consistency loss between the two images.
[0038] The second stitching process involves stitching the registered images together: First, the registered images are fused and reconstructed using a reconstruction network to obtain a stitched image. The stitched image is then upsampled to the original image size to obtain a sampled image. The sampled image and the registered image are then overlaid along their respective channels to obtain a superimposed image. Finally, the superimposed image is reconstructed using a reconstruction network to obtain the final, complete image of the sorting process.
[0039] In this embodiment, the reconstruction network branch adopts an encoder-decoder structure, while the reconstruction network uses a Mobile-Unet network. The reconstruction network primarily uses content masks and seam masks to guide the direction of image fusion. The seam mask is generated using UDIS++, and edge extraction is performed on it to obtain the optimal seam. The optimal seam is then expanded and ANDed with the seam mask to obtain the content mask. The loss function mainly includes content loss and seam loss, which, combined with their respective weights, form the loss function of the reconstruction network branch. The reconstruction network branch and the reconstruction network branch are primarily linked by a consistency loss.
[0040] Content loss includes: ; ; In the formula, L p Indicates perceived loss; S lr This represents a stitched image representing the output of the reconstructed network branches; , These represent the target image content mask and the reference image content mask for reconstructing the network branch, respectively; , These represent the distorted target image and reference image after reconstructing the network branches, respectively; This represents the multiplication of each element of the matrix; V represents the output features after passing through the VGG-19 network; || ||2 represents the L2 norm.
[0041] Seam losses include: ; In the formula, , represents the seam mask of the target image and reference image of the reconstructed network branch, respectively; || ||1 represents the L1 norm.
[0042] Consistency loss includes: ; In the formula, D This indicates an operation that adjusts the resolution of the reconstructed network branch to match the output resolution of the reconstructed network branch. S hr This represents a complete sorting scene image output from the reconstructed network branch. S hr1 This represents the registered image after reconstructing the input of the network branch.
[0043] The object recognition subunit uses a recognition model to identify objects and their states. Specifically, the recognition model includes a category recognition part and a state recognition part; the category recognition part identifies the type of object and classifies the identified objects into static and dynamic objects.
[0044] In this embodiment, the category recognition part includes: a backbone network, a global branch, local branches, a fusion network, and a classification network; wherein, ConvNeXt is used as the backbone network for feature extraction, the output of the third block is used as the input of the local branch, and the output of the fourth block is used as the input of the global branch. In the local branch, a feature enhancement attention module and a multi-channel convolutional local resampling module are employed. In the multi-channel convolutional local resampling module, the output of the third block first undergoes feature extraction through five convolutional modules, and then features of different receptive field sizes are extracted using 1×1 and 3×3 convolutional kernels respectively; concatenation is used for stitching, and the output is then passed through a 1×1 convolutional layer. The output of the multi-channel convolutional local resampling module serves as the input of the feature enhancement attention module. In the feature enhancement attention module, the output of the multi-channel convolutional local resampling module first undergoes feature extraction and normalization processing through a 1×1 convolutional layer and a BN layer, and then passes through another 1×1 convolutional layer and... Softplus The operation generates a weight matrix; the output of the BN layer is then normalized using the L2 norm through another branch. Finally, the outputs of both branches are multiplied by the original input, and the result is normalized again through a 1×1 convolutional layer and a BN layer before being output. In the global branch, a feature-enhanced attention module and an adaptive average pooling layer are used to process the output of the fourth block.
[0045] In the fusion network, the output features of the global branch and local branches are added together, the output features of the global branch are mapped, and then feature fusion is performed before the fused features are input into the classification network. The feature addition operation includes: ; In the formula, f add Indicates the additive characteristic; f J , f Q These represent the output characteristics of the local branch and the global branch, respectively.
[0046] Mapping operations include: ; In the formula, f Y This indicates the mapping characteristics.
[0047] The fusion operation includes: ; In the formula, f R This indicates the fusion feature.
[0048] In this embodiment, the classification network is constrained by a loss function to perform classification. The loss function used is CosFace loss.
[0049] For the state recognition part, a binary classification network model is used to classify objects, resulting in classification results for static and dynamic objects.
[0050] In this embodiment, the local data analysis unit also includes an object recognition subunit, used to identify and classify objects in the state information collected along the robot's path. Simultaneously, the local data analysis unit also includes an early warning subunit, used to determine the distance to dynamic objects. If the distance exceeds a safe distance threshold, the system either re-plans the path or stops and waits until the distance exceeds the safe distance threshold before proceeding with the terminated path.
[0051] Both the first and second instruction generation units generate corresponding instructions based on an instruction generation model. Specifically, the instruction generation model includes a local instruction generation part and a global instruction generation part; the local instruction generation part is used to generate robot motion instructions; and the global instruction generation part is used to generate robot movement instructions. The instruction generation function is achieved by calling the corresponding parts.
[0052] In this embodiment, the global instruction generation section includes: a spatial model building block, a trajectory prediction block, a global trajectory planning block, and a first actuator. The spatial model building block is used to build models of objects in the global analysis results, obtaining a virtual original map with coordinate information. The trajectory prediction block is used to predict the trajectory of dynamic objects, obtaining prediction results. The global trajectory planning block plans the robot's path based on the virtual original map and the prediction results, obtaining a planned path. The first actuator converts the planned path into action instructions (such as speed, acceleration, turning angle, etc.), generating robot movement instructions.
[0053] Specifically, the trajectory prediction block adopts an encoder-decoder structure, including an encoding module, a fusion module, and a decoding module. Taking the historical trajectory information, runtime temporal information (such as position coordinates, velocity, acceleration, etc.), and auxiliary information (such as object type) of the tracked object as input, the encoding module uses a trajectory encoder, a state encoder, and an auxiliary encoder to extract features. For the input historical trajectory information, the trajectory encoder first convolves it with convolutional kernels of different sizes to capture trajectory change details, and then uses a gated Transformer to encode the extracted features, obtaining the encoded trajectory features. X aux The state encoder first learns spatial dependency weights using the STGAT graph attention layer for the input runtime state temporal information, then performs a one-dimensional convolution on it, and finally uses residual connections to obtain the runtime trajectory features. X gated The auxiliary encoder uses a three-layer known perceptron to encode auxiliary information, thus obtaining auxiliary features. X final The fusion module uses two cross-attention operations to fuse the three features mentioned above. Then, linear transformation and normalization operations are applied to enhance the output of the encoding module. Finally, a gating mechanism is used to obtain the fused features. Y .in, X att1 This represents the result of the first cross-attention attempt; X att2 This represents the result of the second cross-attention; E Indicates enhanced features; S This represents the gating mechanism signal. The decoding module first inputs the fused features into a differential network to generate different differential features, and then decodes the future trajectory using a GRU decoder. Y fianl This yields the predicted trajectory.
[0054] in, ; ; In the formula, This represents the Sigmoid function. Gate This indicates a fully connected network.
[0055] ; ; In the formula, GUR i Indicates the first i A modal GUR decoder; C i Indicates the first i A contextual feature network; Indicates the first i The degree of radicality of each modality; m net This represents a differentiated network.
[0056] The global trajectory planning block uses the predicted trajectories of static and dynamic objects as obstacles for path planning. The process includes: Step 1: Create a raster map based on the virtual original map and dynamic and static objects, and initialize the parameters (number of iterations, learning rate discount rate, and greedy coefficient, etc.).
[0057] Step 2: Use the Q-Learning algorithm to explore the environment and convert the Q-table into the original pheromone matrix: ; In the formula, Z 0 represents the initial value of pheromones; Indicates the robot's state zi Take action a Arrival Status zj corresponding Q value; Indicates the robot's state zi Take action a Minimum obtained Q value; Indicates the robot's state zi Take action a The maximum obtained Q value.
[0058] Step 3: Ants use pseudo-random state transition rules to find the next node and update the tabu list by adding the current node to the tabu list to prevent ants from visiting the same node repeatedly.
[0059] Step 4: Determine if the ant has entered the dead zone. If the ant has entered the dead zone, update its path locally with pheromones; otherwise, proceed to Step 5.
[0060] Methods for updating local information include: ; In the formula, l Indicates the attenuation factor; k Indicates the index of a node in the path; n The index representing the node where the deadlock occurred; Indicates that the index is k and k +1 corresponds to the pheromone concentration between nodes; q This represents the pheromone constant.
[0061] Step 5: Determine if the ant has reached the destination. If it has, proceed to Step 6; otherwise, return to Step 3.
[0062] Step 6: Global pheromone update.
[0063] Methods for updating pheromones for all ants that have reached the destination include: ; in, ; ; In the formula, r Indicates the pheromone volatile factor; L k Ants k The path length in this iteration; K Indicates the total number of indexes; m This represents the normalized reward and punishment factor; L worst This represents the longest path in the current iteration; L best This represents the shortest path in the current iteration.
[0064] Step 7: Determine if the maximum number of iterations has been reached. If it has, output the optimal path and iteration curve; otherwise, repeat steps 3 to 6.
[0065] The local instruction generation section includes a classification block, a contour extraction block, a local path planning block, and a second actuator. The classification block, based on image recognition technology, classifies the objects to be sorted, and obtains the endpoint of the local path planning based on the classification results. The contour extraction block identifies the object contours using an object contour recognition algorithm. The second actuator converts the planned path into motion instructions (such as velocity, acceleration, rotation angle, torque, etc.), generating robot motion instructions.
[0066] Specifically, the process of object contour recognition algorithm includes: (1) Binarize the acquired object image using RGB standard deviation to obtain a binarized image.
[0067] (2) Obtain the region of interest (ROI) from the binarized image.
[0068] (3) Gaussian filtering and edge detection are performed on the ROI region. Then, Hough circle transformation and Hough line transformation are used to detect all straight lines in the ROI region, thereby obtaining the object shape. The robot end-effector sorting mechanism effectively grasps the object by judging its shape.
[0069] The local path planning block employs an improved particle swarm optimization algorithm to implement path planning for the robot's end-effector sorting mechanism. The process includes: Step 1: Initialize parameters (including joint velocity, joint acceleration, torque, rotation angle, etc.), set parameters such as particle swarm size, maximum number of iterations, initial inertia weight, end weight, external file threshold, etc., and randomly generate the initial position and velocity of particles.
[0070] Step 2: Evaluate particle fitness, calculate the multi-objective function value for each particle, and determine the non-dominated level of each particle.
[0071] The objective function includes: ; In the formula, G 1. G 2 represents operational impact and operational energy consumption, respectively; N Indicates the total number of joints; n Indicates joint n ; T Indicates the time for a single sorting operation; j n Indicates joint n At any moment t The acceleration; q n Indicates joint n At any moment t angular velocity of rotation; t n Indicates joint n At any moment t The driving torque.
[0072] Step 3: Update the external file, add non-dominated solutions to the external file, calculate the crowding of each solution in the file, delete dominated redundant solutions, and if the file size exceeds the threshold... N Delete the solution with the lowest congestion.
[0073] Crowding includes: ; In the formula, y rowd,il Indicates the first in the filei The solution is at the th solution. l Crowding on each target; f l ( i ) indicates the first i The solution of the first... l One target value; f l,max , f l,min Indicates the first in the file l The maximum and minimum values of each target.
[0074] Step 4: Determine if the termination condition is met. If the current iteration count reaches... T max Output the Pareto optimal solution set from the external archive, select the optimal solution using the TOPSIS method, and the process ends; otherwise, continue to the next step.
[0075] Step 5: Select the global optimal particle, calculate the selection probability of each solution in the external archive, and select the global optimal particle based on the probability.
[0076] The selection probabilities include: ; In the formula, Represents any solution in the external file; A rchive Represents the set of solutions; rank ( s ) represents the solution s Non-dominated hierarchy; y rowd ( s () indicates the total congestion level.
[0077] Step 6: Update the inertia weights and calculate the current iteration number. t The corresponding inertia weight.
[0078] The inertia weights include: ; In the formula, g Indicates inertia weight; g end , g start These represent the end weight and the start weight, respectively.
[0079] Step 7: Update particle velocity and position. Based on inertia weight, individual optimality, and global optimality, update the particle velocity and position.
[0080] Step 8: Increment the iteration count by 1 to bring the current iteration count closer to the maximum iteration count, return to step 2, and repeat the above process.
[0081] The execution management module includes a storage unit and a controller unit. The storage unit is used to store status information, running information, and running instructions. The controller unit is connected to the first actuator and the second actuator and is used to drive the robot to run based on the running instructions.
[0082] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A robot sorting trajectory self-optimization system based on reinforcement learning, characterized in that, include: The sensing module is used to acquire status information of the sorting work scenario and the robot's operating information; The data processing module is used to construct an instruction generation model and obtain operating instructions based on the instruction generation model, the state information, and the operating information; the operating instructions include: robot movement instructions and robot action instructions; The execution management module is used to store the status information, the running information, and the running instructions, and to drive the robot to run based on the running instructions to complete the sorting action.
2. The robot sorting trajectory self-optimization system based on reinforcement learning according to claim 1, characterized in that, The sensing module includes: a first data acquisition unit, a runtime data acquisition unit, and a second data acquisition unit; The first data acquisition unit is integrated on the robot body and is used to acquire the status information in the robot's operating scenario in real time; The operation data acquisition unit is used to acquire the operation information; The second data acquisition unit is used to acquire the status information of the complete sorting work scenario.
3. The robot sorting trajectory self-optimization system based on reinforcement learning according to claim 2, characterized in that, The data processing module includes: a global data analysis unit, a local data analysis unit, a first instruction generation unit, and a second instruction generation unit; The global data analysis unit is connected to the second data acquisition unit and is used to analyze the status information of the complete sorting work scenario to obtain global analysis results. The local data analysis unit is connected to the first data acquisition unit and the operation data acquisition unit, and is used to analyze the state information and operation information in the robot operation scenario to obtain local analysis results; The first instruction generation unit is connected to the global data analysis unit and the local data analysis unit, and is used to generate the robot movement instruction based on the global analysis result and the local analysis result; The second instruction generation unit is connected to the local data analysis unit and is used to generate the robot action instructions based on the local analysis results.
4. The robot sorting trajectory self-optimization system based on reinforcement learning according to claim 3, characterized in that, The global data analysis unit includes an information splicing subunit and an object recognition subunit. The information splicing subunit is used to splice the state information of the complete sorting work scene obtained by several second data acquisition units to obtain a complete sorting work scene image. The object recognition subunit is used to perform object recognition and state recognition on the complete sorting work scene image to obtain global analysis results.
5. A robot sorting trajectory self-optimization system based on reinforcement learning according to claim 4, characterized in that, The local data analysis unit is equipped with an object recognition subunit, which is used to identify and classify objects in the state information collected in the running scene along the robot's running path; it is also equipped with an early warning subunit, which is used to judge the distance of dynamic objects, and when the distance exceeds the safe distance threshold, it will re-plan the path or stop and wait until the distance is greater than the safe distance threshold, and then execute the aborted path.
6. The robot sorting trajectory self-optimization system based on reinforcement learning according to claim 1, characterized in that, The execution management module includes: a storage unit and a controller unit; The storage unit is used to store the status information, the running information, and the running instructions; The controller unit is used to drive the robot to run based on the running instructions.
7. The robot sorting trajectory self-optimization system based on reinforcement learning according to claim 3, characterized in that, Both the first instruction generation unit and the second instruction generation unit generate corresponding instructions based on an instruction generation model. The instruction generation model includes a local instruction generation part and a global instruction generation part. The local instruction generation part is used to generate robot motion instructions, and the global instruction generation part is used to generate robot movement instructions. The local instruction generation section includes: a classification block, a contour extraction block, a local path planning block, and a second actuator; the classification block is used to classify the sorted objects based on image recognition technology, and determine the endpoint of the local path planning based on the classification results; the contour extraction block identifies the object contour using an object contour recognition algorithm; the robot's end-effector sorting mechanism grasps the object based on the object contour recognition results; the second actuator converts the local planned path into action instructions, generating robot action instructions; The global instruction generation section includes: a spatial model building block, a trajectory prediction block, a global trajectory planning block, and a first actuator; the spatial model building block is used to build models of objects in the global analysis results to obtain a virtual original map with coordinate information; the trajectory prediction block is used to predict the trajectory of dynamic objects to obtain prediction results; the global trajectory planning block plans the robot's travel path based on the virtual original map and the prediction results to obtain a planned path; the first actuator converts the global planned path into action instructions to generate robot movement instructions.