Path planning method, device and equipment for double-arm robot and medium

Abstract

The embodiment of the invention discloses a path planning method and device for a double-arm robot, equipment and a medium. The method comprises the following steps: acquiring environment data and a path planning task of the double-arm robot; determining a three-dimensional environment model according to the environment data; determining a first path according to the three-dimensional environment model and the path planning task; and adjusting the first path according to the path cost of the path planning task, and determining a target path. According to the technical scheme, the final target path is determined according to the real-time environment data and the path cost, and the accuracy and reliability of path planning are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of path planning, and in particular to a path planning method, device, equipment and medium for a dual-arm robot. Background Art

[0002] With the rapid development of robot-related technologies, people's demand for robots is getting higher and higher. Autonomous navigation is an important function of mobile robots. In real environments, obstacles often appear dynamically, such as pedestrians and vehicles. To deal with dynamic obstacles, it is necessary to update the robot's path planning in time to avoid collisions and conflicts.

[0003] The common approach currently is to use perception technologies, such as vision or LiDAR, to detect and track obstacles in real time to determine the robot's path. However, sometimes a robot needs to perform multiple tasks at the same time and needs to plan a path that can meet multiple target tasks. Therefore, the problem of how to determine a path that meets the target tasks based on dynamic environmental changes and other factors is prone to occur. Summary of the invention

[0004] The present invention provides a path planning method, device, equipment and medium for a dual-arm robot, which determine the final target path according to real-time environmental data and path cost, thereby improving the accuracy and reliability of path planning.

[0005] According to one aspect of the present invention, a path planning method for a dual-arm robot is provided, comprising:

[0006] Acquire environmental data and path planning tasks for dual-arm robots;

[0007] determining a three-dimensional environment model according to the environment data;

[0008] Determine a first path according to the three-dimensional environment model and the path planning task;

[0009] The first path is adjusted according to the path cost of the path planning task to determine a target path.

[0010] Optionally, determining a three-dimensional environment model according to the environment data includes:

[0011] Filtering the current environmental data based on a preset filtering method to obtain filtered environmental data; wherein the preset filtering method includes Kalman filtering or particle filtering;

[0012] Construct a three-dimensional environmental model based on the filtered environmental data.

[0013] Optionally, determining the first path according to the three-dimensional environment model and the path planning task includes:

[0014] Extracting image data from the three-dimensional environment model;

[0015] Inputting the image data into a set neural network to obtain predicted obstacle positions and risk point positions;

[0016] A first path is determined according to the obstacle position, the risk point position and the path planning task.

[0017] Optionally, determining the first path according to the obstacle position, the risk point position, and the path planning task includes:

[0018] Determine a fuzzy rule set according to the path planning task and the risk point location;

[0019] A first path is generated through fuzzy logic control according to the obstacle position and the fuzzy rule.

[0020] Optionally, after generating the first path, the method further includes:

[0021] The positions of obstacles in the first path are optimized by setting optimization parameters to obtain an optimized path.

[0022] Optionally, adjusting the first path according to the path cost of the path planning task to determine the target path includes:

[0023] Dividing the path planning task of the dual-arm robot into a plurality of sub-path planning tasks;

[0024] Determine the target cost corresponding to each sub-path planning task;

[0025] Determine the path planning of each sub-path planning task in parallel according to the target cost;

[0026] The path planning of each sub-path planning task is integrated to obtain an integrated path planning;

[0027] The first path is adjusted according to the fused path planning to obtain a target path.

[0028] Optionally, the path planning task includes path planning from a starting node to a target node;

[0029] Determining the target cost corresponding to each sub-path planning task includes:

[0030] Determine the position and number of nodes from the starting node to the target node in the sub-path planning task respectively;

[0031] Determine the distance cost from the starting node to the target node according to the position of the starting node to the target node;

[0032] Determine the cost from the starting node to the target node according to the number of nodes from the starting node to the target node and the set cost parameter;

[0033] The target cost corresponding to each sub-path planning task is determined according to the distance cost and the cost cost.

[0034] According to another aspect of the present invention, there is provided a path planning device for a dual-arm robot, comprising:

[0035] The acquisition module is used to obtain environmental data and the path planning task of the dual-arm robot;

[0036] An environment model determination module, used to determine a three-dimensional environment model according to the environment data;

[0037] A determination module, configured to determine a first path according to the three-dimensional environment model and the path planning task;

[0038] An adjustment module is used to adjust the first path according to the path cost of the path planning task to determine a target path.

[0039] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0040] at least one processor; and

[0041] a memory communicatively connected to the at least one processor; wherein,

[0042] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the path planning method for the dual-arm robot described in any embodiment of the present invention.

[0043] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the path planning method for a dual-arm robot described in any embodiment of the present invention when executed.

[0044] The technical solution of the embodiment of the present invention obtains environmental data and a path planning task of a dual-arm robot; determines a three-dimensional environmental model according to the environmental data; determines a first path according to the three-dimensional environmental model and the path planning task; and adjusts the first path according to the path cost of the path planning task to determine a target path. This technical solution determines the final target path according to real-time environmental data and path cost, thereby improving the accuracy and reliability of path planning.

[0045] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 is a flow chart of a path planning method for a dual-arm robot provided according to Embodiment 1 of the present invention;

[0048] Figure 2 is a flow chart of a path planning method for a dual-arm robot provided according to Embodiment 2 of the present invention;

[0049] Figure 3 is a schematic structural diagram of a path planning device for a dual-arm robot provided according to a third embodiment of the present invention;

[0050] Figure 4 It is a structural schematic diagram of an electronic device provided according to Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first", "second" and "target" in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] Embodiment 1

[0054] Figure 1 This is a flow chart of a dual-arm robot path planning method provided according to the first embodiment of the present invention. This embodiment is applicable to the case of dynamically planning the path of a dual-arm robot. The method can be executed by a dual-arm robot path planning device. The dual-arm robot path planning device can be implemented in the form of hardware and / or software. The dual-arm robot path planning device can be configured in an electronic device with data processing capabilities. Figure 1 As shown, the method includes:

[0055] S110, obtaining environmental data and a path planning task for the dual-arm robot.

[0056] Among them, environmental data can be understood as the current environment in which the dual-arm robot is working in real time. Environmental data may include data such as obstacle locations and terrain features. The environmental data in this embodiment can use laser radar, ultrasonic sensors and cameras to perceive the obstacle locations and terrain features in the working environment in real time to obtain current real-time environmental data; wherein the camera is used to provide visual information. The path planning task can be the path planning task that the dual-arm robot currently needs to reach each node.

[0057] In this embodiment, laser radar, ultrasonic sensors and cameras can be used to perceive in real time the location of obstacles and terrain features in the working environment and the path planning tasks that the dual-arm robot currently needs to perform.

[0058] S120: Determine a three-dimensional environment model according to the environment data.

[0059] Among them, the three-dimensional environment model can be understood as an environment representation with three-dimensional spatial characteristics created by computer technology, which can simulate and display the shape, size, position and spatial relationship between objects in the real world. In this embodiment, laser radar, ultrasonic sensors and cameras can be used to perceive the obstacle position and terrain characteristics in the working environment in real time, and the environmental data such as Kalman filter and particle filter can be used for data fusion to construct a three-dimensional environment model. In this embodiment, the three-dimensional environment model and multi-sensor data fusion technology can be used to more accurately perceive and process dynamic changes and complex environments, and improve the accuracy and reliability of path planning.

[0060] In this embodiment, optionally, determining a three-dimensional environmental model based on environmental data includes: filtering the environmental data at the current moment based on a preset filtering method to obtain filtered environmental data; and constructing a three-dimensional environmental model based on the filtered environmental data.

[0061] Among them, the preset filtering methods include Kalman filtering or particle filtering. The Kalman filtering method can estimate the current state through the estimated value of the previous state and the observed value of the current state. Particle filtering can approximate the probability density function of the system through a group of random particle samples; in this embodiment, the random particle samples can be propagated in the state space and their weights can be updated according to the observed data, thereby realizing the estimation of the system state. Kalman filtering is used to process data fusion of linear systems, and particle filtering is used to process data fusion of nonlinear systems. The filtered environmental data can be the data obtained after data fusion of the environmental data by filtering.

[0062] In this embodiment, the environmental data at the current moment can be filtered according to Kalman filtering or particle filtering to achieve the purpose of data fusion; then the environmental state at the current moment is estimated based on the filtered environmental data, different environmental state estimates at the current moment can be determined according to different filtering methods, and the three-dimensional environmental model can be determined based on the environmental state estimate at the current moment.

[0063] Specifically, the Kalman filter part includes the calculation method of the Kalman gain K(t) as follows:

[0064] K(t)=P(t|t-1)H T (HP(t|t-1)H T +R) -1 ;

[0065] The prediction covariance P(t|t-1) is calculated as follows:

[0066] P(t|t-1)=AP(t-1)A T +Q;

[0067] The updated covariance matrix P(t) in this embodiment is calculated as follows:

[0068] P(t)=(IK(t)H)P(t|t-1);

[0069] The particle filter part includes the weight update method:

[0070] Resampling step for the current state estimate:

[0071] The environment model is constructed as follows:

[0072] Right now:

[0073] Among them, X(t) is the state estimate at the current moment, I is the identity matrix, K(t) is the Kalman gain, H is the observation matrix, Z(t) is the environmental data at the current moment, P(t|t-1) is the prediction covariance, A is the state transfer matrix, Q is the process noise covariance, and R is the measurement noise covariance. is the weight of the ith particle, is the observation probability, is the state transition probability, is the proposed distribution, and M(t) is the three-dimensional environment model at the current moment.

[0074] In this embodiment, the environmental data is filtered by Kalman filtering or particle filtering to improve the accuracy of the three-dimensional environmental model. In this embodiment, adaptive filtering and incremental learning technology can be combined with a real-time feedback mechanism to dynamically adjust the real-time environmental changes and task progress, ensuring the flexibility and adaptability of path planning.

[0075] S130: Determine a first path according to the three-dimensional environment model and the path planning task.

[0076] Among them, the first path can be understood as the original planned path preliminarily determined according to the three-dimensional environment model and the path planning task. In this embodiment, the first path may include multiple planned paths. In this embodiment, the first path can be obtained according to the image data obtained in the three-dimensional environment model and the various nodes in the path planning task.

[0077] S140: Adjust the first path according to the path cost of the path planning task to determine the target path.

[0078] Among them, the path cost can be understood as the cost of each planned path in the path planning task. The path cost can include the distance cost price and the actual cost price. The target path can be the planned path obtained by adjusting each planned path by the path cost. The target path in this embodiment can dynamically adjust the current path planning according to the changes in the actual environment so that the target path meets the actual needs set by the technician. In this embodiment, the planned path can be adjusted according to the distance cost price and the actual cost price of the path planning task to determine the target path that meets the path cost requirements.

[0079] In this embodiment, optionally, the first path is adjusted according to the path cost of the path planning task to determine the target path, including: dividing the path planning task of the dual-arm robot into multiple sub-path planning tasks; determining the target cost corresponding to each sub-path planning task; determining the path planning of each sub-path planning task in parallel according to the target cost; fusing the path planning of each sub-path planning task to obtain a fused path planning; adjusting the first path according to the fused path planning to obtain the target path.

[0080] Among them, multiple sub-path planning tasks can be obtained by dividing the path planning task according to a set method. In this embodiment, the path planning task of the dual-arm robot is divided into multiple sub-path planning tasks according to the planning space in the path planning task. In this embodiment, each subtask can independently process part of the path planning in the path planning task. The target cost may include a distance cost and a cost cost. Fusion can be understood as an operation of superimposing and synthesizing the path planning of the subtasks. In this embodiment, the fused path planning can be understood as superimposing and synthesizing the path planning of each sub-path planning task into an overall path planning. The target path can be adjusted according to the fused path planning to obtain an overall planning path that is more in line with actual needs. In this embodiment, the original planned path can be adjusted according to the superimposed synthesis of the overall path planning to obtain the target path.

[0081] In this embodiment, the target cost corresponding to each sub-path planning task is determined, and then the path planning of each sub-path planning task is determined in parallel according to the target cost. The target cost corresponding to each sub-path planning task is determined by assigning each sub-task to different computing nodes for parallel processing, and the path planning of the sub-path planning task at the current moment is determined according to the distance cost and cost cost in the target cost. In this embodiment, by decomposing the path planning task into multiple sub-tasks and assigning them to different computing nodes for parallel processing, the computing efficiency is significantly improved, and the real-time requirements in complex environments can be met.

[0082] In this embodiment, the first path is adjusted according to the fused path planning. A specific method for obtaining the target path is to make a detailed comparison between the fused path and the path in the first path, determine the main differences between the two, and make local adjustments to the differences in the first path that are significantly different from the fused path to obtain the final target planned path.

[0083] In this embodiment, optionally, the target cost corresponding to each sub-path planning task is determined, including: respectively determining the position and number of nodes from the starting node to the target node in the sub-path planning task; determining the distance cost from the starting node to the target node according to the position from the starting node to the target node; determining the cost from the starting node to the target node according to the number of nodes from the starting node to the target node and the set cost parameters; determining the target cost corresponding to each sub-path planning task according to the distance cost and the cost cost.

[0084] Among them, the path planning task includes path planning from the starting node to the target node. The starting node can be the starting node of the path planning. The target node can be understood as the final node of the path planning. The position can be understood as the coordinate position of each node. The number of nodes can be understood as the number of nodes from the starting node to the target node. The distance cost can be understood as the distance cost from the starting node to the target node. Setting the cost parameter can be understood as a set path cost parameter. In this embodiment, the cost parameter is set to the path cost between two adjacent nodes; illustratively, the cost parameter can be set to the path cost parameter from node k to node k+1. In this embodiment, the cost parameter can be set according to actual needs. The cost can be understood as the actual path cost from the starting node to the target node.

[0085] Specifically, in this embodiment, the specific method of determining the distance cost from the starting node to the target node according to the position of the starting node to the target node is to define a heuristic function h(n) for estimating the distance cost from the starting node n to the target node:

[0086]

[0087] Among them, x n ,y n , z n is the position coordinate of the starting node n, x goal ,y goal , z goal is the position coordinate of the target node.

[0088] In this embodiment, the specific method of determining the cost from the starting node to the target node according to the number of nodes from the starting node to the target node and the set cost parameter is to define a cost function g(n) for estimating the actual cost from the starting node to the target node n:

[0089] Among them, c k is the path cost parameter from node k to node k+1, and m is the number of nodes from the starting node to the target node.

[0090] In this embodiment, the specific method of determining the target cost corresponding to each sub-path planning task according to the distance cost and the cost cost can be to add the distance cost and the cost cost to obtain the target cost corresponding to each sub-path planning task. In this embodiment, an evaluation function f(n) can be defined to comprehensively evaluate the target cost from the starting node to the target node: f(n) = g(n) + h(n).

[0091] Through such a setting in this embodiment, the target cost of each sub-path planning task can be comprehensively evaluated according to the distance cost and cost cost of each sub-path planning task, so as to determine a path that meets the cost requirements.

[0092] Furthermore, in this embodiment, a heuristic search algorithm can be used as a path planning algorithm, and the processing function The specific implementation is to decompose the path planning task into N sub-path planning tasks, each of which independently processes part of the path planning. The path planning of the i-th sub-task at time t is as follows:

[0093]

[0094] In this embodiment, each sub-planning task can be assigned to different computing nodes for parallel processing, and the target cost of each sub-planning task is obtained as follows:

[0095]

[0096] In this embodiment, each computing node can synchronize the calculation results after processing the subtasks, and integrate the path planning of each subtask to obtain the overall path after synthesis as follows:

[0097]

[0098] Among them, P total (t) is the overall path planning at time t, is the path planning of the ith subtask at time t, N is the total number of subtasks, M(t) is the 3D environment model at time t, C(t) is the constraint condition at time t, is the function for processing the i-th subtask, is the calculation process of the i-th computing node, h(n) is the heuristic function, estimating the cost from node n to the target node, g(n) is the cost function, estimating the actual cost from the starting node to node n, f(n) is the evaluation function, comprehensively evaluating the cost from the starting node to the target node, c k is the path cost from node k to node k+1, m is the number of nodes on the path, x n ,y n , z n is the coordinate of node n, x goal ,y goal , z goal is the coordinate of the target node.

[0099] In this embodiment, through such a setting, the path planning task can be divided into various sub-path planning tasks, which are sent to the computing nodes for parallel processing to determine the target cost of each sub-path planning task, thereby determining each sub-path planning, improving the computing efficiency, and adjusting the path planning in the first path according to the fused path planning to obtain a more optimized target path that meets actual needs. In addition, this embodiment adopts a distributed computing architecture to decompose the path planning task for parallel processing, making full use of the multi-core processing capabilities of modern computers, and improving the scalability and processing capabilities of the algorithm.

[0100] The technical solution of the embodiment of the present invention obtains environmental data and a path planning task of a dual-arm robot; determines a three-dimensional environmental model according to the environmental data; determines a first path according to the three-dimensional environmental model and the path planning task; and adjusts the first path according to the path cost of the path planning task to determine a target path. This technical solution determines the final target path according to real-time environmental data and path cost, thereby improving the accuracy and reliability of path planning.

[0101] Embodiment 2

[0102] Figure 2 This is a flow chart of a dual-arm robot path planning method provided according to the second embodiment of the present invention. This embodiment is optimized based on the above embodiment. The specific optimization is: determining the first path according to the three-dimensional environment model and the path planning task, including: extracting image data in the three-dimensional environment model; inputting the image data into the set neural network to obtain the predicted obstacle position and risk point position; determining the first path according to the obstacle position, risk point position and path planning task. Figure 2 As shown, the method includes:

[0103] S210, obtaining environmental data and a path planning task for the dual-arm robot.

[0104] S220: Determine a three-dimensional environment model according to the environment data.

[0105] S230: extracting image data from the three-dimensional environment model.

[0106] Among them, the image data can be understood as the image features obtained by extracting the three-dimensional environment model through a convolutional neural network. The image data in this embodiment may include obstacle features and risk point features. In this embodiment, image data such as environmental data, obstacle features, and risk point features in the three-dimensional environment model at multiple different historical moments can be extracted. Specifically, in this embodiment, a convolutional neural network CNN can be used to process image data in a three-dimensional environment model, and the method of extracting features is:

[0107] f CNN (M(t)) = σ(WCNN *M(t)+b CNN );

[0108] Among them, W CNN is the convolution kernel weight, b CNN is the bias, * represents the convolution operation, σ is the activation function, f CNN (M(t)) is the feature after the convolutional neural network processes the three-dimensional environment model.

[0109] S240: Input the image data into the set neural network to obtain the predicted obstacle position and risk point position.

[0110] Among them, the setting neural network can be a pre-set neural network that can be used to predict environmental data. The setting neural network in this embodiment can be a recursive neural network. The predicted obstacle position and risk point position can be based on the current three-dimensional environment model to predict the obstacle position and risk point position at the next moment or set moment. In this embodiment, the image data extracted from the three-dimensional environment model at different historical moments is also input into the recursive neural network, and the predicted obstacle position and risk point position at the next moment are output.

[0111] Specifically, in this embodiment, a recursive neural network RNN ​​is used to process time series data and predict future obstacles and risk points:

[0112]

[0113] Among them, h t represents the image feature sequence after being processed by the convolutional neural network CNN, f RNN Represents the processing of RNN;

[0114] Furthermore, in this embodiment, the convolutional neural network CNN and the recurrent neural network RNN ​​can be trained by historical data such as the environmental data and the corresponding obstacle features and risk point annotations contained in the historical path planning data. In this embodiment, the back propagation algorithm and the gradient descent method can also be used to optimize the parameters of the convolutional neural network CNN and the recurrent neural network RNN, and the minimized prediction error can be:

[0115] Among them, Y(t) is the actual location of obstacles and risk points, is the model's predicted value.

[0116] S250: Determine a first path according to the obstacle position, the risk point position, and the path planning task.

[0117] In this embodiment, fuzzy logic control can be performed to generate an initial path planning strategy based on the obstacle location, risk point location, and task requirements in the path planning task.

[0118] In this embodiment, optionally, the first path is determined according to the obstacle position, the risk point position and the path planning task, including: determining a fuzzy rule set according to the path planning task and the risk point position; and generating the first path through fuzzy logic control according to the obstacle position and the fuzzy rules.

[0119] Among them, the fuzzy rule set can be constructed according to the task requirements and risk point locations in the path planning task.

[0120] In this embodiment, a fuzzy rule set can be constructed according to the task requirements and risk point locations in the path planning task, and then fuzzy logic control can be performed based on the constructed fuzzy rule set and obstacle locations to process the uncertainty information in the prediction results and generate a preliminary path planning strategy, that is, the first path can be:

[0121]

[0122] Among them, μ is the fuzzy logic function, R is the fuzzy rule set, To predict the obstacle location.

[0123] In this embodiment, optionally, after generating the first path, the method further includes: optimizing the positions of obstacles in the first path by setting optimization parameters to obtain an optimized path.

[0124] The setting of the optimization parameters may be pre-set optimization parameters, which may be used to optimize the data in the first path. The setting of the optimization parameters in this embodiment may be setting the optimization parameters by a reinforcement learning method, which may be set according to actual needs.

[0125] In this embodiment, the set optimization parameters in the reinforcement learning method can be used to optimize the obstacle positions in the first path to obtain an optimized path planning strategy:

[0126]

[0127] Among them, γ is the discount factor, r is the reward function, P(k) is the path planning strategy at time k, S(k) is the state at time k, P fuzzy (t) is the preliminary path planning strategy generated by simulation logic, P opt (t) is the optimized path planning strategy, λ is the regularization parameter, is the i-th predicted obstacle position, Y i (t) is the actual obstacle position of the ith object.

[0128] Through such a setting in this embodiment, the path planning strategy of the dual-arm robot can be determined by combining fuzzy logic control and reinforcement learning methods, and the path planning results can be comprehensively optimized to ensure better practical application effects of path planning.

[0129] S260: Adjust the first path according to the path cost of the path planning task to determine the target path.

[0130] The technical solution of the embodiment of the present invention obtains environmental data and the path planning task of the dual-arm robot; determines a three-dimensional environmental model based on the environmental data; extracts image data from the three-dimensional environmental model; inputs the image data into a set neural network to obtain predicted obstacle positions and risk point positions; determines a first path based on the obstacle positions, risk point positions, and the path planning task. The first path is adjusted according to the path cost of the path planning task to determine the target path. This technical solution determines the final target path based on real-time environmental data and path cost, thereby improving the accuracy and reliability of path planning.

[0131] Furthermore, in this embodiment, laser radar, ultrasonic sensors and cameras are used to perceive the position of obstacles and terrain features in the working environment in real time, and the collected environmental data is fused through Kalman filtering and particle filtering to construct a three-dimensional model of the environment; a deep learning model is used to predict potential obstacles and risk points in the path, and an adaptive path planning strategy is generated through fuzzy logic control and reinforcement learning technology based on environmental data and task requirements; during the path execution process, a feedback mechanism is used for real-time adjustment, and a distributed computing architecture is used to perform parallel processing and optimization of path tasks, and mobile obstacles in the path are dynamically monitored. Through path replanning and dynamic adjustment technology, the continuity and safety of the path are ensured.

[0132] In addition, the present embodiment provides a path perception system for dual-arm robot path planning, which can solve the problem in the prior art that the path required by the dual-arm robot is more complex than that of the single-arm robot, the planning difficulty is relatively large, and a large amount of simulation and debugging work is required, which increases the difficulty of implementation.

[0133] Embodiment 3

[0134] Figure 3 Schematic diagram of a dual-arm robot path planning device according to Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0135] An acquisition module 310 is used to acquire environmental data and a path planning task of a dual-arm robot;

[0136] An environment model determination module 320, for determining a three-dimensional environment model according to the environment data;

[0137] A first path determination module 330, configured to determine a first path according to the three-dimensional environment model and the path planning task;

[0138] The target path determination module 340 is used to adjust the first path according to the path cost of the path planning task to determine the target path.

[0139] Optionally, the environmental model determination module 320 is specifically used to filter the environmental data at the current moment based on a preset filtering method to obtain filtered environmental data; wherein the preset filtering method includes Kalman filtering or particle filtering; and construct a three-dimensional environmental model based on the filtered environmental data.

[0140] Optionally, the first path determination module 330 includes:

[0141] A data extraction unit, used for extracting image data from a three-dimensional environment model;

[0142] A position prediction unit, used to input image data into a set neural network to obtain predicted obstacle positions and risk point positions;

[0143] The path determination unit is used to determine the first path according to the obstacle position, the risk point position and the path planning task.

[0144] Optionally, the path determination unit is specifically used to determine a fuzzy rule set according to the path planning task and the risk point location; and generate a first path through fuzzy logic control according to the obstacle location and the fuzzy rules.

[0145] Optionally, the device further includes an optimization module, which is used to optimize the positions of obstacles in the first path by setting optimization parameters after generating the first path, so as to obtain an optimized path.

[0146] Optionally, the target path determination module 340 includes:

[0147] A task division unit, used for dividing the path planning task of the dual-arm robot into multiple sub-path planning tasks;

[0148] A cost determination unit, used to determine the target cost corresponding to each sub-path planning task;

[0149] A path planning unit, used to determine the path planning of each sub-path planning task in parallel according to the target cost;

[0150] A fusion unit, used to fuse the path planning of each sub-path planning task to obtain a fused path planning;

[0151] The adjustment unit is used to adjust the first path according to the fused path planning to obtain a target path.

[0152] Optionally, the path planning task includes planning a path from a start node to a target node;

[0153] The cost determination unit is specifically used to determine the position and number of nodes from the starting node to the target node in the sub-path planning task; determine the distance cost from the starting node to the target node according to the position of the starting node to the target node; determine the cost from the starting node to the target node according to the number of nodes from the starting node to the target node and the set cost parameters; determine the target cost corresponding to each sub-path planning task according to the distance cost and the cost cost.

[0154] A dual-arm robot path planning device provided in an embodiment of the present invention can execute a dual-arm robot path planning method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0155] Embodiment 4

[0156] Figure 4 1 is a schematic diagram of the structure of an electronic device provided according to Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0157] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0158] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0159] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a path planning method for a dual-arm robot.

[0160] In some embodiments, the path planning method of the dual-arm robot can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the path planning method of the dual-arm robot described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the path planning method of the dual-arm robot by any other appropriate means (e.g., by means of firmware).

[0161] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0162] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0163] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0165] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0166] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0167] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0168] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A path planning method for a dual-arm robot, It is characterized in that include: Acquire environmental data and path planning tasks for dual-arm robots; determining a three-dimensional environment model according to the environment data; Determine a first path according to the three-dimensional environment model and the path planning task; The first path is adjusted according to the path cost of the path planning task to determine a target path.

2. The method according to claim 1, It is characterized in that Determining a three-dimensional environment model according to the environment data includes: Filtering the current environmental data based on a preset filtering method to obtain filtered environmental data; wherein the preset filtering method includes Kalman filtering or particle filtering; Construct a three-dimensional environmental model based on the filtered environmental data.

3. The method according to claim 1, It is characterized in that Determining a first path according to the three-dimensional environment model and the path planning task includes: Extracting image data from the three-dimensional environment model; Inputting the image data into a set neural network to obtain predicted obstacle positions and risk point positions; A first path is determined according to the obstacle position, the risk point position and the path planning task.

4. The method according to claim 3, It is characterized in that Determining a first path according to the obstacle position, the risk point position, and the path planning task includes: Determine a fuzzy rule set according to the path planning task and the risk point location; A first path is generated through fuzzy logic control according to the obstacle position and the fuzzy rule.

5. The method according to claim 4, It is characterized in that After generating the first path, it also includes: The positions of obstacles in the first path are optimized by setting optimization parameters to obtain an optimized path.

6. The method according to claim 1, It is characterized in that The first path is adjusted according to the path cost of the path planning task to determine a target path, including: Dividing the path planning task of the dual-arm robot into a plurality of sub-path planning tasks; Determine the target cost corresponding to each sub-path planning task; Determine the path planning of each sub-path planning task in parallel according to the target cost; The path planning of each sub-path planning task is integrated to obtain an integrated path planning; The first path is adjusted according to the fused path planning to obtain a target path.

7. The method according to claim 6, It is characterized in that The path planning task includes path planning from a starting node to a target node; Determining the target cost corresponding to each sub-path planning task includes: Determine the position and number of nodes from the starting node to the target node in the sub-path planning task respectively; Determine the distance cost from the starting node to the target node according to the position of the starting node to the target node; Determine the cost from the starting node to the target node according to the number of nodes from the starting node to the target node and the set cost parameter; The target cost corresponding to each sub-path planning task is determined according to the distance cost and the cost cost.

8. A path planning device for a dual-arm robot, It is characterized in that include: The acquisition module is used to obtain environmental data and the path planning task of the dual-arm robot; An environment model determination module, used to determine a three-dimensional environment model according to the environment data; A first path determination module, used to determine a first path according to the three-dimensional environment model and the path planning task; The target path determination module is used to adjust the first path according to the path cost of the path planning task to determine the target path.

9. An electronic device, It is characterized in that The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the path planning method for the dual-arm robot according to any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the path planning method for a dual-arm robot according to any one of claims 1 to 7 when executed.

Images