A method, system, device, and medium for robotic path planning

Abstract

The application provides a robot path planning method, system, device and medium. The method is based on a target scene, constructs a two-dimensional working environment model of a robot, constructs a state vector based on observation information of the two-dimensional working environment model at the current moment through a policy network in a TD3 network model, and outputs a continuous action instruction at the current moment based on the kinematic constraint of the robot. A value network in the TD3 network model outputs a current estimated value based on the state vector at the next moment and the continuous action instruction at the current moment. The value network simultaneously outputs a value evaluation result of the continuous action instruction at the current moment. A single-step reward function outputs a single-step reward based on the state vector and the continuous action instruction at the current moment. The value network and the policy network are sequentially updated based on the single-step reward. While ensuring reaching the target position and obstacle avoidance safety, the method can effectively suppress the risk of detour accumulation and entanglement, and provides reliable path planning for the robot.

Description

Technical Field

[0001] This invention belongs to the field of robot autonomous navigation and path planning technology, specifically relating to a method, system, device and medium for path planning of a tethered robot. Background Technology

[0002] Tethered robots are connected to anchor points or a parent platform via cables, enabling continuous power and communication supply. They are suitable for tasks such as underwater operations, inspection of hazardous environments, and exploration of confined spaces. However, in complex obstacle environments, tethered cables are prone to contacting, detouring around, or even becoming entangled with obstacles, leading to a shrinking of feasible movement space, reduced movement efficiency, and in severe cases, mission failure or equipment damage. Therefore, path planning for tethered robots must not only meet the traditional mobile robot requirements of "reaching the target and avoiding obstacles safely," but also simultaneously consider cable length constraints, detouring trends, and the evolution of topological states.

[0003] Existing path planning methods mainly include grid search methods, sampling methods, and optimization methods. Grid search methods are convenient for incorporating cost maps, but when topological states such as cable loop sides and loop counts are included in the planning dimension, state space expansion and reduced real-time performance can easily occur. Sampling methods offer some flexibility in continuous space, but due to the limitations of randomness and local heuristics, they struggle to consistently obtain high-quality trajectories in scenarios with multiple solutions, such as "feasible in the short term but prone to entanglement in the long term" or "feasible but costly." Optimization methods require constructing differentiable or optimizable constraint expressions, but cable-obstacle contact switching, tangent-circular arc configuration updates, and loop count jumps are non-smooth and event-driven, easily leading to optimization instability or parameter sensitivity.

[0004] Therefore, there is an urgent need for a path planning method that can directly generate control commands in a continuous action space and explicitly perceive cable geometry and topological constraints during the learning process, thereby obtaining a safe path with low entanglement risk. Summary of the Invention

[0005] To address the problem that existing technologies cannot effectively plan paths for tethered robots, this invention provides a method, system, device, and medium for path planning of tethered robots.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] In a first aspect, embodiments of this disclosure provide a method for path planning of a tethered robot, comprising the following steps:

[0008] Step S1: Based on the target scenario, construct a two-dimensional working environment model of the tethered robot and set the kinematic constraints of the tethered robot;

[0009] Step S2: Construct a TD3 network model to provide path planning for the tethered robot. The TD3 network model includes a policy network. The policy network constructs a state vector based on the observation information of the two-dimensional working environment model at the current moment, and outputs continuous action commands at the current moment based on the kinematic constraints of the tethered robot. The state vector includes cable constraint representation information.

[0010] Step S3: Execute continuous action instructions in the two-dimensional work environment model and output the state vector at the next moment. The TD3 network model also includes a value network and a single-step reward function. The value network outputs the current estimated value based on the state vector at the next moment and the continuous action instructions at the current moment. The value network also outputs the value evaluation result of the continuous action instructions at the current moment. The single-step reward function outputs the single-step reward based on the state vector at the next moment and the continuous action instructions at the current moment.

[0011] Step S4: The value network constructs a target value based on the single-step reward to characterize the quality of the state vector at the next moment and the continuous action instructions at the current moment, and updates it by reducing the temporal difference error between the current estimated value of the value network and the target value. The policy network updates it along the gradient direction that increases the action value based on the value evaluation result. The single-step reward function is updated based on an adaptive mechanism.

[0012] Step S5: Repeat steps S2-S4 until the tethered robot reaches the target position and completes path planning.

[0013] Furthermore, the two-dimensional operating environment model of the tethered robot includes at least an anchor point, the tethered robot, the target location, and obstacles;

[0014] The state vector also includes the pose information of the tethered robot, the target relative pose information, and the obstacle relative geometric information;

[0015] The cable constraint characterization information includes the cable length estimate and the cable routing topology.

[0016] When obtaining the estimated cable length, an arc-straight-arc cable model combined with a contact stack topology maintenance mechanism is used to approximate the actual path under the cable tensioned state. The cable path includes the incoming arc segment and the straight segment between the tangent point on the obstacle contact side, as well as the outgoing arc segment leaving the obstacle contact side. The length of each path segment is determined by combining the anchor point position, the tethered robot position, obstacle geometry parameters, and the cable routing topology. Therefore, the estimated cable length is... ,include:

[0017] ;

[0018] In the formula, the estimated cable length is... Indicates the set of obstacles Under constraint, when the cable is fully tightened, from the anchor point To the robot end The total length; , This indicates the anchor point location at the fixed end of the cable. , Indicates the position of the tethered robot end; parameters This indicates the number of times the cable effectively wraps around an obstacle along the path. At that time, the cable was not in contact with any obstacle, and its length was [length missing]. ,when At that time, the cable path was composed of multiple straight segments and circular arc segments spliced ​​together sequentially, among which, Indicates the cable at the An ingress point on an obstacle, Indicates the cable at the The point of exit on an obstacle; For the first The effective wrapping radius of an obstacle is used to characterize the geometric contact constraint between the cable and the obstacle; The actual angle of wrap of the cable on the obstacle is determined by the location of the tangent point and the number of turns; symbol Represents the Euclidean norm;

[0019] The cable routing topology includes at least routing side information and the number of loops. The number of loops is a time-consistent counting result of the number of loops obtained based on the central angle of the arc segment and the angle of the whole loop.

[0020] Furthermore, the single-step reward include:

[0021] ;

[0022] In the formula, To achieve the target reward item; This is a penalty item for near-disability; Rewards for progressing through the mission; For cable restraint risk penalty items; This is the smoothing term for the direction of travel.

[0023] Furthermore, the termination of the reward item include:

[0024] ;

[0025] In the formula, This indicates that the tethered robot is at any time. Distance to the target location; To reach the threshold; Event indicators representing collisions or entry into inaccessible areas. ; This represents a one-time positive reward for the tethered robot's successful arrival. ; The penalty magnitude for collisions or entering infeasible areas. If triggered or Then, in this training round at time... If the preset number of training rounds has not been reached, the process will restart in the next training round. A training round refers to the process of updating the value network and the policy network.

[0026] Furthermore, the task progress reward items include:

[0027] ;

[0028] In the formula, For a moment Distance to the target location; Weighting coefficients for task progress reward items;

[0029] Based on local observation vectors minimum distance Construct the near obstacle penalty term The local observation vector is obtained by the robot through discrete sampling in multiple sector directions based on the observation information. Each component in the equation represents the distance to the nearest obstacle in the corresponding direction. The near obstacle penalty term... include:

[0030] ;

[0031] In the formula, For the obstacle avoidance safety item, the weighting coefficient is , and ; The near-obstacle penalty threshold; when the distance between the tethered robot and the obstacle is less than... At that time, punishment followed The decrease is linearly increased when No punishment is imposed at the time;

[0032] The task progress reward items include:

[0033]

[0034] in, For the current action, For the previous action, All of these are weighting coefficients for the reward item, and all are greater than 0.

[0035] Furthermore, the cable constraint risk penalty item is obtained. When, it includes the following steps:

[0036] Step A1: Construct the initial penalty term ,include:

[0037] ;

[0038] In the formula, and All are weighting coefficients, and all are greater than 0; This represents optimizing the remaining available cable margin. This represents an optimization of cable winding complexity;

[0039] Let the anchor point be b, and the end position of the tethered robot be... The straight-line distance from the anchor point to the tethered robot is... for: ;

[0040] Based on cable length estimation Define the remaining available cable margin. for: ;

[0041] remaining available cable margin Normalization and compression mapping are performed, and the result is used as the maximum cable length. To obtain the optimized remaining available cable margin :

[0042] ;

[0043] For the entanglement complexity, let the estimated cable length be... The maintained obstacle-avoidance sequence includes The number of obstacle avoidance events that are active, and the number of events that are active. The number of detours corresponding to each obstacle avoidance event is: , The number of loops indicates the strength, and the entanglement complexity is... for:

[0044] ;

[0045] Based on the maximum allowed complexity Regarding the complexity of entanglement Normalization is performed to obtain the optimized cable winding complexity. :

[0046] ;

[0047] Step A2: Construct a risk trend penalty model based on one-sided penalties for positive increments. :

[0048] ;

[0049] In the formula, and All are weighting coefficients, and all are greater than 0; This represents the change in the margin of remaining available cable. ; This represents the change in the complexity of optimizing cable winding. ;

[0050] Step A3: Construct hard constraints for cable length based on the cable length estimation calculation method ,include:

[0051] ;

[0052] In the formula, For severe punishment;

[0053] Step A4: Construct soft boundary penalty ,include:

[0054] ;

[0055] In the formula, It is a soft boundary. , This is the soft boundary coefficient, used to limit the soft boundary within a reasonable range. It is generally set to 0.1 and can be adjusted according to the maximum length of the cable. These are the weighting coefficients, and ;

[0056] Step A5: Based on the initial penalty term Risk trend penalty model Hard constraints on cable length and soft boundary penalty The cable constraint term is obtained. ,include:

[0057] .

[0058] Furthermore, when the single-step reward function is updated based on an adaptive mechanism, it includes the following steps:

[0059] Step B1: Based on the weighting coefficients Weighting coefficients Weighting coefficients Weighting coefficients and weighting coefficients Construct a fixed weight set ; and set a fixed weight Expanded to a function that changes with the state. ;

[0060] Step B2: Optimize the remaining available cable margin And optimize the complexity of cable winding. By performing weighted aggregation, the risk profile is obtained, resulting in an interpretable scalar risk score. :

[0061] ;

[0062] In the formula, For risk aggregation coefficient, ;

[0063] Step B3: Transform the scalar risk score using a saturation mapping. Transform into Enhancement coefficient within the range :

[0064] ;

[0065] ;

[0066] in, ; The coefficient representing the steepness of the change. ; This is the risk trigger threshold; It is an exponentially decaying function;

[0067] Step B4: Based on the enhancement coefficient Define the adaptive form of each weight coefficient in the fixed weight set:

[0068] ;

[0069] in, The basic weight for the low-risk phase; Increase weighting for high-risk phases;

[0070] Step B5: When scalar risk scoring The value is smaller at the current moment compared to the previous moment, and the enhancement coefficient is smaller. As the value approaches zero, the policy network will be more inclined to use tasks to advance reward items in the next time step. Improve efficiency; when scalar risk scoring When the current time step is larger than the previous time step, and the enhancement coefficient is... When the value increases, the policy network will, in the next time step, add the cable constraint risk penalty item. Magnify and actively choose the direction of action to untangle or avoid entanglement.

[0071] In a second aspect, embodiments of this disclosure provide a system for path planning of a tethered robot, comprising:

[0072] The initialization unit is configured to construct a two-dimensional working environment model of the tethered robot based on the target scene and set the kinematic constraints of the tethered robot.

[0073] The instruction output unit is configured to construct a TD3 network model for path planning of the tethered robot. The TD3 network model includes a policy network. The policy network constructs a state vector based on the observation information of the two-dimensional working environment model at the current moment and outputs continuous action instructions based on the kinematic constraints of the tethered robot. The state vector includes cable constraint representation information.

[0074] The execution update unit is configured to execute continuous action instructions in the two-dimensional work environment model and output the state vector at the next moment. The TD3 network model also includes a value network. The value network obtains a single-step reward based on the state vector at the next moment and the continuous action instructions at the current moment. The policy network and the value network are updated simultaneously based on the single-step reward.

[0075] The output unit is configured to repeatedly execute the instruction output unit and the execution update unit until the tethered robot reaches the target position and completes the path planning.

[0076] In a third aspect, embodiments of this disclosure provide an electronic device, characterized in that the electronic device comprises:

[0077] At least one processor; and,

[0078] The memory is communicatively connected to the at least one processor; wherein,

[0079] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the tethered robot path planning method.

[0080] In a fourth aspect, embodiments of this disclosure provide a non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the tethered robot path planning method described above.

[0081] Compared with the prior art, the present invention has the following beneficial technical effects:

[0082] This invention provides a method, system, device, and medium for path planning of a tethered robot. The method constructs a two-dimensional operating environment model of the tethered robot based on a target scenario and sets kinematic constraints for the robot. A policy network in a TD3 network model constructs a state vector based on observation information from the two-dimensional operating environment model at the current moment and outputs continuous action commands for the current moment based on the robot's kinematic constraints. A value network in the TD3 network model outputs a current estimated value based on the state vector at the next moment and the continuous action commands at the current moment. The value network also outputs a value assessment result for the continuous action commands at the current moment. A single-step reward function is based on the state vector at the next moment and the continuous action commands at the current moment. The system continues to execute action commands and outputs a single-step reward. The value network constructs a target value based on the single-step reward to characterize the quality of the next state vector and the current continuous action command. It updates the value network by reducing the temporal difference error between the current estimated value and the target value. The policy network updates along the gradient direction that increases the action value based on the value evaluation result. The single-step reward function is updated using an adaptive mechanism. The TD3 network model is based on reinforcement learning decision-making. During training, it can explicitly perceive changes in the feasible region and safety costs caused by cable constraints. While ensuring the safety of reaching the target position and obstacle avoidance, it can effectively suppress the risks of detour accumulation and entanglement, providing reliable path planning for tethered robots.

[0083] Furthermore, when obtaining the estimated cable length, a tangent-circular-tangent model is adopted, combined with a contact stack topology maintenance mechanism. This enables real-time and accurate characterization of the actual cable wrapping path and its topological relationships in scenarios involving continuous multi-obstacle navigation. Compared to traditional path description methods that rely solely on straight-line distances or simplified assumptions about single obstacles, this method more realistically reflects the contact, navigation, and switching processes between the cable and obstacles, effectively addressing the issues of path length calculation and navigation status updates under conditions of continuous multi-obstacle contact. By hierarchically recording and dynamically maintaining contact relationships, topological jumps caused by obstacle switching, changes in the navigation side, or sudden changes in local visibility can be significantly reduced. This reduces the risks of sudden discontinuities, jitter, and misjudgments during path planning, improving the stability, continuity, and reliability of the planning results. This offers significant advantages for enhancing the autonomous planning capabilities and safe operation performance of tethered robots in complex obstacle environments. Attached Figure Description

[0084] Figure 1 A flowchart 100 of a tethered robot path planning method according to an embodiment of the present disclosure is shown;

[0085] Figure 2 A schematic diagram of the tangent-circular arc-tangent cable geometry under the condition of a circular equivalent obstacle according to an embodiment of the present disclosure is shown.

[0086] Figure 3 This diagram illustrates the state transition of the touch stack maintenance cable routing topology according to an embodiment of the present disclosure.

[0087] Figure 4 shows a schematic diagram of the path of the tethered robot in a test round according to an embodiment of the present disclosure;

[0088] Figure 5 A diagram of an apparatus for a tethered robot path planning method according to an embodiment of the present disclosure is shown. Detailed Implementation

[0089] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0090] Figure 1 A flowchart 100 of a tethered robot path planning method according to an embodiment of this disclosure is shown, as follows: Figure 1 As shown, it includes the following steps:

[0091] In step S101, a two-dimensional working environment model of the tethered robot is constructed based on the target scenario, and kinematic constraints of the tethered robot are set.

[0092] Specifically, the two-dimensional operating environment model of the tethered robot includes at least an anchor point, the tethered robot, the target location, and a set of obstacles.

[0093] The tethered robot employs a differential propulsion motion model, setting control constraints such as velocity, angular velocity, acceleration, and steering angle. In one optional implementation, the two-dimensional planning plane can be determined by the anchor point, the current position of the tethered robot, and the target position. After projecting obstacles onto the two-dimensional plane, state construction and decision-making are performed to reduce the planning dimensionality and improve learning stability.

[0094] Next, proceed to step S102.

[0095] In step S102, a TD3 network model is constructed to provide path planning for the tethered robot. The TD3 network model includes a policy network. The policy network constructs a state vector based on the observation information of the two-dimensional working environment model at the current moment, and outputs continuous action commands at the current moment based on the kinematic constraints of the tethered robot. The state vector includes cable constraint representation information.

[0096] Specifically, for each decision time t of the policy network, observation information is obtained based on the two-dimensional working environment model and a state vector is constructed. The constructed state vector includes the pose information of the tethered robot, the relative pose information of the target, and the relative geometric information of the obstacle.

[0097] The pose information of the tethered robot includes: position, heading angle, and optional velocity information;

[0098] The target relative pose information includes: the target's distance and orientation relative to the tethered robot's coordinate system;

[0099] The relative geometric information of the obstacle includes: the distance and orientation to the nearest obstacle, or a distance sampling vector formed by point cloud;

[0100] The cable constraint characterization information includes the cable length estimate and the cable routing topology.

[0101] It should be noted that, in this embodiment, the method for calculating the cable length estimate is as follows: the "cable tension assumption" is adopted, that is, it is assumed that the cable is in a taut state without relaxation at any decision moment, and the cable shape is determined by the visible straight line segment between the anchor point and the tethered robot and the edge segment around the obstacle; when the obstacle can be equivalent to a circle, the cable winding configuration uses the geometric model of "tangent-circular arc-tangent" to calculate the cable length estimate.

[0102] Specifically, when approximating the actual path under cable tension using a circular arc-straight line-circular arc cable model combined with a contact stack topology maintenance mechanism, the cable path includes the incoming circular arc segment and the straight line segment between the tangent point on the obstacle contact side, as well as the outgoing circular arc segment leaving the obstacle contact side. The length of each path segment is determined by combining the anchor point position, the tethered robot position, obstacle geometric parameters, and the cable routing topology, forming a cable length estimation calculation method. The estimated cable length is then obtained based on this method. ,include:

[0103] ;

[0104] In the formula, the estimated cable length is... Indicates the set of obstacles Under constraint, when the cable is fully tightened, from the anchor point To the robot end The total length; , This indicates the anchor point location at the fixed end of the cable. , Indicates the position of the tethered robot end; parameters This indicates the number of times the cable effectively wraps around an obstacle along the path. At that time, the cable was not in contact with any obstacle, and its length was [length missing]. ,when At that time, the cable path was composed of multiple straight segments and circular arc segments spliced ​​together sequentially, among which, Indicates the cable at the An ingress point on an obstacle, Indicates the cable at the The point of exit on an obstacle; For the first The effective wrapping radius of an obstacle is used to characterize the geometric contact constraint between the cable and the obstacle; The actual angle of wrap of the cable on the obstacle is determined by the location of the tangent point and the number of turns; symbol Represents the Euclidean norm;

[0105] To be more specific, Figure 2 A schematic diagram of the tangent-circular arc-tangent cable geometry under the condition of a circular equivalent obstacle, as shown in the embodiment of this disclosure, is illustrated. Figure 2 As shown, under the assumption of cable tightening, the path of the cable from anchor point A to tethered robot R consists of several straight line segments and circular arc segments. This embodiment uses a circular equivalent obstacle as an example to illustrate the calculation process of the tangent-circular arc-tangent model.

[0106] The cable routing topology includes at least routing side information and the number of loops. The number of loops is a time-consistent count of loops obtained based on the central angle of the arc segment and the angle of the entire loop. Specifically, it is implemented by maintaining the multi-obstacle routing topology through a contact stack. This includes: the contact stack records the sequence of obstacles that the cable contacts or circumvents in sequence, as well as the corresponding routing side information; and pushes and pops are triggered based on visibility determination and changes in the loop count, so that the cable constraint representation information can reflect the topology of continuous multi-obstacle routing.

[0107] For visibility determination: if line segment AR intersects or crosses the equivalent circle of obstacle O, the cable is considered entangled, and visibility is compromised. If they do not intersect, visibility is not compromised. If they intersect, cross, or there is a maintained bypass state, the bypass configuration calculation begins: For tangent point solving and arc segment determination, let the obstacle circle center be C and the radius be r. Taking anchor point A and tethered robot R as external points, calculate the external tangents of the circle and the set of tangent points. Combine the bypass side to select the corresponding tangent points T_in and T_out, where the bypass side includes clockwise and counterclockwise directions, and calculate the arc segment central angle according to the unified angle direction and interval normalization rules. When the number of laps is n, it can be... It is expressed as the sum of the base radian and the full radian, representing the cumulative length of multiple revolutions.

[0108] To continuously update the cable geometry and obtain accurate cable length estimates, a contact stack topology maintenance mechanism is used to update the topology state of the contact stack-maintained bypass topology. For example... Figure 3 As shown, the contact stack elements include: obstacle identifier (id), bypass side (side), number of bypass loops (k), push time, and push pose. When the tether's direct visibility is compromised and the bypass risk index increases, or when a line segment intersects an obstacle for the first time and forms a stable bypass configuration, the corresponding obstacle and its bypass side are pushed onto the top of the stack. When the obstacle at the top of the stack no longer obstructs the visibility between the anchor point and the tethered robot, and the number of bypass loops falls back to near the threshold or the bypass configuration can be replaced by a shorter feasible path, popping is triggered; after popping, the tether configuration is recalculated based on the new obstacle at the top of the stack or the direct path. The topology state is defined as a summary of the (side, k) of the top element and the stack depth, and together with the estimated tether length, it serves as the state vector input to the policy network, enabling the learned policy to have memory and foresight regarding topology evolution.

[0109] Next, proceed to step S103.

[0110] In step S103, continuous action instructions are executed in the two-dimensional work environment model, and the state vector at the next moment is output. The TD3 network model also includes a value network and a single-step reward function. The value network outputs the current estimated value based on the state vector at the next moment and the continuous action instructions at the current moment. The value network also outputs the value evaluation result of the continuous action instructions at the current moment. The single-step reward function outputs the single-step reward based on the state vector at the next moment and the continuous action instructions at the current moment.

[0111] Specifically, the single-step reward include:

[0112] ;

[0113] In the formula, To achieve the target reward item; This is a penalty item for near-disability; Rewards for progressing through the mission; For cable restraint risk penalty items; This is the smoothing term for the direction of travel.

[0114] Among them, the termination reward item include:

[0115] ;

[0116] In the formula, This indicates that the tethered robot is at any time. Distance to the target location; To reach the threshold; Event indicators representing collisions or entry into inaccessible areas. ; This represents a one-time positive reward for the tethered robot's successful arrival. ; The penalty magnitude for collisions or entering infeasible areas. If triggered or Then, in this training round at time... If the preset number of training rounds has not been reached, the process will restart in the next training round. It should be noted that this embodiment considers the tethered robot reaching the target location and colliding with or entering an inaccessible area as two key termination events, and incorporates them into the termination reward item. In this implementation, through this highly discriminative termination reward structure, the algorithm can clearly distinguish between successful arrival and safe failure at the value network level, and generate significant gradient guidance for termination events. This accelerates the formation of correct behavioral boundaries in the early training phase of the policy network and improves convergence stability. It should be further noted that the training round described in this embodiment refers to the process of updating the value network, policy network, and single-step reward function once.

[0117] Among them, the task progress reward item include:

[0118] ;

[0119] In the formula, For a moment Distance to the target location; This refers to the weighting coefficient for task progress reward items; it should be noted that the task progress reward items... The effectiveness of approaching the target position is characterized by the difference in the target distance, thereby improving the arrival efficiency. When the tethered robot moves towards the target direction, A decrease in altitude results in a positive reward; a move away from the target results in a negative reward.

[0120] Based on local observation vectors minimum distance Construct the near obstacle penalty term The local observation vector is obtained by the robot through discrete sampling in multiple sector directions based on observation information, with each component representing the distance to the nearest obstacle in the corresponding direction. Near-obstacle penalty term. include:

[0121] ;

[0122] In the formula, For the obstacle avoidance safety item, the weighting coefficient is , and ; The near-obstacle penalty threshold; when the distance between the tethered robot and the obstacle is less than... At that time, punishment followed The decrease linearly increases, thus providing a continuous risk gradient signal before a collision occurs. Sometimes penalties are not imposed to avoid unnecessary disruption to normal traffic flow.

[0123] The cable restraint risk penalty item This approach explicitly incorporates the safety boundaries and entanglement risks of tethered systems into the policy network. Unlike general mobile robot planning, the risks of tethered platforms often exhibit a "progressive accumulation" characteristic. This means that risks don't only appear upon contact with obstacles or in extremely short moments, but may manifest as a continuous deterioration of the tether configuration over a multi-step sequence of actions, eventually leading to deeper entanglement or exhaustion of length margin. Therefore, this embodiment constructs tether constraint rewards from two complementary perspectives: risk level and risk trend. This allows the policy network to avoid prolonged periods of high risk while simultaneously responding quickly to early signs of risk deterioration.

[0124] After obtaining the aforementioned cable constraint risk penalty item When, it includes the following steps:

[0125] Step A1: Considering that cable risk accumulates over time, we first start with the risk level of the current configuration to construct an initial penalty term by forming a stable safety preference for the strategy on a global scale. ,include:

[0126] ;

[0127] In the formula, and All are weighting coefficients, and all are greater than 0; This represents optimizing the remaining available cable margin. This represents an optimization of cable winding complexity;

[0128] It should be noted that the initial penalty term The purpose is to incorporate the additional cost of cable length and the complexity of detours as a continuous cost into the optimization objective. In other words, even if the tethered robot can continue to advance geometrically, it will be strategically avoided due to the high risk level of the cable, thereby maintaining the safety margin and recoverability of the cable configuration at the global level.

[0129] To be more specific, after optimizing the remaining available cable margin At that time, let the anchor point position be b, and the tethered robot end position be... The straight-line distance from the anchor point to the tethered robot is... for: ;

[0130] Based on cable length estimation Define the remaining available cable margin. for: ;

[0131] remaining available cable margin Normalization and compression mapping are performed, and the result is used as the maximum cable length. To obtain the optimized remaining available cable margin :

[0132] ;

[0133] For the entanglement complexity, let the estimated cable length be... The maintained obstacle-avoidance sequence includes The number of obstacle avoidance events that are active, and the number of events that are active. The number of detours corresponding to each obstacle avoidance event is: , The number of loops indicates the strength, and the entanglement complexity is... for:

[0134] ;

[0135] Based on the maximum allowed complexity Regarding the complexity of entanglement Normalization is performed to obtain the optimized cable winding complexity. :

[0136] .

[0137] Step A2: Simply penalizing risk levels may lead to short-term gains resulting in long-term risk accumulation. Therefore, this embodiment introduces a risk trend penalty model. This is used to detect whether cable risk is continuously worsening. Considering that the nature of risk trend is an incremental signal, a risk trend penalty model is constructed by unilaterally penalizing positive increments. :

[0138] ;

[0139] In the formula, and All are weighting coefficients, and all are greater than 0; This represents the change in the margin of remaining available cable. ; This represents the change in optimizing the complexity of cable winding. ;

[0140] It should be noted that the risk trend penalty model When the risk is mitigating, i.e., the increment is negative, no additional penalty is imposed, thus avoiding suppression of proactive untangling or risk reduction behaviors. When the risk increases, i.e., the increment is positive, the penalty appears immediately and increases with the degree of deterioration, suppressing action sequences that continuously deepen entanglement or continuously deplete cable length margin at the gradient level of the policy network. In other words, the initial penalty term... The risk trend penalty model represents the long-term constraint provided by the risk level term. This indicates short-term suppression provided by the trend term. The two are complementary in the time domain and can significantly improve the strategy's sensitivity to the entanglement formation process and its ability to avoid it in advance.

[0141] Step A3: Regarding the hard constraints related to cable length, it is necessary to ensure that the cable length does not exceed the safety limit. If the maximum allowed length is exceeded, it is considered a serious over-limit risk and the round is terminated with a severe penalty. Therefore, a hard constraint on cable length is constructed based on the cable length estimation method described above. ,include:

[0142] ;

[0143] In the formula, For severe punishment, 0;

[0144] Step A4: To avoid "critical oscillations" when the policy approaches the upper limit, a soft boundary is introduced outside of hard termination, and continuous penalties are applied to the portion exceeding the soft boundary, thus constructing a soft boundary penalty. ,include:

[0145] ;

[0146] In the formula, It is a soft boundary. , This is the soft boundary coefficient, used to limit the soft boundary within a reasonable range. It is generally set to 0.1 and can be adjusted according to the maximum length of the cable. For soft boundary penalty weights, and ;

[0147] It should be noted that soft boundary penalties It can gradually become conservative before the system approaches its limits, reducing the probability of triggering hard termination, thereby improving training stability and creating a more robust safety margin during deployment.

[0148] Step A5: Based on the initial penalty term Risk trend penalty model Hard constraints on cable length and soft boundary penalty The cable constraint term is obtained. ,include:

[0149] .

[0150] In this embodiment, the task progress reward item include:

[0151]

[0152] in, For the current action, For the previous action, All of these are weighting coefficients for the reward item, and all are greater than 0.

[0153] In this embodiment, when updating the policy network and value network simultaneously based on the single-step reward, the weight of the penalty term in the single-step reward is adjusted based on the cable constraint representation information, so that the policy network can automatically change its preference for "efficiency" and "safety" according to the current risk situation at each decision moment.

[0154] Specifically, when the single-step reward function is updated based on an adaptive mechanism, the following steps are included:

[0155] Step B1: Based on the weighting coefficients Weighting coefficients Weighting coefficients Weighting coefficients and weighting coefficients Construct a fixed weight set ; and set a fixed weight Expanded to a function that changes with the state. ;

[0156] Step B2: Optimize the remaining available cable margin And optimize the complexity of cable winding. By performing weighted aggregation, the risk profile is obtained, resulting in an interpretable scalar risk score. :

[0157] ;

[0158] In the formula, For risk aggregation coefficient, Scalar risk scoring It can simultaneously reflect the current level of risk and the trend of risk changes, and is only sensitive to the direction of risk increase, thus consistent with the fact that entangled risk has a gradual cumulative characteristic. Based on scalar risk scoring. The size of the constraint weights is adjusted according to a monotonic function so that the constraint weights are close to the base value when the risk is low, and gradually increase the constraint weights when the risk increases.

[0159] Step B3: Transform the scalar risk score using a saturation mapping. Transform into Enhancement coefficient within the range :

[0160] ;

[0161] ;

[0162] in, ; The coefficient representing the steepness of the change. ; This is the risk trigger threshold; It is an exponentially decaying function;

[0163] Step B4: Based on the enhancement coefficient Define the adaptive form of each weight coefficient in the fixed weight set:

[0164] ;

[0165] in, The basic weight for the low-risk phase; Increase weighting for high-risk phases;

[0166] Step B5: The cable-related penalty term is no longer determined by a fixed weight coefficient during training, but is automatically adjusted according to the current risk situation: when the tethered robot is in an open area and the cable risk is low at the current moment, the scalar risk score is adjusted. When the current time step is smaller than the previous time step, and the enhancement coefficient is smaller. As the value approaches zero, the policy network will be more inclined to use tasks to advance reward items in the next time step. Improve efficiency; when signs of continued risk deterioration or approaching the soft boundary appear at the current moment, scalar risk scoring... When the current time step is larger than the previous time step, and the enhancement coefficient is... When the value increases, the policy network will, in the next time step, add the cable constraint risk penalty item. It significantly amplifies, thereby suppressing further entanglement and length consumption, and actively selects the direction of action to untangle or avoid entanglement.

[0167] It should be noted that in some other embodiments, the weight coefficients of the cable-related penalty terms can always be in an automatically adjustable state, while in order to reduce the computational load of the model, they can also be adjusted only during the training phase.

[0168] Next, proceed to step S104.

[0169] In step S104, the value network constructs a target value based on a single-step reward to characterize the quality of the state vector at the next moment and the continuous action instructions at the current moment, and updates it by reducing the temporal difference error between the current estimate of the value network and the target value. The policy network updates itself based on the value evaluation result along the gradient direction that increases the action value. The single-step reward function is updated based on an adaptive mechanism.

[0170] It should be noted that the target value of the quality output by the value network, the update based on the temporal difference error, and the update along the gradient direction that increases the value of the action are all functions inherent to the value network itself. This embodiment does not limit or improve its functions.

[0171] At step S105, steps S102-S104 are repeated until the tethered robot reaches the target position and the path planning is completed.

[0172] To verify the feasibility and performance of the method proposed in this embodiment, an experiment was conducted in a Python environment based on the deep learning framework PyTorch. A training environment was built in Gazebo, and rviz was used for visualization analysis to evaluate the algorithm's performance. The environment was projected on a two-dimensional plane, with the task area divided by walls and circular obstacles placed within it. The tethered underwater robot inputs environmental information into the algorithm based on the given target location point and then completes the path planning task according to the algorithm's output. Other parameters were set as follows during training: learning rate 0.0005, discount factor... The value is 0.99, which is the soft update parameter. The target motion noise is 0.005. The learning rate is 0.2, the cable untangling criterion is 0.02 rad, and the weight update learning rate is... The smoothing coefficient is 0.001. The upper bound of each weight in step B3 is 0.9. The lower bound of the weight is 7.0. It is 0.4.

[0173] Figure 4 shows the path of a tethered robot in a test round under the condition of a fixed starting point and target point in a small-sized and relatively simple test environment. The red line represents the historical path traversed by the tethered robot, the black line represents the real-time position of the tethered robot, the white circles represent obstacles, and the red dots represent target points. The target points are set in a clockwise order, starting from the upper right corner. As shown in Figure 4(a), when the tethered robot moves from the starting position to the first target point, the interference risk between the cable and the obstacle is low, and the tethered robot can directly approach the target point with minimal cable constraint. As shown in Figure 4(b), when the tethered robot turns from the first target point to the second target point, if a direct approach method is used, it is easy for the cable to make contact with or even become entangled with the obstacle. In this case, guided by the path planning method proposed in this embodiment, the tethered robot does not move directly to the target point, but prioritizes untangling path adjustments to reduce the risk of cable entanglement. After the cable topology is optimized, it continues to advance towards the target point. The results shown in Figures 4(c) and 4(d) are consistent with the above pattern. That is, during subsequent target switching, when directly reaching the target point would increase the risk of cable entanglement or detour between the cable and obstacles, the path planning method proposed in this embodiment can comprehensively consider the target arrival requirements and the cable constraint state, prioritize generating untangling or detour-avoiding paths, and then guide the robot to safely reach the corresponding target position. As can be seen from Figure 4, the path planning method proposed in this embodiment can not only guide the tethered robot to complete multi-target sequence arrival tasks, but also actively suppress the risk of cable entanglement during target switching, avoiding obstacle cable entanglement problems caused by directly approaching the target, thereby improving the safety, topological rationality, and task execution stability of path planning.

[0174] A second embodiment of the present invention also provides a system for path planning of a tethered robot, comprising:

[0175] The initialization unit is configured to construct a two-dimensional working environment model of the tethered robot based on the target scene and set the kinematic constraints of the tethered robot.

[0176] The action output unit is configured to construct a TD3 network model that provides path planning for the tethered robot. The TD3 network model includes a policy network. The policy network constructs a state vector based on the observation information of the two-dimensional working environment model at the current moment and outputs continuous action commands at the current moment based on the kinematic constraints of the tethered robot. The state vector includes cable constraint representation information.

[0177] The reward output unit is configured to execute continuous action instructions in the two-dimensional work environment model and output the state vector at the next moment. The TD3 network model also includes a value network and a single-step reward function. The value network outputs the current estimated value based on the state vector at the next moment and the continuous action instructions at the current moment. The value network also outputs the value evaluation result of the continuous action instructions at the current moment. The single-step reward function outputs the single-step reward based on the state vector at the next moment and the continuous action instructions at the current moment.

[0178] The update unit is configured such that the value network constructs a target value based on a single-step reward to characterize the quality of the state vector at the next moment and the continuous action instructions at the current moment, and updates it by reducing the temporal difference error between the current estimate of the value network and the target value. The policy network updates along the gradient direction that increases the action value based on the value evaluation result. The single-step reward function is updated based on an adaptive mechanism.

[0179] The output unit is configured to repeatedly execute the action output unit, the reward output unit, and the update unit until the tethered robot reaches the target position and completes the path planning.

[0180] The third embodiment of the present invention also provides an electronic device, the electronic device comprising:

[0181] At least one processor; and,

[0182] The memory is communicatively connected to the at least one processor; wherein,

[0183] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the tethered robot path planning method of any of the foregoing embodiments.

[0184] The fourth embodiment of the present invention also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the tethered robot path planning method described in any of the foregoing embodiments.

[0185] The fifth embodiment of the present invention also provides a computer program product, which includes a computing program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to perform the tethered robot path planning method of any of the foregoing embodiments.

[0186] Figure 5The illustration shows a method or device 1000 implementing an embodiment of the present invention. In some embodiments, more or fewer devices may be included than illustrated. In some embodiments, it may be implemented using a single or multiple devices. In some embodiments, it may be implemented using cloud-based or distributed devices.

[0187] like Figure 5 As shown, device 1000 includes a processor 1001 for performing various appropriate operations and processes based on programs and / or data stored in read-only memory (ROM) 1002 or programs and / or data loaded from storage portion 1008 into random access memory (RAM) 1003. Processor 1001 may be a multi-core processor or may contain multiple processors. In some embodiments, processor 1001 may include a general-purpose main processor and one or more special coprocessors, such as a central processing unit (CPU), graphics processing unit (GPU), neural network processor (NPU), digital signal processor (DSP), etc. Various programs and data required for the operation of device 1000 are also stored in RAM 1003. Processor 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0188] The processor and memory described above are used together to execute programs stored in the memory. When the program is executed by a computer, it can implement the methods, steps, or functions described in the above embodiments.

[0189] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, touchscreen, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed. Figure 5 The diagram only shows a portion of the components and does not imply that the device 1000 only includes... Figure 5 The components shown.

[0190] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer or its associated components. The computer may be, for example, a mobile terminal, smartphone, personal computer, laptop computer, in-vehicle human-machine interface device, personal digital assistant, media player, navigation device, game console, tablet computer, wearable device, smart TV, Internet of Things system, smart home, industrial computer, server, or a combination thereof.

[0191] Although not shown, in this embodiment of the invention, a computer-readable storage medium is provided having a computer program / instructions stored thereon, which, when executed by a processor, implements the tethered robot path planning method described in the embodiment.

[0192] Storage media in embodiments of the present invention include articles that are permanent and non-permanent, removable and non-removable, capable of storing information by any method or technology. Examples of storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0193] Although not shown, embodiments of the present invention also provide a computer program product, including: a computer program / instructions that, when executed by a processor, implement the tethered robot path planning method described in the embodiments.

[0194] The methods, programs, systems, apparatuses, etc., in embodiments of the present invention can be executed or implemented in one or more networked computers, or practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks can be performed by remote processing devices connected via a communication network.

Claims

1. A method for path planning of a tethered robot, characterized in that, Includes the following steps: Step S1: Based on the target scenario, construct a two-dimensional working environment model of the tethered robot and set the kinematic constraints of the tethered robot; Step S2: Construct a TD3 network model to provide path planning for the tethered robot. The TD3 network model includes a policy network. The policy network constructs a state vector based on the observation information of the two-dimensional working environment model at the current moment, and outputs continuous action commands at the current moment based on the kinematic constraints of the tethered robot. The state vector includes cable constraint representation information. Step S3: Execute continuous action instructions in the two-dimensional work environment model and output the state vector at the next moment. The TD3 network model also includes a value network and a single-step reward function. The value network outputs the current estimated value based on the state vector at the next moment and the continuous action instructions at the current moment. The value network also outputs the value evaluation result of the continuous action instructions at the current moment. The single-step reward function outputs the single-step reward based on the state vector at the next moment and the continuous action instructions at the current moment. Step S4: The value network constructs a target value based on the single-step reward to characterize the quality of the state vector at the next moment and the continuous action instructions at the current moment, and updates it by reducing the temporal difference error between the current estimated value of the value network and the target value. The policy network updates it along the gradient direction that increases the action value based on the value evaluation result. The single-step reward function is updated based on an adaptive mechanism. Step S5: Repeat steps S2-S4 until the tethered robot reaches the target position and completes path planning.

2. The method for path planning of a tethered robot according to claim 1, characterized in that, The two-dimensional operating environment model of the tethered robot includes at least an anchor point, the tethered robot, the target location, and obstacles; The state vector also includes the pose information of the tethered robot, the target relative pose information, and the obstacle relative geometric information; The cable constraint characterization information includes the cable length estimate and the cable routing topology. When obtaining the estimated cable length, an arc-straight-arc cable model combined with a contact stack topology maintenance mechanism is used to approximate the actual path under the cable tensioned state. The cable path includes the incoming arc segment and the straight segment between the tangent point on the obstacle contact side, as well as the outgoing arc segment leaving the obstacle contact side. The length of each path segment is determined by combining the anchor point position, the tethered robot position, obstacle geometry parameters, and the cable routing topology. Therefore, the estimated cable length is... ,include: ； In the formula, the estimated cable length is... Indicates the set of obstacles Under constraint, when the cable is fully tightened, from the anchor point To the robot end The total length; , This indicates the anchor point location at the fixed end of the cable. , Indicates the position of the tethered robot end; parameters This indicates the number of times the cable effectively wraps around an obstacle along the path. At that time, the cable was not in contact with any obstacle, and its length was [length missing]. ,when At that time, the cable path was composed of multiple straight segments and circular arc segments spliced ​​together sequentially, among which, Indicates the cable at the An ingress point on an obstacle, Indicates the cable at the The point of exit on an obstacle; For the first The effective wrapping radius of an obstacle is used to characterize the geometric contact constraint between the cable and the obstacle; The actual angle of wrap of the cable on the obstacle is determined by the location of the tangent point and the number of turns; symbol Represents the Euclidean norm; The cable routing topology includes at least routing side information and the number of loops. The number of loops is a time-consistent counting result of the number of loops obtained based on the central angle of the arc segment and the angle of the whole loop.

3. The method for path planning of a tethered robot according to claim 1, characterized in that, The single-step reward include: ； In the formula, To achieve the target reward item; This is a penalty item for near-disability; Rewards for progressing through the mission; For cable restraint risk penalty items; This is the smoothing term for the direction of travel.

4. The method for path planning of a tethered robot according to claim 3, characterized in that, The termination reward item include: ； In the formula, This indicates that the tethered robot is at any time. Distance to the target location; To reach the threshold; Event indicators representing collisions or entry into inaccessible areas. ; This represents a one-time positive reward for the tethered robot's successful arrival. ; The penalty magnitude for collisions or entering infeasible areas. If triggered or Then, in this training round at time... If the preset number of training rounds has not been reached, the process will restart in the next training round. A training round refers to the process of updating the value network and the policy network.

5. The method for path planning of a tethered robot according to claim 3, characterized in that, The task progress reward items include: ； In the formula, For a moment Distance to the target location; Weighting coefficients for task progress reward items; Based on local observation vectors minimum distance Construct the near obstacle penalty term The local observation vector is obtained by the robot through discrete sampling in multiple sector directions based on the observation information. Each component in the equation represents the distance to the nearest obstacle in the corresponding direction. The near obstacle penalty term... include: ； In the formula, For the obstacle avoidance safety item, the weighting coefficient is , and ; The near-obstacle penalty threshold; when the distance between the tethered robot and the obstacle is less than... At that time, punishment followed The decrease is linearly increased when No punishment is imposed at the time; The task progress reward items include: ； in, For the current action, For the previous action, All of these are weighting coefficients for the reward item, and all are greater than 0.

6. The method for path planning of a tethered robot according to claim 3, characterized in that, The cable constraint risk penalty item is obtained. When, it includes the following steps: Step A1: Construct the initial penalty term ,include: ； In the formula, and All are weighting coefficients, and all are greater than 0; This represents optimizing the remaining available cable margin. This represents an optimization of cable winding complexity; Let the anchor point be b, and the end position of the tethered robot be... The straight-line distance from the anchor point to the tethered robot is... for: ; Based on cable length estimation Define the remaining available cable margin. for: ; remaining available cable margin Normalization and compression mapping are performed, and the result is used as the maximum cable length. To obtain the optimized remaining available cable margin : ； For the entanglement complexity, let the estimated cable length be... The maintained obstacle-avoidance sequence includes The number of obstacle avoidance events that are active, and the number of events that are active. The number of detours corresponding to each obstacle avoidance event is: , The number of loops indicates the strength, and the entanglement complexity is... for: ； Based on the maximum allowed complexity Regarding the complexity of entanglement Normalization is performed to obtain the optimized cable winding complexity. : ； Step A2: Construct a risk trend penalty model based on one-sided penalties for positive increments. : ； In the formula, and All are weighting coefficients, and all are greater than 0; This represents the change in the margin of remaining available cable. ; This represents the change in the complexity of optimizing cable winding. ; Step A3: Construct hard constraints for cable length based on the cable length estimation calculation method ,include: ； In the formula, For severe punishment; Step A4: Construct soft boundary penalty ,include: ； In the formula, It is a soft boundary. , This is the soft boundary coefficient, used to limit the soft boundary within a reasonable range. It is generally set to 0.1 and can be adjusted according to the maximum length of the cable. These are the weighting coefficients, and ; Step A5: Based on the initial penalty term Risk trend penalty model Hard constraints on cable length and soft boundary penalty The cable constraint term is obtained. ,include: 。 7. The method for path planning of a tethered robot according to claim 6, characterized in that, When the single-step reward function is updated based on an adaptive mechanism, the following steps are included: Step B1: Based on the weighting coefficients Weighting coefficients Weighting coefficients Weighting coefficients and weighting coefficients Construct a fixed weight set ; and set a fixed weight Expanded to a function that changes with the state. ; Step B2: Optimize the remaining available cable margin And optimize the complexity of cable winding. By performing weighted aggregation, the risk profile is obtained, resulting in an interpretable scalar risk score. : ； In the formula, For risk aggregation coefficient, ; Step B3: Transform the scalar risk score using a saturation mapping. Transform into Enhancement coefficient within the range : ； ； in, ; The coefficient representing the steepness of the change. ; This is the risk trigger threshold; It is an exponentially decaying function; Step B4: Based on the enhancement coefficient Define the adaptive form of each weight coefficient in the fixed weight set: ； in, The basic weight for the low-risk phase; Increase weighting for high-risk phases; Step B5: When scalar risk scoring The value is smaller at the current moment compared to the previous moment, and the enhancement coefficient is smaller. As the value approaches zero, the policy network will be more inclined to use tasks to advance reward items in the next time step. Improve efficiency; when scalar risk scoring When the current time step is larger than the previous time step, and the enhancement coefficient is... When the value increases, the policy network will, in the next time step, add the cable constraint risk penalty item. Magnify and actively choose the direction of action to untangle or avoid entanglement.

8. A system for path planning of a tethered robot, characterized in that, The method for tethered robot path planning based on any one of claims 1-7 includes: The initialization unit is configured to construct a two-dimensional working environment model of the tethered robot based on the target scene and set the kinematic constraints of the tethered robot. The instruction output unit is configured to construct a TD3 network model for path planning of the tethered robot. The TD3 network model includes a policy network. The policy network constructs a state vector based on the observation information of the two-dimensional working environment model at the current moment and outputs continuous action instructions based on the kinematic constraints of the tethered robot. The state vector includes cable constraint representation information. The execution update unit is configured to execute continuous action instructions in the two-dimensional work environment model and output the state vector at the next moment. The TD3 network model also includes a value network. The value network obtains a single-step reward based on the state vector at the next moment and the continuous action instructions at the current moment. The policy network and the value network are updated simultaneously based on the single-step reward. The output unit is configured to repeatedly execute the instruction output unit and the execution update unit until the tethered robot reaches the target position and completes the path planning.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, The memory is communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the tethered robot path planning method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method of tethered robot path planning as described in any one of claims 1 to 7.

Images