Industrial robot mechanical arm intelligent obstacle avoidance method based on ac neural network and wsc-h algorithm

Abstract

The application relates to an industrial robot mechanical arm intelligent obstacle avoidance method based on an AC neural network and a WSCA-H algorithm, and solves the defect that it is difficult to intelligently define the safety distance of a robot mechanical arm from an obstacle during operation compared with the prior art. The application comprises the following steps: data acquisition; construction of a mechanical arm obstacle avoidance identification network; training of an AC neural network; calculation of a mechanical arm obstacle avoidance path. The application optimizes the obstacle avoidance path, improves the task execution efficiency, and can effectively improve the production efficiency of the system.

Description

Technical Field

[0001] This invention relates to the field of real-time obstacle avoidance technology for robotic arms, specifically to an intelligent obstacle avoidance method for industrial robot robotic arms based on AC neural networks and WSCA-H algorithms. Background Technology

[0002] In recent years, robotic arm technology has developed rapidly and has been widely applied in various fields such as medicine, manufacturing, and aerospace. However, the diverse working environments in which robotic arms are used present varying obstacle locations and sizes. Therefore, research on obstacle avoidance trajectory planning is essential for achieving automatic control of robotic arms in complex environments. The primary problem that obstacle avoidance-based path planning needs to solve is to reasonably describe the environment model. Traditional methods for representing environment models include grid representation, spatial graphical methods, and minimum polygon bounding box methods. These methods suffer from drawbacks such as high memory consumption, slow solution speed, and low efficiency.

[0003] Pan J et al. calculated obstacle boundaries by estimating the trajectory of moving obstacles over a short time and constructed a cost function for the symbolic Euclidean distance transformation between the robot sphere and the obstacle's position boundary to achieve trajectory planning. They proposed sweep volume as an obstacle avoidance model for the robotic arm in continuous time steps, and Ho J et al. calculated the cost based on the distance between the sweep volume and the obstacle. Miossec S et al. proposed a rigorous convex hull generation method to calculate continuous gradient approach distances and generate free-collision optimized humanoid robot poses. i,S et al. used the Gilbert-Johnson-Keerthi algorithm to calculate the minimum distance between the robotic arm and obstacles.

[0004] However, in dynamic obstacle environments, the distance-based modeling methods mentioned above lose the speed information of obstacles and lack the ability to predict whether a collision will occur, thus reducing the safety of the system.

[0005] Therefore, how to better utilize optimization algorithms to flexibly adjust the soft threshold based on the intelligent definition of the safe distance is a technical problem that urgently needs to be solved to achieve precise obstacle avoidance for robotic arms. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies in intelligently defining the safe distance between a robot arm and obstacles during operation, and to provide an intelligent obstacle avoidance method for industrial robot arms based on AC neural networks and WSCA-H algorithms to solve the above problems.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] An intelligent obstacle avoidance method for an industrial robot arm based on AC neural network and WSCA-H algorithm is proposed. The industrial robot arm includes a reciprocating joint assembly, with a sensor placed at each 45° wrist position of the reciprocating joint assembly. The distance data between the target object and the sensor is acquired using these sensors. The obstacle avoidance method for the industrial robot arm includes the following steps:

[0009] 11) Data acquisition: The collected distance data is preprocessed by removing noise, eliminating outliers, interpolating data or filtering, and the preprocessed distance data is converted into understandable data, including maps and obstacle location information.

[0010] 12) Constructing the obstacle avoidance recognition network for the robotic arm: Construct the obstacle avoidance recognition network for the robotic arm, namely the AC neural network, using actor network and critic network;

[0011] 13) Training the AC neural network: Input understandable data into the AC neural network for training to obtain the distance threshold d. Th ;

[0012] 14) Calculation of obstacle avoidance path for robotic arm: This involves considering the distance threshold d. Th Given a range, use the WSCA-H optimization algorithm to optimize the safe distance threshold d. Th The soft threshold is optimized to enable it to intelligently obtain the optimal safe distance threshold. The obtained optimal safe distance threshold is then fed into the AC neural network for iterative training, and finally the global obstacle avoidance strategy of the robotic arm is obtained.

[0013] The construction of the obstacle avoidance recognition network for the robotic arm includes the following steps:

[0014] 21) Set the AC neural network to be divided into policy network and value network. The policy network is the Actor network and the value network is the Critic network.

[0015] 22) Given a state input to the Actor network, the Actor network is responsible for generating actions. These actions are based on the current state and output through the Actor network. The generated actions are then used to interact with the environment to generate new states and reward signals.

[0016] The Critic network uses these new state and reward signals to evaluate the value of the state under the current policy. At the same time, the Critic network predicts the value of the next state, which helps to calculate the advantage function.

[0017] 23) Define the advantage function Adv in the Critic network. t Used to connect the Actor network and the Critic network, it measures the additional value of taking an action relative to the average policy in a given state.

[0018] Advantage function Adv t The computation relies on the state value estimate provided by the Critic network. The Actor network uses the advantage function to update the policy, thereby improving overall performance by encouraging policies that produce positive advantage actions.

[0019] The value estimation of the Critic network guides the policy optimization of the Actor network, enabling it to explore the environment and increase cumulative rewards. This reward system rewards the action values ​​output by the Actor network, and optimizes the safety distance involved in the reward system to automatically obtain an optimal safety distance. The training of the AC neural network includes the following steps:

[0020] 31) Training the Actor network:

[0021] The sensor-collected and converted data in an understandable form is input into the Actor network, which then generates a probability distribution for each possible action given a given state.

[0022] The policy gradient method is used to train the Actor network, and the policy is optimized through gradient ascent to maximize the expected reward.

[0023] The loss function incorporates a pruning ratio strategy to prevent drastic changes in the policy gradient method during a single iteration. Its formula is expressed as:

[0024]

[0025]

[0026] A_loss=-Adv t logP(A t ,S t )

[0027] in, It is a loss function. Is the new strategy in state s t Take action a t The probability, Is the old strategy in state s? t Take action a t The probability, It is the advantage function at time step t, which measures the effectiveness of taking action a. t The difference between the actual return and the expected return of the strategy, ε is a preset clipping ratio used to limit the magnitude of strategy updates, and is a decimal between 0 and 1. clip(x,y) is a min-max function that returns the minimum of the two parameters, Q.π (s t ,a t ) is the action-state value function, which is the expected reward obtained by performing action a in state s and then following π, r. i+1 It is the reward value, γ∈[0,1] is the discount factor for assigning weights to the reward at different time steps, Adv t It is the current dominant function value, A t It's the action value, S t Here, P is the state value, P is the probability, and A_loss is the loss function.

[0028] For reward design based on distance-based reward and penalty functions,

[0029]

[0030] It is a reward for success, which is considered to be when the robot is a certain distance d from the obstacle. t Less than the distance threshold d Th ,

[0031]

[0032] It's a collision penalty:

[0033]

[0034] It's a distance penalty.

[0035]

[0036] This is beyond the scope of workplace penalties:

[0037]

[0038] The penalty is running out of steps:

[0039]

[0040] It is a penalty for non-smooth motion, where n s This indicates the number of times the robot moved back and forth in the last 10 steps:

[0041]

[0042] 32) Training the Critic network:

[0043] The Critic network uses the input state and action to generate an estimate of the expected reward of the action in the current state, and estimates the state value or action value under the current policy to assist in policy optimization.

[0044] Its expression is:

[0045] V t =R t +γ*V t+1

[0046] Adv t =(R t +γ*V t+1 -V t )+γ*μ*Adv t+1

[0047] C_loss=(R t +γ*V t+1 -V t ) 2 ,

[0048] Among them, V t It is the total return value, V t+1 It is the future earnings value, R t It is the current reward value, γ is the discount factor, μ is the trade-off factor, and Adv is the current reward value. t It is the current advantage value, adv t+1 C_loss is the future advantage value, and C_loss is the loss function.

[0049] The calculation of the obstacle avoidance path of the robotic arm includes the following steps:

[0050] 41) Regarding the safety distance d in the above reward system Th Given a range, the WSCA-H optimization algorithm is used to perform soft threshold optimization and adjustment, enabling it to be intelligently defined and find the optimal value of the distance to the user.

[0051] 42) Regarding the safe distance d in the actor network reward formula Th Given a range, design an optimization algorithm that mimics the real situation of animal hunting. Use mathematical formulas to simulate the strategies of hunters to capture prey under different mechanisms. Evolve the safe distance between the robot's wrist and obstacles. Since the algorithm that mimics animal hunting may get stuck in local optima, use soft threshold optimization to perform secondary optimization on the global search position and the local development position.

[0052] During the optimization process, the predator is a candidate solution, and the prey approaches the optimal solution, i.e. the best safe distance, with iteration. It is then input into the reward system and trained iteratively with the AC neural network to obtain the final obstacle avoidance strategy, thereby reducing the systematic performance deviation caused by the conventional fixed threshold turning strategy.

[0053] 421) Global search phase:

[0054] Predators randomly inhabit a location and use two strategies to find prey:

[0055]

[0056] Where X(t) and X(t+1) are the positions of the individual in the current and next iterations, respectively, t is the iteration number, and X... rand (t) represents the randomly selected individual position, X rabbit (t) represents the prey position, i.e., the position of the individual with optimal fitness. r1, r2, r3, r4, and q are all random numbers between [0, 1]. q is used to randomly select the strategy to be adopted. X m (t) represents the average position of individuals, expressed as:

[0057]

[0058] Among them, X k (t) represents the position of the kth individual in the population, and M represents the population size;

[0059] 422) The conversion between global search and local development:

[0060] Based on the transformation of prey's escape energy between search and different exploitation behaviors, escape energy is defined as:

[0061]

[0062] Where E0 is the initial energy of the prey, which is a random number between [-1, 1] and is automatically updated in each iteration, t is the number of iterations, T is the maximum number of iterations, and when E>1, the search phase is entered, and when |E|<1, the development phase is entered.

[0063] 423) Partial Development Phase

[0064] Define r as a random number between [0, 1], used to select different development strategies.

[0065] When 0.5 ≤ E < 1 and r > 0.5, a soft encirclement strategy is adopted for position update.

[0066] X(t+1)=ΔX(t)-E|JX rabbit (t)-X(t)|

[0067] Where, △X(t)=X rabbit X(t) represents the difference between the prey's location and the individual's current location, and J is a random number between [0,2].

[0068] When |E| < 0.5 and r ≥ 0.5, a hard siege strategy is adopted for position update:

[0069] X(t+1)=X rabbit (t)-E|ΔX(t)|;

[0070] When 0.5 ≤ |E| < 1 and r < 0.5, a fast-dive soft encirclement strategy is adopted for position update:

[0071]

[0072] Y = X rabbit (t)-E|JX rabbit (t)-X(t)|

[0073] Z = Y + S * LF (2)

[0074] Where f() is the fitness function, S is a 2-dimensional random vector with elements being random numbers between [0,1], LF() is the mathematical expression for Levi's flight, and X(t) is the individual position of the siegeer;

[0075] When |E| < 0.5 and r < 0.5, a fast dive hard encirclement strategy is adopted for position update:

[0076]

[0077] Y = X rabbit (t)-E|JX rabbit (t)-X m (t)|

[0078] Z = Y + S * LF (2);

[0079] Among them, X m (t) represents the average position of the attackers;

[0080] 424) Position Iteration Optimization

[0081] The global and local search positions obtained by mimicking animal predation algorithms are updated, and the WSCA-H algorithm is used to perturb the positions after each iteration to avoid the algorithm getting trapped in local optima.

[0082] A novel algorithm is proposed that applies a weight update mechanism based on individual fitness adjustment, expressed as:

[0083]

[0084] Where t represents the current iteration number, x i (t) represents the candidate solution with the highest fitness in the t-th iteration. In t iterations, x represents x iThe weights obtained by (t), where m represents the size of the candidate solution set;

[0085] After determining the weight of each candidate solution, a subset of candidates with higher fitness is selected to calculate the component of the mean:

[0086]

[0087] Among them, c t The size of the preferred candidate solutions participating in the average position calculation decreases as the number of iterations increases.

[0088] To eliminate the potential for algorithmic oscillations caused by excessive inertial weights on candidate solutions during iteration, the WSCA-H candidate solution iterative update equation is as follows:

[0089]

[0090] in, Let x be the position of the i-th individual in the d-th dimension of the t-th generation; * R represents the optimal position of an individual in the iteration; E controls the search speed and direction; r5 is a random number in [0, 2π]; r7 is a random number in [0, 1], and the position update method is selected as sine or cosine.

[0091] 43) The WSCA-H algorithm optimizes the safe distance between the robot arm and obstacles. It represents the position of each individual robot as a safe distance, searches for the optimal safe distance using these individual positions, assigns the optimal solution output by the algorithm to this safe distance, and then feeds it into the reward system of the AC neural network for training.

[0092] 431) Initialize the population: including the upper and lower limits of the search safety distance, the maximum number of iterations T of the algorithm, and the random initialization position of the population;

[0093] 432) Evaluation of fitness function: Calculate the fitness value of each individual based on the fitness function f(), and save the best individual in the population;

[0094] 433) Individual position update: The position of each individual is updated using the predation strategy and position iteration update method in the WSCA-H algorithm, moving towards the global optimum;

[0095] 434) Repeated iteration: Repeat steps 432) and 433) until the stopping condition is met;

[0096] 435) Output: Output the optimal safe distance value, which will be used as the standard for distance evaluation in the reward system.

[0097] Beneficial effects

[0098] This invention presents an intelligent obstacle avoidance method for industrial robot arms based on AC neural networks and the WSCA-H algorithm. Compared with existing technologies, this method introduces the WSCA-H optimization algorithm to achieve intelligent adjustment of the safety distance, thereby obtaining optimal soft threshold adjustment. This optimization algorithm can dynamically adjust the safety distance, ensuring the robot's safety during obstacle avoidance. Compared with traditional fixed-threshold obstacle avoidance strategies, it is more flexible, enabling the robot to adapt to complex and changing working environments. The algorithm can be adjusted according to different environmental requirements, better addressing various scenarios and task demands compared to traditional methods. By optimizing the obstacle avoidance path, this invention improves task execution efficiency and effectively enhances system productivity. Attached Figure Description

[0099] Figure 1 This is a sequence diagram of the method of the present invention;

[0100] Figure 2 This is a comparison chart of the average planned paths for obstacle avoidance by the industrial robot arm of the present invention;

[0101] Figure 3 This is a comparison chart of the average execution time for obstacle avoidance by the industrial robot arm of the present invention. Detailed Implementation

[0102] To provide a better understanding of the structural features and effects achieved by the present invention, a detailed description is provided below, accompanied by preferred embodiments and accompanying drawings:

[0103] like Figure 1 As shown, the intelligent obstacle avoidance method for industrial robot robotic arms based on AC neural networks and WSCA-H algorithms of the present invention includes a reciprocating motion joint assembly. A sensor is placed at each 45° wrist of the reciprocating motion joint assembly to obtain distance data between the target object and the sensor.

[0104] The obstacle avoidance method for industrial robot arms includes the following steps:

[0105] The first step is data acquisition: The collected distance data is preprocessed by removing noise, eliminating outliers, interpolating or filtering data, and converting the preprocessed distance data into understandable data, including maps and obstacle location information.

[0106] The second step is to construct the obstacle avoidance recognition network for the robotic arm: the obstacle avoidance recognition network for the robotic arm, namely the AC neural network, is constructed using actor networks and critic networks.

[0107] AC neural networks integrate policy gradient and value function estimation methods, making them particularly suitable for handling problems with continuous action spaces. This structure allows agents to make efficient decisions in complex environments and adapt quickly to changes in the environment. Its technical challenges include stability issues during training, high variance, and sensitivity to network structure and hyperparameters. Nevertheless, its advantages lie in its ability to effectively combine policy and value functions to achieve asynchronous updates, thereby improving learning efficiency and algorithm stability. Furthermore, the generalization ability of AC neural networks allows them to apply learned policies to new environments, which is particularly important in fields such as intelligent control and machine learning. With further research and technological advancements, AC neural networks are expected to play an increasingly crucial role in future intelligent decision-making systems.

[0108] (1) The AC neural network is divided into a policy network and a value network. The policy network is the Actor network and the value network is the Critic network.

[0109] (2) Given a state input to the Actor network, the Actor network is responsible for generating actions. These actions are based on the current state and output through the Actor network. The generated actions are then used to interact with the environment to generate new states and reward signals.

[0110] The Critic network uses these new state and reward signals to evaluate the value of the state under the current policy. At the same time, the Critic network predicts the value of the next state, which helps to calculate the advantage function.

[0111] (3) Define the advantage function Adv in the Critic network. t The key link connecting the Actor network and the Critic network measures the additional value of taking an action relative to the average policy in a given state.

[0112] Advantage function Adv t The computation relies on the state value estimate provided by the Critic network. The Actor network uses the advantage function to update the policy, thereby improving overall performance by encouraging policies that produce positive advantage actions.

[0113] The value estimation of the Critic network guides the policy optimization of the Actor network, enabling it to explore the environment and increase cumulative rewards. This reward system rewards the action values ​​output by the Actor network, and optimizes the safety distance involved in the reward system so that it automatically obtains an optimal safety distance.

[0114] The third step is training the AC neural network: inputting understandable data into the AC neural network for training to obtain the distance threshold d. Th .

[0115] Optimizing the reward system in AC neural networks to enhance safe distance is of significant technical importance for improving the safety and robustness of agents in complex environments. This optimization can enable agents to learn strategies for maintaining an appropriate safe distance while performing tasks, thereby reducing the risk of collisions or boundary violations, particularly in robot control, autonomous driving, and other automated systems. The main technical challenges include designing a reward function that reflects the importance of safe distance without unduly restricting the agent's exploratory behavior. Furthermore, increasing the consideration of safe distance may lead to increased complexity in the state and action spaces, thus increasing the computational burden on the algorithm. Despite these challenges, the advantages of optimizing the reward system to enhance safe distance are obvious. It not only improves the robustness of the agent, enabling it to better adapt to changing environments, but also reduces human intervention and enhances the system's autonomy. In the long term, this optimization can reduce accidents and failures, leading to better economic benefits. Considering all these factors, optimizing the reward system in AC neural networks to enhance safe distance is a challenging yet valuable task. Through careful design of reward mechanisms and agent learning strategies, it is possible to improve the agent's performance and adaptability while ensuring safety.

[0116] (1) Training the Actor network:

[0117] The sensor-collected and converted data in an understandable form is input into the Actor network, which then generates a probability distribution for each possible action given a given state.

[0118] The policy gradient method is used to train the Actor network, and the policy is optimized through gradient ascent to maximize the expected reward.

[0119] The loss function incorporates a clipping parameter strategy to prevent drastic changes in the policy gradient method during a single iteration. Its formula is expressed as:

[0120]

[0121]

[0122] A_loss=-Adv t logP(A t ,S t )

[0123] in, It is a loss function. Is the new strategy in state s t Take action a t The probability, Is the old strategy in state s? t Take action a t The probability, It is the advantage function at time step t, which measures the effectiveness of taking action a. t The difference between the actual return and the expected return of the strategy, ε is a preset clipping ratio used to limit the magnitude of strategy updates, and is a decimal between 0 and 1. clip(x,y) is a min-max function that returns the minimum of the two parameters, Q. π (s t ,a t ) is the action-state value function, which is the expected reward obtained by performing action a in state s and then following π, r. i+1 It is the reward value, γ∈[0,1] is the discount factor for assigning weights to the reward at different time steps, Adv t It is the current dominant function value, A t It's the action value, S t Here, P is the state value, P is the probability, and A_loss is the loss function.

[0124] For reward design based on distance-based reward and penalty functions,

[0125]

[0126] It is a reward for success, which is considered to be when the robot is a certain distance d from the obstacle. t Less than the distance threshold d Th ,

[0127]

[0128] It's a collision penalty:

[0129]

[0130]

[0131]

[0132] This is beyond the scope of workplace penalties:

[0133]

[0134] The penalty is running out of steps:

[0135]

[0136] It is a penalty for non-smooth motion, where ns This indicates the number of times the robot moved back and forth in the last 10 steps:

[0137]

[0138] (2) Training the Critic Network: The Critic Network is set based on the gap between the actual reward and the predicted reward. The Critic Network is trained by minimizing this error signal. This error signal provides feedback to the Agent, indicating which actions are better choices, thereby prompting the Agent to learn better strategies.

[0139] The Critic network uses the input state and action to generate an estimate of the expected reward of the action in the current state, and estimates the state value or action value under the current policy to assist in policy optimization.

[0140] Its expression is:

[0141] V t =R t +γ*V t+1

[0142] Adv t =(R t +γ*V t+1 -V t )+γ*μ*Adv t+1

[0143] C_loss=(R t +γ*V t+1 -V t ) 2 ,

[0144] Among them, V t It is the total return value, V t+1 It is the future earnings value, R t It is the current reward value, γ is the discount factor, μ is the trade-off factor, and Adv is the current reward value. t This is the current advantage value, Adv. t+1 C_loss is the future advantage value, and C_loss is the loss function.

[0145] Step 4: Calculation of the obstacle avoidance path for the robotic arm: This involves considering the distance threshold d. Th Given a range, use the WSCA-H optimization algorithm to optimize the safe distance threshold d. Th By optimizing the soft threshold, the robot arm can intelligently obtain the optimal safe distance threshold. The obtained safe distance threshold is then fed into the AC neural network for iterative training. Finally, a global obstacle avoidance strategy for the robot arm is obtained, thereby reducing the systematic performance deviation caused by the conventional fixed threshold steering strategy.

[0146] The WSCA-H algorithm aims to improve global search capability and local search efficiency by integrating the advantages of two algorithms. While this integration presents technical challenges, such as the complexity of algorithm integration, the difficulty of parameter tuning, the need for convergence analysis, and potentially increased computational complexity, it also offers significant advantages and profound implications. The main technical challenge lies in effectively integrating the search mechanism from the WSCA-H algorithm into an animal predation optimization algorithm while maintaining its efficiency and adaptability. This involves fine-tuning the algorithm's parameters and conducting thorough experimental testing to verify the new algorithm's performance. Furthermore, theoretical analysis of the new algorithm's convergence is required to demonstrate its reliability in searching for the optimal solution.

[0147] In terms of advantages, the introduction of the WSCA-H algorithm will enhance the global search capability of animal predation optimization algorithms, especially when facing multimodal and complex optimization problems. The search mechanism used can increase the diversity of the search process, thereby increasing the probability of finding the global optimum and improving the robustness of the algorithm. Furthermore, the WSCA-H algorithm promotes more efficient searching within local regions in animal predation optimization algorithms, accelerating convergence. It provides a new strategy for solving complex optimization problems and promotes the development of bio-inspired optimization algorithms, expanding their application areas. It also promotes the integration and interaction of algorithms, providing new ideas for future theoretical research and practical applications. In summary, although there are certain challenges in technical implementation, the attempt to apply the WSCA-H algorithm to position updates after iteration in animal predation optimization algorithms is worthy of further research and exploration due to its potentially significant advantages and broad application prospects. Through careful design and thorough testing, this integration has the potential to address the limitations that existing algorithms may encounter in certain situations, providing more effective solutions to optimization problems.

[0148] (1) For the safety distance d in the above reward system Th Given a range, the WSCA-H optimization algorithm is used to optimize and adjust the soft threshold, enabling it to be intelligently defined and find the optimal value of the distance.

[0149] (2) Regarding the safety distance d in the actor network reward formula Th Given a range, design an optimization algorithm that mimics the real situation of animal hunting. Use mathematical formulas to simulate the strategies of hunters to capture prey under different mechanisms. Evolve the safe distance between the robot's wrist and obstacles. Since the algorithm that mimics animal hunting may get stuck in local optima, use soft threshold optimization to perform secondary optimization on the global search position and the local development position.

[0150] During the optimization process, the predator is a candidate solution, and the prey approaches the optimal solution, i.e. the best safe distance, with iteration. It is then input into the reward system and trained iteratively with the AC neural network to obtain the final obstacle avoidance strategy, thereby reducing the systematic performance deviation caused by the conventional fixed threshold turning strategy.

[0151] A1) Global search phase:

[0152] Predators randomly inhabit a location and use two strategies to find prey:

[0153]

[0154] Where X(t) and X(t+1) are the positions of the individual in the current and next iterations, respectively, t is the iteration number, and X... rand (t) represents the randomly selected individual position, X rabbit (t) represents the prey position, i.e., the position of the individual with optimal fitness. r1, r2, r3, r4, and q are all random numbers between [0, 1]. q is used to randomly select the strategy to be adopted. X m (t) represents the average position of individuals, expressed as:

[0155]

[0156] Among them, X k (t) represents the position of the kth individual in the population, and M represents the population size;

[0157] A2) Conversion between global search and local development:

[0158] Based on the transformation of prey's escape energy between search and different exploitation behaviors, escape energy is defined as:

[0159]

[0160] Where E0 is the initial energy of the prey, which is a random number between [-1, 1] and is automatically updated in each iteration, t is the number of iterations, T is the maximum number of iterations, and when E>1, the search phase is entered, and when |E|<1, the development phase is entered.

[0161] A3) Partial Development Phase

[0162] Define r as a random number between [0, 1], used to select different development strategies.

[0163] When 0.5 ≤ E < 1 and r > 0.5, a soft encirclement strategy is adopted for position update.

[0164] X(t+1)=ΔX(t)-E|JX rabbit (t)-X(t)|

[0165] Where, △X(t)=X rabbit X(t) represents the difference between the prey's position and the individual's current position, and J is a random number between [0,2].

[0166] When |E| < 0.5 and r ≥ 0.5, a hard siege strategy is adopted for position update:

[0167] X(t+1)=X rabbit (t)-E|ΔX(t)|;

[0168] When 0.5 ≤ |E| < 1 and r < 0.5, a rapid dive soft encirclement strategy is adopted, and the encirclement strategy is updated using the position updates of individual attackers:

[0169]

[0170] Y = X rabbit (t)-E|JX rabbit (t)-X(t)|

[0171] Z = Y + S * LF (2)

[0172] Where f() is the fitness function, S is a 2-dimensional random vector with elements being random numbers between [0,1], LF() is the mathematical expression for Levi's flight, and X(t) is the individual position of the siegeer;

[0173] When |E|<0.5 and r<0.5, a rapid dive hard encirclement strategy is adopted, and the encirclement strategy is updated using the average position update of the besiegers:

[0174]

[0175] Y = X rabbit (t)-E|JX rabbit (t)-X m (t)|

[0176] Z = Y + S * LF (2);

[0177] Among them, X m (t) represents the average position of the attackers;

[0178] A4) Position Iteration Optimization

[0179] The global and local search positions obtained by mimicking animal predation algorithms are updated, and the WSCA-H algorithm is used to perturb the positions after each iteration to avoid the algorithm getting trapped in local optima.

[0180] A novel algorithm is proposed that applies a weight update mechanism based on individual fitness adjustment, expressed as:

[0181]

[0182] Where t represents the current iteration number, x i (t) represents the candidate solution with the highest fitness in the t-th iteration. In t iterations, x represents x i The weights obtained by (t), where m represents the size of the candidate solution set;

[0183] After determining the weight of each candidate solution, a subset of candidates with higher fitness is selected to calculate the component of the mean:

[0184]

[0185] Among them, c t The size of the preferred candidate solutions participating in the average position calculation decreases as the number of iterations increases.

[0186] To eliminate the potential for algorithmic oscillations caused by excessive inertial weights on candidate solutions during iteration, the WSCA candidate solution iterative update equation is as follows:

[0187]

[0188] in, Let x be the position of the i-th individual in the d-th dimension of the t-th generation; * R is the optimal position of the individual in the iteration; E controls the search speed and direction; r5 is a random number in [0, 2π]; r7 is a random number in [0, 1], and the position update method is selected as sine or cosine.

[0189] (3) WSCA-H optimizes the safe distance between the robot arm and obstacles.

[0190] The position of each individual is represented as a safe distance. The optimal safe distance is obtained by searching for the individual positions. The optimal solution output by the algorithm is assigned to the optimal safe distance and fed into the reward system of the AC neural network for training.

[0191] B1) Population initialization: including the upper and lower limits of the search safety distance, the maximum number of iterations T of the algorithm, and the random initialization position of the population;

[0192] B2) Evaluation of fitness function: Calculate the fitness value of each individual based on the fitness function f(), and save the best individual in the population;

[0193] B3) Individual position update: The position of each individual is updated using the predation strategy and position iteration update method in the WSCA-H algorithm, moving towards the global optimal solution;

[0194] B4) Repeated iteration: Repeat steps B2) and B3) until the stopping condition is met;

[0195] B5) Output: Output the optimal safe distance value, which will be used as the standard for distance evaluation in the reward system.

[0196] The WSCA-H optimization algorithm is an improved algorithm that combines bionics and mathematical optimization principles. It aims to simulate the hunting and escape strategies of animals to achieve intelligent optimization of both global and local searches. When applied to complex optimization problems, this algorithm employs a unique mechanism to ensure flexibility during iteration and avoid getting trapped in local optima.

[0197] The WSCA-H algorithm is constructed based on a mathematical simulation of animal predation behavior. The core idea is to simulate the optimization process by mimicking animal predation strategies. In this model, the predator represents a candidate solution, and the prey represents the global optimum. Through iterative optimization of the candidate solution's position, the algorithm eventually approaches the global optimum. To avoid oscillations caused by excessive inertial weights during iterations, the WSCA-H algorithm introduces a unique mechanism to adjust weights and optimize position updates. This mechanism ensures that the algorithm can be appropriately perturbed after each iteration, increasing flexibility and reducing the risk of getting trapped in local optima. The algorithm considers adjusting the fitness function, using dynamic weight updates and position perturbations to ensure that the algorithm can still effectively search for the optimal solution in the later stages of iteration.

[0198] The WSCA-H optimization algorithm can be effectively applied to various complex optimization problems, especially in environments requiring a balance between global and local searches. Its flexibility and adaptability make it suitable for applications such as obstacle avoidance and path optimization in industrial robot arms.

[0199] like Figure 2 As shown, this is a comparison of the average planned paths for obstacle avoidance by an industrial robot arm. Regarding path optimization, tests were conducted in four experimental scenarios with different complexities. It was found that the obstacle avoidance path of the WSCA-H algorithm is nearly one-third shorter than the other two algorithms. In the first experimental scenario, the path lengths of the RRT, DRL, and WSCA-H algorithms were 0.92, 1.29, and 1.46, respectively; in the second experimental scenario, the path lengths were 0.93, 1.33, and 1.34; in the third experimental scenario, the path lengths were 0.96, 1.36, and 1.35; and in the fourth experimental scenario, the path lengths were 0.98, 1.12, and 1.30. The test results demonstrate that the WSCA-H algorithm can more effectively reduce the obstacle avoidance path length and improve overall efficiency.

[0200] like Figure 3As shown in the figure, this is a comparison of the average execution time of obstacle avoidance for industrial robot arms. Regarding obstacle avoidance execution time, in four experimental scenarios with different complexities, the WSCA-H algorithm's obstacle avoidance execution time is significantly lower than other algorithms. In the first experimental scenario, the obstacle avoidance times for RRT, DRL, and WSCA-H algorithms were 1.30, 6.77, and 22.39, respectively; in the second experimental scenario, they were 1.30, 8.91, and 25.34; in the third experimental scenario, they were 1.30, 7.64, and 25.58; and in the fourth experimental scenario, they were 1.33, 6.57, and 21.60. The test results demonstrate that the WSCA-H algorithm has higher efficiency and reliability in obstacle avoidance for robot arms, fully reflecting its superiority.

[0201] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for intelligent obstacle avoidance of an industrial robot arm based on AC neural network and WSCA-H algorithm, wherein the industrial robot arm includes a reciprocating joint assembly, and a sensor is installed at each 45° wrist position of the reciprocating joint assembly to acquire distance data between the target object and the sensor; characterized in that, The obstacle avoidance method for industrial robot arms includes the following steps: 11) Data acquisition: The collected distance data is preprocessed by removing noise, eliminating outliers, interpolating or filtering data, and converting the preprocessed distance data into understandable data, including maps and obstacle location information. 12) Constructing the obstacle avoidance recognition network for the robotic arm: Construct the obstacle avoidance recognition network for the robotic arm, namely the AC neural network, using actor networks and critic networks; 13) Training the AC neural network: Input understandable data into the AC neural network for training to obtain the distance threshold. ; 14) Calculation of obstacle avoidance path for robotic arm: based on distance threshold Given a range, use the WSCA-H optimization algorithm to optimize the safe distance threshold. Soft threshold optimization is performed to enable it to intelligently obtain the optimal safe distance threshold. The obtained optimal safe distance threshold is then fed into the AC neural network for iterative training to finally obtain the global obstacle avoidance strategy of the robotic arm. The calculation of the obstacle avoidance path of the robotic arm includes the following steps: 141) Regarding the safe distance in the reward system Given a range, the WSCA-H optimization algorithm is used to perform soft threshold optimization and adjustment, enabling it to be intelligently defined and find the optimal value of the distance to the user. 142) Regarding the safe distance in the actor network reward formula Given a range, design an optimization algorithm that mimics the real situation of animal hunting. Use mathematical formulas to simulate the strategies of hunters to capture prey under different mechanisms. Evolve the safe distance between the robot's wrist and obstacles. Since the algorithm that mimics animal hunting may get stuck in local optima, use soft threshold optimization to perform secondary optimization on the global search position and the local development position. During the optimization process, the predator is a candidate solution, and the prey approaches the optimal solution, i.e. the best safe distance, with iteration. It is then input into the reward system and trained iteratively with the AC neural network to obtain the final obstacle avoidance strategy, thereby reducing the systematic performance deviation caused by the conventional fixed threshold turning strategy. 1421) Global search phase: Predators randomly inhabit a location and use two strategies to find prey: ， Where X(t) and X(t+1) are the positions of the individual in the current and next iterations, respectively, t is the iteration number, and X... rand (t) represents the randomly selected individual position, X rabbit (t) represents the prey position, i.e., the position of the individual with the optimal fitness. r1, r2, r3, r4, and q are all random numbers between [0, 1]. q is used to randomly select the strategy to be adopted. X m (t) represents the average position of individuals, expressed as: ， Among them, X k (t) represents the position of the kth individual in the population, and M represents the population size; 1422) The conversion between global search and local development: Based on the transformation of prey's escape energy between search and different exploitation behaviors, escape energy is defined as: ， Where E0 is the initial energy of the prey, which is a random number between [-1, 1] and is automatically updated in each iteration, t is the number of iterations, T is the maximum number of iterations, and when E>1, the search phase is entered, and when |E|<1, the development phase is entered. 1423) Partial Development Phase Define r as a random number between [0, 1], used to select different development strategies. When 0.5 ≤ E < 1 and r > 0.5, a soft encirclement strategy is adopted for position update. ， Where, △X(t)=X rabbit X(t) represents the difference between the prey's location and the individual's current location, and J is a random number between [0,2]. When |E| < 0.5 and r ≥ 0.5, a hard siege strategy is adopted for position update: ； When 0.5 ≤ |E| < 1 and r < 0.5, a fast-dive soft encirclement strategy is adopted for position update: ， ， ， Where f() is the fitness function, S is a 2-dimensional random vector with elements being random numbers between [0,1], and LF() is the mathematical expression for Levi's flight; For the individual positions of the attackers; When |E| < 0.5 and r < 0.5, a fast-dive hard-surround strategy is adopted for position update: ， ， ； in, Average position of the attackers; 1424) Position Iteration Optimization The global and local search positions obtained by mimicking animal predation algorithms are updated, and the WSCA-H algorithm is used to perturb the positions after each iteration to avoid the algorithm getting trapped in local optima. A novel algorithm is proposed that applies a weight update mechanism based on individual fitness adjustment, expressed as: ， Where t represents the current iteration number, This represents the candidate solution with the highest fitness in the t-th iteration. In t iterations The obtained weights, where m represents the size of the candidate solution set; After determining the weight of each candidate solution, a subset of candidates with higher fitness is selected to calculate the component of the mean: ， in, The size of the preferred candidate solutions participating in the average position calculation decreases as the number of iterations increases. ; To eliminate the potential for algorithmic oscillations caused by excessive inertial weights on candidate solutions during iteration, the WSCA-H candidate solution iterative update equation is as follows: ， in, Let i be the position of the i-th individual in the d-th dimension of the t-th generation; This represents the optimal position of the individual during the iteration; Control the search speed and direction; A random number in the range [0, 2π]. For a random number in the range [0,1], choose either sine or cosine as the position update method; 143) WSCA-H optimizes the safe distance between the robot arm and obstacles. The position of each individual is represented as a safe distance. The optimal safe distance is obtained by searching for the individual positions. The optimal solution output by the algorithm is assigned the optimal safe distance and fed into the reward system of the AC neural network for training. 1431) Population initialization: including the upper and lower limits of the search safety distance, the maximum number of iterations T of the algorithm, and the random initialization position of the population; 1432) Evaluation of fitness function: Calculate the fitness value of each individual based on the fitness function f(), and save the best individual in the population; 1433) Individual position update: The position of each individual is updated using the predation strategy and position iteration update method in the WSCA-H algorithm, moving towards the global optimum; 1434) Repeated iteration: Repeat steps 1432) and 1433) until the stopping condition is met; 1435) Output result: Output the optimal safe distance value, which will be used as the standard for distance evaluation in the reward system.

2. The intelligent obstacle avoidance method for industrial robot robotic arms based on AC neural networks and WSCA-H algorithm according to claim 1, characterized in that, The construction of the obstacle avoidance recognition network for the robotic arm includes the following steps: 21) Set the AC neural network to be divided into policy network and value network. The policy network is the Actor network and the value network is the Critic network. 22) Given a state input to the Actor network, the Actor network is responsible for generating actions. These actions are based on the current state and output through the Actor network. The generated actions are then used to interact with the environment to generate new states and reward signals. The Critic network uses these new state and reward signals to evaluate the value of the state under the current policy. At the same time, the Critic network predicts the value of the next state, which helps to calculate the advantage function. 23) Define the dominance function in the Critic network. Used to connect the Actor network and the Critic network, it measures the additional value of taking an action relative to the average policy in a given state. Advantage function The computation relies on the state value estimate provided by the Critic network. The Actor network uses the advantage function to update the policy, thereby improving overall performance by encouraging policies that produce positive advantage actions. The value estimation of the Critic network guides the policy optimization of the Actor network, enabling it to explore the environment and increase cumulative rewards. The reward system rewards the action values ​​output by the Actor network, and optimizes the safety distance involved in the reward system so that it automatically obtains an optimal safety distance.

3. The intelligent obstacle avoidance method for industrial robot robotic arms based on AC neural networks and WSCA-H algorithms according to claim 1, characterized in that, The training of the AC neural network includes the following steps: 31) Training the Actor network: The sensor-collected and converted data in an understandable form is input into the Actor network, which then generates a probability distribution for each possible action given a given state. The policy gradient method is used to train the Actor network, and the policy is optimized through gradient ascent to maximize the expected reward. The loss function incorporates a pruning ratio strategy to prevent drastic changes in the policy gradient method during a single iteration. Its formula is expressed as: )， ， ， in, It is a loss function. Is the new strategy in the state? Take action below The probability, Is the old strategy in the state? Take action below The probability, It is the advantage function at time step t, which measures the action taken. The difference between the actual return and the expected return of the strategy `clip(x,y)` is a preset clipping ratio used to limit the magnitude of policy updates; it is a decimal between 0 and 1. The maximum function returns the minimum value among two arguments. It is the action-state value function, which is the expected reward obtained by performing action a in state s and then following π. It is the reward value, and γ∈[0,1] is the discount factor for assigning weights to the reward at different time steps. It is the current dominant function value. It's the action value. P is the state value, and P is the probability. The loss function; For reward design based on distance-based reward and penalty functions, ， It is a reward for success, which is considered to be the distance the robot gets from the obstacle. Less than the distance threshold , ， It's a collision penalty: ， It's a distance penalty. ， This is beyond the scope of workplace penalties: ， The penalty is running out of steps: ， It is a penalty for non-smooth motion, in which This indicates the number of times the robot moved back and forth in the last 10 steps: ； 32) Training the Critic network: The Critic network uses the input state and action to generate an estimate of the expected reward of the action in the current state, and estimates the state value or action value under the current policy to assist in policy optimization. Its expression is: ， ， ， in It is the total revenue value. It is the future earnings value. This is the current reward value. It is a discount factor. It is a trade-off factor. This is the current advantage value. It is a future advantage value. Loss function.

Images