A path planning method for flexible hand grabbing operation of deep-sea submersible

Abstract

The present invention provides a deep-sea submersible flexible hand grasping operation path planning method, which belongs to the deep-sea submersible path planning technology, and specifically includes the following steps: initializing parameters; using random initialization to initialize the original population, using a data preprocessing method based on the kmeans algorithm to perform data preprocessing operations on the randomly initialized population; using the Q_learning algorithm to obtain the most suitable parameter combination for the operation of the particle swarm algorithm; calculating whether the number of consecutive unupdated particles is greater than a threshold, and replacing the component of the current particle in this dimension with the component of the global optimal solution in this dimension; using an improved simulated annealing algorithm strategy to update the global optimal solution in the population; until the requirement of the number of iterations is met, the optimal planning path of the flexible hand is output. The technical solution of the present invention overcomes the problem that the prior art cannot plan the grasping operation path of the submersible flexible hand in a deep-sea environment.

Description

Technical Field

[0001] The present invention relates to the technical field of deep-sea submersible path planning, and in particular to a deep-sea submersible flexible hand grasping operation path planning method. Background Art

[0002] In many deep-sea operations, how to efficiently and accurately grasp the target object is one of the key technologies. This requires the deep-sea submersible to be equipped with a highly flexible flexible hand and combined with advanced path planning algorithms to achieve precise grasping tasks.

[0003] Although traditional rigid manipulators can provide large torque output, they are difficult to adapt to changing working conditions in complex deep-sea environments. In contrast, due to its unique flexibility, flexible hands can better adapt to unknown or changing working environments, reduce the risk of collision with the environment, and improve the safety and flexibility of operations. In addition, flexible hands also have good load capacity, which can effectively reduce the overall weight of deep-sea submersibles and improve their maneuverability and endurance. Path planning of flexible hands is crucial to ensure the success of the mission. Reasonable path planning can not only avoid collisions with seabed obstacles, but also reduce unnecessary energy consumption and improve work efficiency. Especially when performing delicate grasping tasks, such as collecting fragile biological samples or repairing deep-sea facilities, accurate path planning is indispensable.

[0004] Therefore, there is a need for a path planning method for flexible hand grasping operations of a submersible suitable for deep-sea environments. Summary of the invention

[0005] The main purpose of the present invention is to provide a method for planning a path for a flexible hand grasping operation of a deep-sea submersible, so as to solve the problem in the prior art that the flexible hand grasping operation path of a submersible in a deep-sea environment cannot be planned.

[0006] To achieve the above object, the present invention provides a method for planning a path for a deep-sea submersible flexible hand grasping operation, which specifically includes the following steps:

[0007] S1, initializing parameters, including: map start and end point information, terrain and threat constraint information, and flexible hand movement characteristic information. The flexible hand movement characteristic information includes: flexible hand movement time, movement speed, track length, and pitch angle.

[0008] S2, use random initialization to initialize the original population, and use the data preprocessing method based on the kmeans algorithm to perform data preprocessing operations on the randomly initialized population.

[0009] S3, use the Q_learning algorithm to obtain the most suitable parameter combination for the particle swarm algorithm operation.

[0010] S4, calculate whether the number of consecutive unupdated particles is greater than the threshold , if greater than , it means that the particle has not been updated for a long time, and in each dimension, the component of the current particle in that dimension will be replaced by the component of the global optimal solution in that dimension with probability c.

[0011] S5, uses the improved simulated annealing algorithm strategy to update the global optimal solution in the population.

[0012] S6, repeat steps S3-S5 until the number of iterations is met, and output the optimal planning path of the flexible hand.

[0013] Furthermore, in step S2, the initialization of the original population using random initialization specifically includes the following steps:

[0014] S2.1, assuming the path is dimensional vector, i.e. points are fitted, particles form a community. For example, in describing the particle path In the coordinate community, A particle is represented as a Dimensional vector:

[0015] (1);

[0016] Indicates the path Coordinate point location, Indicates the first Points coordinate.

[0017] Furthermore, in step S2, the data preprocessing operation of performing data preprocessing on the randomly initialized population using the data preprocessing method based on the kmeans algorithm specifically includes the following steps:

[0018] S2.2, The cluster centers are set to , calculate the Euclidean distance from each individual to each cluster center :

[0019] (2);

[0020] in, Represents the number of individuals to plan the path Take coordinates as an example, Indicates Individuals, that is, paths Coordinate point location, Indicates cluster centers, among which Indicates The individual properties, Indicates The cluster center attributes.

[0021] S2.3, calculate the cluster center of each cluster:

[0022] (3);

[0023] in, For the The center of the cluster, For the The number of individuals in the cluster.

[0024] S2.4, calculate the average fitness value of individuals in each cluster to distinguish the differences between different clusters:

[0025] (4);

[0026] in, It is The average fitness value of the cluster, Is an individual The fitness value of .

[0027] S2.5, sort the average fitness values ​​of all clusters. Taking the minimization problem as an example, all clusters are arranged in ascending order according to the average fitness value, and individuals with smaller fitness values ​​are regarded as dominant individuals.

[0028] Furthermore, the following steps are specifically included when selecting the dominant individual:

[0029] S2.5.1, first, assume The extraction rate for each cluster is the set of clusters sorted in ascending order. , and then proceed Iterations.

[0030] S2.5.2, starting from the first cluster, select a class cluster in order in each iteration and extract from the cluster The individuals are added to the set as dominant individuals. After each iteration, By reducing the value of by 5, as many dominant individuals as possible can be selected to improve the quality of candidate solutions while ensuring population diversity.

[0031] S2.5.3, after completing After iterations, in the set The number of dominant individuals extracted from , the population is insufficient Individuals will be randomly generated, and the final output is the preprocessed population .

[0032] Furthermore, step S3 specifically includes the following steps:

[0033] S3.1, initialize the Q table of each particle, find the particle with the best fitness value from the particles, the Q table corresponding to the particle is the current optimal Q table, and the Q table update formula is:

[0034] (5);

[0035] In the formula, , is the state at the tth iteration and t+1th iteration; , are the actions at the tth iteration and t+1th iteration respectively, and are learning rate and discount factor, respectively, with a value range of [0,1], For particles in state Next action The instant rewards you receive, The particle is in the state Take Action The expected Q value of is the maximum expected future reward over all possible actions in the new state.

[0036] Furthermore, step S3 specifically includes the following steps:

[0037] S3.2, select particles from the initial population after data preprocessing, and predict each particle through the Q_learning algorithm combined with the optimal Q table. After obtaining the prediction result, the global search , random search , comprehensive search and local search Select the behavior that best suits particle search from the four behaviors, and set the reward function to:

[0038] (6);

[0039] (7);

[0040] in, After update The fitness value of To update the previous The fitness value of is the maximum number of iterations of the algorithm, and action is the selected behavior.

[0041] S3.3, use formula (8) and formula (9) to update the position and velocity of the particle. After all particles are updated, select the particle with the best fitness value as the optimal particle, and update the Q table according to the particle's position and performance:

[0042] (8);

[0043] ;

[0044] (9);

[0045] In the formula, S is the number of particles, is the position of the particle at the tth iteration, is the maximum number of iterations, is the inertia weight parameter; and To name the learning factor, we control the positive knowledge obtained from the individual and group historical optimal positions in each iteration. and are two random numbers in [0,1], and are the historical optimal positions of the individual and the group, For the The speed at which the iterations are performed.

[0046] Furthermore, step S4 is specifically as follows:

[0047] The number of consecutive unupdated particles is ,when Exceeding the threshold When To update the position of the current particle in each dimension , the value of c is 0.7; The calculation formula is:

[0048] (10);

[0049] in, and Respectively The upper and lower bounds of .

[0050] Furthermore, step S5 specifically includes the following steps:

[0051] S5.1, introduce simulated annealing algorithm to update the global optimal solution of the population ,parameter The calculation formula is set as follows:

[0052] (11);

[0053] In the formula, is the current system temperature, is the natural exponential function, is the fitness value of the current global optimal solution at the tth iteration.

[0054] S5.2, using Eq. To update ,in, It is a parameter that controls the temperature change rate, and its value range is [0,1].

[0055] S5.3, The calculation formula is:

[0056] (12).

[0057] S5.4, after the specified number of iterations, execute the temperature recovery formula shown in formula (12):

[0058] (13);

[0059] in, is the specified number of iterations, set to 200, is a constant, set to 0.05.

[0060] The present invention has the following beneficial effects:

[0061] The present invention combines the particle swarm algorithm with the Q_learning algorithm, constructs a Q table for each particle, provides dynamic selection of particle parameters, selects appropriate search strategies at different iteration stages, and balances the global search capability and local search capability of the algorithm. In terms of data preprocessing, the algorithm uses a data preprocessing strategy based on the kmeans clustering algorithm, which divides the algorithm's initialization population into multiple different clusters, sorts the clusters according to the average fitness values ​​of the clusters, and selects individuals with better performance as the initialization population in proportion, thereby improving the quality of the initial solution. In addition, the present invention improves the update strategy of the global optimal solution in the particle swarm algorithm by integrating the improved simulated annealing algorithm, enhances the optimization capability of the algorithm, and avoids falling into local convergence. At the same time, the algorithm records the number of consecutive unupdated numbers of each particle. When it exceeds a certain threshold, the position of the current particle in each dimension will be updated according to the global optimal solution. This strategy accelerates the convergence of particles to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the specific implementation or the prior art description. Obviously, the drawings described below are some implementations of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0063] Figure 1 A flow chart of a method for planning a path for a deep-sea submersible flexible hand grasping operation according to the present invention is shown.

[0064] Figure 2 The flowchart of the data preprocessing operation of the data preprocessing method based on the kmeans algorithm of the present invention on a randomly initialized population is shown.

[0065] Figure 3 A structural diagram of a deep-sea submersible with a flexible hand according to the present invention is shown.

[0066] The following is a description of the attached drawings:

[0067] 10. Deep-sea submersible; 20. Flexible hands. DETAILED DESCRIPTION

[0068] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] A method for planning a path for a deep-sea submersible flexible hand grasping operation, specifically comprising the following steps:

[0070] S1, initializing parameters, including: map start and end point information, terrain and threat constraint information, and flexible hand movement characteristic information. The flexible hand movement characteristic information includes: flexible hand movement time, movement speed, track length, and pitch angle.

[0071] S2, use random initialization to initialize the original population, and use the data preprocessing method based on the kmeans algorithm to perform data preprocessing operations on the randomly initialized population.

[0072] S3, use the Q_learning algorithm to obtain the most suitable parameter combination for the particle swarm algorithm operation.

[0073] S4, calculate whether the number of consecutive unupdated particles is greater than the threshold , if greater than , it means that the particle has not been updated for a long time. In each dimension, the component of the current particle in that dimension will be replaced by the component of the global optimal solution in that dimension with probability c, that is, the position of the current particle in each dimension is updated to achieve the effect of accelerating particle convergence. In general, the value of c is 0.7.

[0074] S5, uses the improved simulated annealing algorithm strategy to update the global optimal solution in the population.

[0075] S6, repeat steps S3-S5 until the number of iterations is met, and output the optimal planning path of the flexible hand.

[0076] Combination Figure 1 Briefly describe the principle of the method provided by the present invention:

[0077] The present invention proposes a path planning method for a deep-sea submersible flexible hand grasping operation, such as Figure 1 As shown, the method mainly comprises the following steps:

[0078] 1. First, input the data required for the algorithm to run, including the map's starting point information, end point information, map constraints, and algorithm parameters.

[0079] 2. Next, the data preprocessing process is carried out. Taking the number of particles in the population as N as an example, the initialization population is first randomly generated, and then the initialization population is clustered using the kmeans clustering algorithm. After that, the particles with better performance are selected in proportion as the population after preprocessing (the individuals are 1, 2, 3, ..., N).

[0080] 3. Next, initialize the Q table of each particle. The Q table corresponding to the particles in the population is , and then find the particle with the best fitness value from the particles. The Q table corresponding to the particle is the current optimal Q table. Then, let each particle in the preprocessed population predict the next behavior through the optimal Q table. The particle will choose a behavior from global search, random search, comprehensive search, and local search to carry out the next search process. Each behavior corresponds to a unique parameter combination. The particle will choose different algorithm parameters according to different actions. .

[0081] 4. Next, update the position of the particle through the formula and speed , and calculate the number of consecutive unupdated individuals After all particles have been updated, the particle with the best fitness value is selected as the optimal particle, and the Q table is updated according to the position and performance of the particle.

[0082] 5. Next, while updating the particle's position and velocity, calculate the number of consecutive unupdated particles. , to determine the particle Is it greater than the parameter m? If it is greater than m, it means that the particle has not been updated for a long time. The position of the current particle in each dimension will be updated according to the global optimal solution with probability c to achieve the effect of accelerating particle convergence.

[0083] 6. Next, we will use the simulated annealing algorithm strategy to update the global optimal solution in the population. After completing the update of the particle position and velocity, we will select the global optimal solution in the current particle as the temporary global optimal solution. , if the fitness of the temporary optimal solution is Better than the global optimum of the previous generation , then choose As the global optimal solution of the next generation; if the global optimal solution of the previous generation The fitness value is better than the temporary optimal solution The fitness value is p, and the suboptimal solution is selected with a probability of p. As the next generation of global optimal solution. Next, determine whether the number of iterations meets the requirement. If it does not reach the maximum number of iterations, return to step 3 and execute again.

[0084] like Figure 3 As shown, the deep-sea submersible 10 provided with the flexible hand 20 can realize the functions of collecting fragile biological samples or repairing deep-sea facilities.

[0085] Specifically, in step S2, initializing the original population using random initialization specifically includes the following steps:

[0086] S2.1, assuming the path is dimensional vector, i.e. points are fitted, particles form a community. For example, in describing the particle path In the coordinate community, A particle is represented as a Dimensional vector:

[0087] (1);

[0088] Indicates the path Coordinate point location, Indicates the first Points Coordinates. Similarly, the initialization population includes two populations, Y and Z. In this way ( , , ) represents the coordinates of a point in a path.

[0089] After the population is randomly initialized, a data preprocessing method based on the kmeans algorithm is used to perform data preprocessing operations on the randomly initialized population, and then an initialized population after data preprocessing is obtained, and this population will enter the next algorithm iteration operation.

[0090] The present invention adopts kmeans clustering algorithm as data preprocessing method, by which individuals in the initial population are divided into The purpose is to find the essential differences between individuals in the initial population, so as to screen out some individuals with good attributes and good potential, and let these individuals with good attributes guide the evolutionary direction of the entire population, thereby improving the quality of the candidate solutions of the initial population.

[0091] The main idea of ​​the kmeans algorithm is to set Under the condition of cluster center points, each point (i.e. data record) is divided into the nearest cluster. After all points are assigned, the cluster center point is recalculated based on all points in the cluster (taking the average), and then the cluster center point assignment and update steps are iterated until the cluster center point changes little or reaches the specified number of iterations. Finally, the output of the algorithm is the classified Clusters. Usually, There are essential differences between individuals in the clusters.

[0092] Assume that the initial population is The goal of the kmeans algorithm is to classify Individuals are clustered into specified There are cluster centers, and each individual belongs to only one cluster with the smallest distance from the cluster center.

[0093] Specifically, in step S2, the data preprocessing operation of performing data preprocessing on the randomly initialized population using the data preprocessing method based on the kmeans algorithm specifically includes the following steps:

[0094] S2.2, The cluster centers are set to , calculate the Euclidean distance from each individual to each cluster center :

[0095] (2);

[0096] in, Represents the number of individuals to plan the path Take coordinates as an example, Indicates Individuals, that is, paths Coordinate point location, Indicates cluster centers, among which Indicates The individual properties, Indicates The cluster center attributes.

[0097] S2.3, calculate the cluster center of each cluster:

[0098] (3);

[0099] in, For the The center of the cluster, For the The number of individuals in the cluster. After the kmeans algorithm is executed, the cluster is obtained. .

[0100] S2.4, calculate the average fitness value of individuals in each cluster to distinguish the differences between different clusters:

[0101] (4);

[0102] in, It is The average fitness value of the cluster, is the fitness function, Is an individual The fitness value of .

[0103] S2.5, sort the average fitness values ​​of all clusters. Taking the minimization problem as an example, all clusters are arranged in ascending order according to the average fitness value, and individuals with smaller fitness values ​​are regarded as dominant individuals.

[0104] Specifically, the following steps are included when selecting dominant individuals:

[0105] S2.5.1, first, assume The extraction rate for each cluster is the set of clusters sorted in ascending order. , and then proceed Iterations.

[0106] S2.5.2, starting from the first cluster, select a class cluster in order in each iteration and extract from the cluster The individuals are added to the set as dominant individuals. After each iteration, By reducing the value of by 5, as many dominant individuals as possible can be selected to improve the quality of candidate solutions while ensuring population diversity.

[0107] S2.5.3, after completing After iterations, in the set The number of dominant individuals extracted from , the population is insufficient Individuals will be randomly generated, and the final output is the preprocessed population .

[0108] In the data preprocessing method, the input of the algorithm is composed of , count variables and initial extraction ratio Then, the initialized population is clustered and divided into clusters and calculate the average fitness value of each class The next step is to select the dominant individual. First, let is the extraction rate for each cluster, and the clusters sorted in ascending order are Then proceed Iterations. Starting from the first cluster, each iteration selects a cluster in sequence and extracts The individuals are added to the set as dominant individuals. After each iteration, By reducing the value of by 5, we can select as many dominant individuals as possible to improve the quality of candidate solutions while ensuring population diversity. After iterations, in the set The number of dominant individuals extracted from , the population is insufficient Individuals will be randomly generated, and the final output is the preprocessed population .

[0109] First, from Figure 2 As can be seen in , the original initial population is divided into 10 clusters by the kmeans clustering algorithm, and then sorted according to the average fitness value of these 10 clusters. Take the minimization problem as an example: Figure 2 In the figure, the initialized population is divided into ten clusters by the kmeans clustering algorithm, namely cluster 1, cluster 2, ..., cluster 10. The red cluster has the lowest average fitness, the yellow cluster ranks second, and so on. The green cluster has the highest average fitness. Then, individuals are selected as dominant individuals in proportion in each cluster. The red cluster will extract 100% of the individuals, and the extraction ratio of subsequent clusters will decrease by 5% successively. By analogy, the yellow cluster will extract 95% of the individuals, and the green cluster will extract 55% of the individuals. After generating the set of dominant individuals, the insufficient individuals are randomly generated. Finally, the initial population after data preprocessing consists of the set S and the insufficient individuals.

[0110] The particles in the particle swarm algorithm are used as the intelligent agents in the Q_learning algorithm, the particle position is defined as the state, the particle parameter change is defined as the action, the fitness value change is defined as the reward, and the evaluation function is defined as the environment. The Q_learning algorithm can perform adaptive parameter control on the particle swarm algorithm to improve the performance of the particle swarm algorithm.

[0111] The search ability of particles is mainly affected by three parameters: , is the inertia weight, and are individual learning weights and social learning weights. These three parameters will affect the search behavior of particles, thereby affecting the global search and local search capabilities of particles. Determines how much the particle inherits its velocity, The larger it is, the stronger the global search capability is; The smaller the value, the stronger the local search capability. Similarly, setting a larger individual learning weight , helps to enhance global search, while larger social learning weights , which will cause the particles to converge.

[0112] Therefore, the present invention sets four actions to make them consistent with the search behavior of particles, and divides the actions of particles into four categories: {global search ( ), random search ( ), comprehensive search ( ), local search ( )}. Each action corresponds to a different combination of algorithm parameters. By choosing different actions, different parameters of the particle swarm algorithm are obtained to control the search behavior of particles, thereby changing the search ability of particles. The setting method is shown in Table 1.

[0113] Table 1 Action description

[0114] .

[0115] Table 2 Action parameter settings

[0116] .

[0117] In order to improve the performance of the particle swarm optimization algorithm, this paper proposes a new method to promote information exchange between particles. Specifically, this method enhances the interaction between particles by introducing a particle experience sharing mechanism. Each particle maintains a Q table to record its "action experience". During the entire iteration process, the best performing particle ( ) is taken as the global optimal Q table ( ). Each particle will refer to The "experience" in the group is used to share experience. This sharing mechanism imitates the learning process of human society - individuals can not only learn from their own experience, but also acquire knowledge from peers who perform better. In this way, particles can not only retain their own experience accumulation, but also learn from the strategy of the global optimal solution, thereby enhancing the local search ability of the group and increasing the possibility of finding the optimal solution.

[0118] Specifically, step S3 includes the following steps:

[0119] S3.1, initialize the Q table of each particle, find the particle with the best fitness value from the particles, the Q table corresponding to the particle is the current optimal Q table, and the Q table update formula is:

[0120] (5);

[0121] In the formula, , is the state at the tth iteration and t+1th iteration; , are the actions at the tth iteration and t+1th iteration respectively, and are learning rate and discount factor, respectively, with a value range of [0,1], For particles in state Next action The instant rewards you receive, The particle is in the state Take Action The expected Q value of is the maximum expected future reward over all possible actions in the new state.

[0122] In the field of reinforcement learning, the reward function guides the learning process of the agent. Lack of effective rewards may lead to inefficient learning or even failure to achieve the optimal strategy. Similarly, in the combination of particle swarm optimization algorithm and reinforcement learning, the design of the reward function is crucial.

[0123] Specifically, step S3 includes the following steps:

[0124] S3.2, select particles from the initial population after data preprocessing, and predict each particle through the Q_learning algorithm combined with the optimal Q table. After obtaining the prediction result, the global search , random search , comprehensive search and local search Select the behavior that best suits particle search from the four behaviors, and set the reward function to:

[0125] (6);

[0126] (7);

[0127] in, After update The fitness value of To update the previous The fitness value of is the maximum number of iterations of the algorithm, and action is the selected behavior.

[0128] S3.3, use formula (8) and formula (9) to update the position and velocity of the particle. After all particles are updated, select the particle with the best fitness value as the optimal particle, and update the Q table according to the particle's position and performance:

[0129] (8);

[0130]

[0131] (9);

[0132] In the formula, S is the number of particles, is the position of the particle at the tth iteration, is the maximum number of iterations, is the inertia weight parameter; and To name the learning factor, we control the positive knowledge obtained from the individual and group historical optimal positions in each iteration. and are two random numbers in [0,1], and are the historical optimal positions of the individual and the group, For the The speed at which the iterations are performed.

[0133] Specifically, step S4 is as follows:

[0134] The number of consecutive unupdated particles is ,when Exceeding the threshold When To update the position of the current particle in each dimension , the value of c is 0.7; when When it is too large, the algorithm converges too quickly, which will reduce the diversity of the population and make it easy to fall into the local optimum. Too small and the algorithm may not work effectively. The calculation formula is:

[0135] (10);

[0136] in, and Respectively The upper and lower bounds of .

[0137] The simulated annealing algorithm strategy is used to update the global optimal solution in the population. After completing the update of the particle position and velocity, the global optimal solution in the current particle is selected as the temporary global optimal solution. , if the fitness of the temporary optimal solution is Better than the global optimal solution of the previous generation , then choose As the global optimal solution of the next generation; if the global optimal solution of the previous generation The fitness value is better than the temporary optimal solution The fitness value of The probability of selecting the suboptimal solution as the global optimal solution for the next generation.

[0138] parameter represents the probability of accepting a suboptimal solution. If set too high, the convergence of the algorithm may be impaired; on the contrary, if If it is set too low, the algorithm efficiency may be reduced and the algorithm may be easily trapped in the local optimal solution.

[0139] Specifically, step S5 includes the following steps:

[0140] S5.1, introduce simulated annealing algorithm to update the global optimal solution of the population ,parameter The calculation formula is set as follows:

[0141] (11);

[0142] In the formula, is the current system temperature, is the natural exponential function, is the fitness value of the current global optimal solution of the tth iteration. Too large will result in the probability tends to 1, which means that the algorithm will accept suboptimal solutions with a higher probability. In the later stages of algorithm iteration, if the probability If it is too large, the algorithm will have difficulty converging.

[0143] S5.2, using Eq. To update ,in, It is a parameter that controls the temperature change rate, and its value range is [0,1].

[0144] S5.3, if If is too large, the probability of finding the global optimal solution using this algorithm will be greater. But this will be very time-consuming. On the contrary, if If it is too small, the search process will be faster, but it may fall into local convergence. Therefore, different from the fixed cooling coefficient of the ordinary simulated annealing algorithm , this algorithm uses a linear decreasing method with the overall iteration to update , The calculation formula is:

[0145] (12).

[0146] S5.4, appropriate temperature control can reduce the risk of local optimal solution. Therefore, the present invention improves the temperature control method of the traditional simulated annealing algorithm. After the specified number of iterations, the temperature recovery formula shown in formula (12) is executed:

[0147] (13);

[0148] in, is the specified number of iterations, set to 200, is a constant, set to 0.05.

[0149] As the population continues to evolve, the diversity of the population will gradually decrease, the similarity of particles will gradually increase, and the algorithm will be more likely to fall into a local optimal solution, so the suboptimal solution The choice of is critical. In an ordered population, It is In the iteration particles. If If it is too small, it will be difficult to jump out of the local optimal solution in the later stages of iteration. If is too large, the particles will oscillate severely and it will be difficult to converge in the early iterations. is between 0 and A random number between is the current iteration number.

[0150] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for planning a path for a deep-sea submersible flexible hand grasping operation, characterized in that: The specific steps include: S1, initializing parameters, including: map start and end point information, terrain and threat constraint information, and flexible hand movement characteristic information, including: flexible hand movement time, movement speed, track length, and pitch angle; S2, using random initialization to initialize the original population, and using the data preprocessing method based on the kmeans algorithm to perform data preprocessing operations on the randomly initialized population; S3, using the Q_learning algorithm to obtain the most suitable parameter combination for the particle swarm algorithm operation; S4, calculate whether the number of consecutive unupdated particles is greater than the threshold , if greater than , it means that the particle has not been updated for a long time, and in each dimension, the component of the current particle in that dimension will be replaced by the component of the global optimal solution in that dimension with probability c; S5, using the improved simulated annealing algorithm strategy to update the global optimal solution in the population; S6, repeating steps S3-S5 until the number of iterations is met, and outputting the optimal planning path of the flexible hand; Step S3 specifically includes the following steps: S3.1, initialize the Q table of each particle, find the particle with the best fitness value from the particles, the Q table corresponding to the particle is the current optimal Q table, and the Q table update formula is: (5)； In the formula, , is the state at the tth iteration and t+1th iteration; , are the actions at the tth iteration and t+1th iteration respectively, and are learning rate and discount factor, respectively, with a value range of [0,1], For particles in state Next action The instant rewards you receive, The particle is in the state Take Action The expected Q value of is the maximum expected future reward of all possible actions in the new state; Step S3 also includes the following steps: S3.2, select particles from the initial population after data preprocessing, and predict each particle through the Q_learning algorithm combined with the optimal Q table. After obtaining the prediction result, the global search , random search , comprehensive search and local search Select the behavior that best suits particle search from the four behaviors, and set the reward function to: （6）； （7）； in, After update The fitness value of To update the previous The fitness value of is the maximum number of iterations of the algorithm, and action is the selected behavior; S3.3, use formula (8) and formula (9) to update the position and velocity of the particle. After all particles are updated, select the particle with the best fitness value as the optimal particle, and update the Q table according to the particle's position and performance: （8）； ； （9）； In the formula, S is the number of particles, is the position of the particle at the tth iteration, is the maximum number of iterations, is the inertia weight parameter; and To name the learning factor, we control the positive knowledge obtained from the individual and group historical optimal positions in each iteration. and are two random numbers in [0,1], and are the historical optimal positions of the individual and the group, For the The speed at which the iterations are performed.

2. The deep-sea submersible flexible hand grasping operation path planning method according to claim 1 is characterized in that: Initializing the original population by random initialization in step S2 specifically includes the following steps: S2.1, assuming the path is dimensional vector, i.e. points are fitted, Particles form a community, with the path For example, in describing the particle path In the coordinate community, A particle is represented as a Dimensional vector: （1）； Indicates the path Coordinate point location, Indicates the first Points coordinate.

3. The deep-sea submersible flexible hand grasping operation path planning method according to claim 2 is characterized in that: In step S2, the data preprocessing method based on the kmeans algorithm is used to perform data preprocessing on the randomly initialized population, which specifically includes the following steps: S2.2, The cluster centers are set as , calculate the Euclidean distance from each individual to each cluster center : (2)； in, Represents the number of individuals to plan the path Take coordinates as an example, Indicates Individuals, that is, paths Coordinate point location, Indicates cluster centers, among which Indicates The individual properties, Indicates The cluster center Attributes; S2.3, calculate the cluster center of each cluster: (3)； in, For the The center of the cluster, For the The number of individuals in the cluster; S2.4, calculate the average fitness value of individuals in each cluster to distinguish the differences between different clusters: (4)； in, It is The average fitness value of the cluster, is the fitness function, Is an individual The fitness value of S2.5, sort the average fitness values ​​of all clusters. Taking the minimization problem as an example, all clusters are arranged in ascending order according to the average fitness value, and individuals with smaller fitness values ​​are regarded as dominant individuals.

4. The deep-sea submersible flexible hand grasping operation path planning method according to claim 3 is characterized in that: The specific steps in selecting dominant individuals are as follows: S2.5.1, first, assume The extraction rate for each cluster is the set of clusters sorted in ascending order. , and then proceed Iterations; S2.5.2, starting from the first cluster, select a class cluster in order in each iteration and extract from the cluster The individuals are added to the set as dominant individuals. After each iteration, The value of is decreased by 5; S2.5.3, after completing After iterations, in the set The number of dominant individuals extracted from , the population is insufficient Individuals will be randomly generated, and the final output is the preprocessed population .

5. The deep-sea submersible flexible hand grasping operation path planning method according to claim 1 is characterized in that: Step S4 is specifically as follows: The number of consecutive unupdated particles is ,when Exceeding the threshold When To update the position of the current particle in each dimension , the value of c is 0.7; The calculation formula is: (10)； in, and Respectively The upper and lower bounds of .

6. The deep-sea submersible flexible hand grasping operation path planning method according to claim 1 is characterized in that: Step S5 specifically includes the following steps: S5.1, introduce simulated annealing algorithm to update the global optimal solution of the population ,parameter The calculation formula is set as follows: (11)； In the formula, is the current system temperature, is the natural exponential function, is the fitness value of the current global optimal solution of the tth iteration; S5.2, using Eq. To update ,in, It is the parameter for controlling the temperature change rate, and its value range is [0,1]; S5.3, The calculation formula is: (12)； S5.4, after the specified number of iterations, execute the temperature recovery formula shown in formula (12): (13)； in, is the specified number of iterations, set to 200, is a constant, set to 0.05.

Images